- Hidden: This just hides the action in the list of actions/functions. You can create function "Test" and set it as hidden, and when you want to use it in a trigger, you won't find it in the action list, even if you search for "Test". You can still access it via custom script though.
- Operator and Custom Script: Haven't done much with those myself, but it looks like they are pretty much the same, except Custom script messes up the GUI completely...
Here's an image where the three functions OperatorWait, ScriptWait and UsualWait are used, which I put the exact same custom script into: "Wait(lp_time, c_timeGame);" (they have time as parameter).
The Operator function (as opposed to the UsualWait function which is called inside the trigger and will execute it's script in it's own function call) just pastes its content script into the trigger. This causes a syntax error, because the trigger does not have the parameter "time" (you could bypass this by using preprocessor-code "#PARAM(time)" instead of "lp_time"). The custom script function seems to do exactly the same, with the exception that you have an extra field in the GUI which doesn't seem to do anything...
- Subfunctions: Also pastes custom script code directly into the trigger, with the addition that you can define function sets which you can execute. The best example here is probably the "If-then-else" Action. It has the subfunctions "If", "Then", and "Else". How they are used can be seen in the custom script section of the action. In the "ifthenelse" case, it checks the result of the "If" subfunctions and then runs either the "Then" or the "Else" subfunctions.
- Restricted: As the tooltip says this probably just marks functions like the "Award Achievement" one, so non-blizzard mapmakers are unable to use them. You probably don't want to flag this one.
But like SoulTaker already said, you probably won't really need any of those except Create Thread. "Native" has to do something with custom libraries I believe, but I have no idea what it does. "Hidden" might be useful for libraries aswell, since you can hide the function to people who use the library. I found Subfunction stuff to be really useful for in-trigger stuff organization and you can do stuff like giving the function multiple actions as "parameter".