Interesting results of your tests, because that means that capability (pass, store, return arrayrefs and structrefs) is being added with LotV, as it is clearly indicated as possible based on the galaxy unit tests. ImperialGood does have a point that such functionality may end up simply being marked as restricted, but one can hope :D
Agreed on data tables, they are best used for when the number of objects is unknown, basically best to treat them as a kind of heap. And I too end up just using mostly global arrays of structs, since I tend to define the size with constants, so it is trivial to alter the gameplay or structure of the game on a large scale (8 player -> 10 player, or 2 talent trees -> 4, and the like). Also, even if the number of a object is unknown, one could still just allocate a sparse array and do hashing for storage. Based on your discovery about 4 or so patches ago, where they massively expanded the memory space, we have a enormous amount of variables one can use (All told, my map, with all dependencies and everything, compiles to like 1% global memory? Not even sure it is possible to get a sizable percentage of the total unless working on a project such as yours).
Well, I agree that global struct arrays are better when you know exactly how many objects you need to create. I would absolutely use them for something with an obvious fixed amount, like defining a global struct array with one struct for each player, to contain easily accessible properties that you want to associate with them.
But I personally find it pretty unmanageable when the number of things is large and/or expected to change over development. Like structs representing heroes and items for the construction and running of a hero selection and item shop. It's a pain when every time you add a hero or item you have to go add 1 to the count.
I also don't think that the slowdown is really worth noting. It's pretty insignificant, as long as your coding is correct. Focusing on reducing the number of checks and runs of triggers is much better. I also find that using the data table makes code a lot more readable and modular. Because you have to define all these getters and setters, the code is just the right amount of verbose for me. I also tend to build any "objects" that use the data table as separate code modules by putting them in separate script "files".
One downside we haven't mentioned is the inability to store function/array pointers in the data table. This really bums me out, I love function pointers. I once worked in a map where there were chat commands and each command was, in the code, part of a really long if/else if/else if/.../else tree. Horrible way of coding. When I wanted to create my own chat command system I created a global struct array to contain the commands and mapped from the entered message to an index in this array, then invoked a function pointer within the corresponding struct.
A workaround for your problem would be to use Triggers instead of function pointers. They are fully supported in GUI and data tables.
Triggers are not as effective, however, they can be used to simulate functions. Parameters can be passed by setting a global variable directly before the trigger call and then assigning that global variable to a local variable inside the trigger before any other operations. (This is actually what Blizzard does when checking the "Create Thread" flag for a custom action)
Its not perfect but thats the technique I'm using since a long time (GUI doesnt support funcrefs at all) and it works really well. Almost all my systems are based on this technique (Including chat messages. :D )
About the whole data tables vs global struct discussion, I think its really just a matter of personal preferrence. I prefer wasting a little bit of global memory space over the pitfalls of data table programming in most cases.
Also, here is something to think about:
Its not clear how much memory the data table itself consumes when using it extensively.
Lets say I have a struct with 10 members of mixed types, mostly booleans and ints. If i create a global array of 1000 structs and only use 700 of them, I wasted memory for 300 of those structs. However, if I created the same construct using data tables the 700 existing entries may consume more internal memory than my 1000 structs, as the data table needs to store way more internal information (Mapping string keys to data). In this case, an additional 10*700 = 7000 strings need to be saved as keys to access the stored data (aka "struct members").
Assuming that ints and booleans each require 4 bytes of memory in Galaxy, The 1000 global structs would require approximately 1000*10*4 = 40000 bytes of memory, or approx 40KB.
With an average key length of 10 chars and 1 byte of required memory per character within strings, the data table would require at least (I say "at least" because it may be more based on string encoding) 700*10*10 = 70000 bytes (70KB) of memory just for storing the identifiers, plus the additional memory required to store the actual values. So even though we stored less information and did not "waste" any memory we still end up with more internal memory consumption overall.
Even though this memory consumption is not visible in the debugger it still exists internally and most likely increases memory usage of the map. Is it a big deal and does it matter? Probably not! But same goes for unused structs. It is something to keep in mind if talking about efficiency.
Assuming that ints and booleans each require 4 bytes of memory in Galaxy, The 1000 global structs would require approximately 1000*10*4 = 40000 bytes of memory, or approx 40KB.
With an average key length of 10 chars and 1 byte of required memory per character within strings, the data table would require at least (I say "at least" because it may be more based on string encoding) 700*10*10 = 70000 bytes (70KB) of memory just for storing the identifiers, plus the additional memory required to store the actual values. So even though we stored less information and did not "waste" any memory we still end up with more internal memory consumption overall.
Even though this memory consumption is not visible in the debugger it still exists internally and most likely increases memory usage of the map. Is it a big deal and does it matter? Probably not! But same goes for unused structs. It is something to keep in mind if talking about efficiency.
Seeing how SC2 uses at least 1 GB of memory, a couple of kilobytes is nothing. In WC3 people thought nothing of allocating lots of 32 kilobyte arrays all over the place (although few ever reached that size).
The main concern about data tables is their scalability. In WC3 the hashtable system (which served a similar purpose) suffered from performance degradation with excessive element storage in it. Do data tables resize their bucket array or just deal with the collisions?
A workaround for your problem would be to use Triggers instead of function pointers. They are fully supported in GUI and data tables.
Triggers are not as effective, however, they can be used to simulate functions. Parameters can be passed by setting a global variable directly before the trigger call and then assigning that global variable to a local variable inside the trigger before any other operations. (This is actually what Blizzard does when checking the "Create Thread" flag for a custom action)
Its not perfect but thats the technique I'm using since a long time (GUI doesnt support funcrefs at all) and it works really well. Almost all my systems are based on this technique (Including chat messages. :D )
Funny enough, that system is being directly incorporated with LotV. There are new natives that let you set Data Table entries, that bind the entry to the trigger. This is meant to fully replicate the default trigger scheme. So you would be able to use the new set Native (i'll see if I can dig up the entries in the Natives.galaxy in Storm) to set various variables, then do a Send Generic event and the triggers that receive the Generic event can then use the corresponding native to retrieve the values set. So you will get to skip the global -> local variable assignment. Basically, with something like Dialog Used, you have the default retrieval functions (Triggering Player, Used Dialog Item, etc.) and these new natives will allow for a generic abstract of that, to compliment Generic events. So you will finally be able to create your own triggers of a sorts. I already use Generic events fairly often, since they are much more effective in many cases (since you are now able to have many threads fire off a single event of your own making, instead of having to call functions in many places or what not).
Rollback Post to RevisionRollBack
To post a comment, please login or register a new account.
@Mille25: Go
Interesting results of your tests, because that means that capability (pass, store, return arrayrefs and structrefs) is being added with LotV, as it is clearly indicated as possible based on the galaxy unit tests. ImperialGood does have a point that such functionality may end up simply being marked as restricted, but one can hope :D
Agreed on data tables, they are best used for when the number of objects is unknown, basically best to treat them as a kind of heap. And I too end up just using mostly global arrays of structs, since I tend to define the size with constants, so it is trivial to alter the gameplay or structure of the game on a large scale (8 player -> 10 player, or 2 talent trees -> 4, and the like). Also, even if the number of a object is unknown, one could still just allocate a sparse array and do hashing for storage. Based on your discovery about 4 or so patches ago, where they massively expanded the memory space, we have a enormous amount of variables one can use (All told, my map, with all dependencies and everything, compiles to like 1% global memory? Not even sure it is possible to get a sizable percentage of the total unless working on a project such as yours).
@Mille25: Go
Well, I agree that global struct arrays are better when you know exactly how many objects you need to create. I would absolutely use them for something with an obvious fixed amount, like defining a global struct array with one struct for each player, to contain easily accessible properties that you want to associate with them.
But I personally find it pretty unmanageable when the number of things is large and/or expected to change over development. Like structs representing heroes and items for the construction and running of a hero selection and item shop. It's a pain when every time you add a hero or item you have to go add 1 to the count.
I also don't think that the slowdown is really worth noting. It's pretty insignificant, as long as your coding is correct. Focusing on reducing the number of checks and runs of triggers is much better. I also find that using the data table makes code a lot more readable and modular. Because you have to define all these getters and setters, the code is just the right amount of verbose for me. I also tend to build any "objects" that use the data table as separate code modules by putting them in separate script "files".
One downside we haven't mentioned is the inability to store function/array pointers in the data table. This really bums me out, I love function pointers. I once worked in a map where there were chat commands and each command was, in the code, part of a really long if/else if/else if/.../else tree. Horrible way of coding. When I wanted to create my own chat command system I created a global struct array to contain the commands and mapped from the entered message to an index in this array, then invoked a function pointer within the corresponding struct.
@MasterWrath: Go
A workaround for your problem would be to use Triggers instead of function pointers. They are fully supported in GUI and data tables.
Triggers are not as effective, however, they can be used to simulate functions. Parameters can be passed by setting a global variable directly before the trigger call and then assigning that global variable to a local variable inside the trigger before any other operations. (This is actually what Blizzard does when checking the "Create Thread" flag for a custom action)
Its not perfect but thats the technique I'm using since a long time (GUI doesnt support funcrefs at all) and it works really well. Almost all my systems are based on this technique (Including chat messages. :D )
About the whole data tables vs global struct discussion, I think its really just a matter of personal preferrence. I prefer wasting a little bit of global memory space over the pitfalls of data table programming in most cases.
Also, here is something to think about:
Its not clear how much memory the data table itself consumes when using it extensively.
Lets say I have a struct with 10 members of mixed types, mostly booleans and ints. If i create a global array of 1000 structs and only use 700 of them, I wasted memory for 300 of those structs. However, if I created the same construct using data tables the 700 existing entries may consume more internal memory than my 1000 structs, as the data table needs to store way more internal information (Mapping string keys to data). In this case, an additional 10*700 = 7000 strings need to be saved as keys to access the stored data (aka "struct members").
Assuming that ints and booleans each require 4 bytes of memory in Galaxy, The 1000 global structs would require approximately 1000*10*4 = 40000 bytes of memory, or approx 40KB.
With an average key length of 10 chars and 1 byte of required memory per character within strings, the data table would require at least (I say "at least" because it may be more based on string encoding) 700*10*10 = 70000 bytes (70KB) of memory just for storing the identifiers, plus the additional memory required to store the actual values. So even though we stored less information and did not "waste" any memory we still end up with more internal memory consumption overall.
Even though this memory consumption is not visible in the debugger it still exists internally and most likely increases memory usage of the map. Is it a big deal and does it matter? Probably not! But same goes for unused structs. It is something to keep in mind if talking about efficiency.
Seeing how SC2 uses at least 1 GB of memory, a couple of kilobytes is nothing. In WC3 people thought nothing of allocating lots of 32 kilobyte arrays all over the place (although few ever reached that size).
The main concern about data tables is their scalability. In WC3 the hashtable system (which served a similar purpose) suffered from performance degradation with excessive element storage in it. Do data tables resize their bucket array or just deal with the collisions?
Funny enough, that system is being directly incorporated with LotV. There are new natives that let you set Data Table entries, that bind the entry to the trigger. This is meant to fully replicate the default trigger scheme. So you would be able to use the new set Native (i'll see if I can dig up the entries in the Natives.galaxy in Storm) to set various variables, then do a Send Generic event and the triggers that receive the Generic event can then use the corresponding native to retrieve the values set. So you will get to skip the global -> local variable assignment. Basically, with something like Dialog Used, you have the default retrieval functions (Triggering Player, Used Dialog Item, etc.) and these new natives will allow for a generic abstract of that, to compliment Generic events. So you will finally be able to create your own triggers of a sorts. I already use Generic events fairly often, since they are much more effective in many cases (since you are now able to have many threads fire off a single event of your own making, instead of having to call functions in many places or what not).