Hi, i want to store about 2.5k int values in bank file. I am using user data in triggers to save and load into bank file. Bank file size ~ 45kb and it stores just null values. So, it will be much bigger with real data. I plan to use it with 8 players in game. So, in game we have 8 bank files. Should i risk to use such big bank files or better to reduce it? Bank file attached.
Hey, I'm probably not the best person to try to answer this, but I'm currently working on something similar. I'm working on a project where I'd like to keep track of the rank of EVERY player that has played. To do that, I'm simply going to store an int value (rank) in a key (key = player handle). On map load, I will update every key in that section in every bank. So far, I've stress-tested the bank with 3 players (online bnet) saving and loading 2,500 keys. I arbitrarily chose 2,500 as it is probably more than the total number of players we'll ever see on one server for this map. I saved and loaded the key count, displaying the loaded value as it loaded each key.
I still need to do a proper stress-test - I'm planning on having 8-10 players in the map with up to 2,500 keys and another bank section for actual player data. I will try to remember to get back to you when I've done a proper test.
Here's an example screenshot, in case anyone is curious about how I'm planning to load the keys (without knowing what the key name is; e.g. the player-handle).
Eventually the map game will fail to load and everyone will be ejected to the post game screen. There is (last confirmed in 2015) a limit to the total amount of bank data that can be synchronized during loading. If this limit is exceeded then the game will fail to load, probably due to timing out due to the volume of data to synchronize. Since each player's bank has to be synchronized this means that the maximum size of the player bank is {bank synchronization limit / player number}, so the more players there are with banks the lower the size limit (assuming all player banks are processed/created the same).
As such your idea is not feasible. It may work initially but eventually it will render the map unplayable, unless everyone deletes their banks which defeats your initial goal anyway.
As for the first question (which is 2 years old)... 2,500 int values needs at least 10,000 bytes (10 kilobytes). Obtaining this efficiency inside banks is not possible, however one can store ints a lot more efficiently than the native int storage functionality. One can pack the ints into a byte buffer of sorts, encode this byte buffer into a base 64 string and then store this base 64 string inside the bank as a string entry. If one makes the base 64 string long enough then the XML overhead can be considered trivial, most of the bank filesize is data. Base 64 is 6 bits per character so a 32 bit int is thus 5 + 1/3 characters. To prevent wasted bits from the fractional character one can pack 3 ints, 96 bits, into 16 characters. Each of these base64 characters is stored as a 1 byte ASCII character so 12 bytes worth of int turn into 16 bytes of file. Hence storing 2,500 int values would result in a bank of approximately 14 kilobytes. However these are uncompressed int values, if one was to compress the buffer with run length encoding one might find as little as 4 kilobytes are needed.
The map BlzEnforcement uses banks that are at least 300 kilobytes for 4 players (1200 kilobytes of banks in total). If one assumes this is the limit (actual limit is probably larger) and 12 players are used that would give 100 kilobytes per player bank. As such it is completely fine and even trivial to store 2,500 int values in a bank.
You're amazing, ImperialGood. Thanks for this reply. I know this topic is quite old, but I've found it fairly difficult to keep track of the changes made to the SC2 editor and thought the best approach was to determine current limits for myself.
I've been working on a custom encryption scheme for banks. It's a painful process as there seem to be a lot of limits in Galaxy with sparse or non-existent documentation. You got me looking to see if something already exists for this - I found Starcode, but it seems it didn't survive the switch to curse and may be deprecated anyway.
I could probably drop my player estimate to 2000, which would take me under the 1200kb size per player (10 players). Additionally, this doesn't take into account RLE possibilities. All in all, this seems like a viable solution. I will work on this when I get time and see about adding RLE. The only thing I'm worried about is if syncing up 12MB on map load will cause mass drops, but I guess we'll see.
Thanks again for your detailed suggestion. If I get something working, I will upload for community use.
EDIT: Wanted to mention the savings this would net me - 2500 (14 + 67) = 202,500b per player. 14 is player handle length. Net savings, in this example, of 71kb.
Actually, the more I think about it, the more I realize that this method is unrealistic for my goals. The primary reason I wanted to keep track of every player was so I would know if someone deleted their bank. While your suggestion solves the space issue, decoding the saved data for all 10 players and cross checking with every other player would no longer be feasible. I always expected it to take a little bit of time at map start-up, but this would significantly increase that time and probably cause SC2 to hang and crash for many players.
This method is, however, totally feasible for a "Top 100" listing, and I plan to use it for that. It's also a great idea for anyone storing a lot of data, particularly at low player counts. Sadly, handling player's who delete their banks is still a problem.
I've been working on a custom encryption scheme for banks. It's a painful process as there seem to be a lot of limits in Galaxy with sparse or non-existent documentation. You got me looking to see if something already exists for this - I found Starcode, but it seems it didn't survive the switch to curse and may be deprecated anyway.
I implemented AES encryption into SC2 so it is not really that limiting. It is currently in use by Undead Assault 3 2015 and Undead Assault Chronicals. It was removed from this site due to file limits however here is a temporary link to it...
Sadly, handling player's who delete their banks is still a problem.
You should not be trying to do that. Banks are subject to data loss so players might not purposely delete their banks but end up losing them unfortunately.
Be aware that it is possible to poison such bank systems. Someone like myself, but with malicious intent, could engineer a poison bank file with fake top 100 leaderboard entries which push out all the genuine ones.
Hi, i want to store about 2.5k int values in bank file. I am using user data in triggers to save and load into bank file. Bank file size ~ 45kb and it stores just null values. So, it will be much bigger with real data. I plan to use it with 8 players in game. So, in game we have 8 bank files. Should i risk to use such big bank files or better to reduce it? Bank file attached.
http://www.youtube.com/user/RussianMapster
Hey, I'm probably not the best person to try to answer this, but I'm currently working on something similar. I'm working on a project where I'd like to keep track of the rank of EVERY player that has played. To do that, I'm simply going to store an int value (rank) in a key (key = player handle). On map load, I will update every key in that section in every bank. So far, I've stress-tested the bank with 3 players (online bnet) saving and loading 2,500 keys. I arbitrarily chose 2,500 as it is probably more than the total number of players we'll ever see on one server for this map. I saved and loaded the key count, displaying the loaded value as it loaded each key.
I still need to do a proper stress-test - I'm planning on having 8-10 players in the map with up to 2,500 keys and another bank section for actual player data. I will try to remember to get back to you when I've done a proper test.
Here's an example screenshot, in case anyone is curious about how I'm planning to load the keys (without knowing what the key name is; e.g. the player-handle).
Eventually the map game will fail to load and everyone will be ejected to the post game screen. There is (last confirmed in 2015) a limit to the total amount of bank data that can be synchronized during loading. If this limit is exceeded then the game will fail to load, probably due to timing out due to the volume of data to synchronize. Since each player's bank has to be synchronized this means that the maximum size of the player bank is {bank synchronization limit / player number}, so the more players there are with banks the lower the size limit (assuming all player banks are processed/created the same).
As such your idea is not feasible. It may work initially but eventually it will render the map unplayable, unless everyone deletes their banks which defeats your initial goal anyway.
As for the first question (which is 2 years old)...
2,500 int values needs at least 10,000 bytes (10 kilobytes). Obtaining this efficiency inside banks is not possible, however one can store ints a lot more efficiently than the native int storage functionality. One can pack the ints into a byte buffer of sorts, encode this byte buffer into a base 64 string and then store this base 64 string inside the bank as a string entry. If one makes the base 64 string long enough then the XML overhead can be considered trivial, most of the bank filesize is data. Base 64 is 6 bits per character so a 32 bit int is thus 5 + 1/3 characters. To prevent wasted bits from the fractional character one can pack 3 ints, 96 bits, into 16 characters. Each of these base64 characters is stored as a 1 byte ASCII character so 12 bytes worth of int turn into 16 bytes of file. Hence storing 2,500 int values would result in a bank of approximately 14 kilobytes. However these are uncompressed int values, if one was to compress the buffer with run length encoding one might find as little as 4 kilobytes are needed.
The map BlzEnforcement uses banks that are at least 300 kilobytes for 4 players (1200 kilobytes of banks in total). If one assumes this is the limit (actual limit is probably larger) and 12 players are used that would give 100 kilobytes per player bank. As such it is completely fine and even trivial to store 2,500 int values in a bank.
You're amazing, ImperialGood. Thanks for this reply. I know this topic is quite old, but I've found it fairly difficult to keep track of the changes made to the SC2 editor and thought the best approach was to determine current limits for myself.
I've been working on a custom encryption scheme for banks. It's a painful process as there seem to be a lot of limits in Galaxy with sparse or non-existent documentation. You got me looking to see if something already exists for this - I found Starcode, but it seems it didn't survive the switch to curse and may be deprecated anyway.
I think I've got a decent handle on how to implement your suggestion. Basically, I would be using a key and value system. It looks like player handles started at 6 digits for the account id (101 Guide to Player Handles), but current accounts seem to be of length 7, so I will reserve 8 ints for the player handle + 1 for the value; that's 48 characters per player. According to //www.sc2mapster.com/forums/resources/tutorials/179542-trigger-bank-facts" rel="noopener nofollow" target="_blank">https://www.sc2mapster.com/forums/resources/tutorials/179542-trigger-bank-facts">this thread, max usable string length for a key is 775. Assuming we do not use any delimiters and maximize storage capacity by relying on string-length we've got the following:
# of Player Entries: 2500
Characters per player: 48
Bytes Required for Section: 35 + SectionName (single value) = 36
Bytes Per Key: 67 + name
Max Key Size: 775
16 Players per Key: 768 characters per key
157 Keys Required:
Total Space Required: (157*768) + [(10*68) + (90*69) + (57*70)] + 36
Size per bank: 131,492 b
Desmos Graph
I could probably drop my player estimate to 2000, which would take me under the 1200kb size per player (10 players). Additionally, this doesn't take into account RLE possibilities. All in all, this seems like a viable solution. I will work on this when I get time and see about adding RLE. The only thing I'm worried about is if syncing up 12MB on map load will cause mass drops, but I guess we'll see.
Thanks again for your detailed suggestion. If I get something working, I will upload for community use.
EDIT: Wanted to mention the savings this would net me - 2500 (14 + 67) = 202,500b per player. 14 is player handle length. Net savings, in this example, of 71kb.
Actually, the more I think about it, the more I realize that this method is unrealistic for my goals. The primary reason I wanted to keep track of every player was so I would know if someone deleted their bank. While your suggestion solves the space issue, decoding the saved data for all 10 players and cross checking with every other player would no longer be feasible. I always expected it to take a little bit of time at map start-up, but this would significantly increase that time and probably cause SC2 to hang and crash for many players.
This method is, however, totally feasible for a "Top 100" listing, and I plan to use it for that. It's also a great idea for anyone storing a lot of data, particularly at low player counts. Sadly, handling player's who delete their banks is still a problem.
I implemented AES encryption into SC2 so it is not really that limiting. It is currently in use by Undead Assault 3 2015 and Undead Assault Chronicals. It was removed from this site due to file limits however here is a temporary link to it...
https://www.dropbox.com/s/x09478z8hqbww2y/Imperial Libraries 1.1.SC2Map?dl=0
You should not be trying to do that. Banks are subject to data loss so players might not purposely delete their banks but end up losing them unfortunately.
Be aware that it is possible to poison such bank systems. Someone like myself, but with malicious intent, could engineer a poison bank file with fake top 100 leaderboard entries which push out all the genuine ones.