It is true that Starcode lost one of it's feats since Banks now don't have such a horrid limit anymore.
But it's still encrypting nicely (and less used space surely isn't too bad too) so it does bug me immensely that it doesn't work correctly sometimes.
exactly, and its not only encrypting well but also it makes it very easy to work with banks because it automatically combines all the integers to a string and so on :D
Hmm, i think i've found a glitch or two, i'm using this in a map to avoid bank editing, but the number sometimes doesn't unencrypt properly.
This is basically how it works:
Generate a random variable between 0 and 100
compress the saved values.
Encrypt the saved values with the random variable
hash the saved values with the variable mod 15
encrypt the variable
hash the variable
compress the variable
add the variable to the end of the string after a space.
Now if you reverse that, about 60% of the time i get an enormous number, and i haven't changed the alphabet or anything.
Any ideas?
The checksum part of starcode is better security than any obfuscation or encryption. Plus the compression makes it so that you're having to convert bases on every cleartext value.
The checksum part may be readable, but if its a good 1-way hash then thats all they'll be able to do, read it and NOT change it.
I've run into an instance where <StarCode> Decompress String is generating "Execution took too long". It is being run inside a loop (for each player), but the length of the string to decompress is only 79 characters? Is there anything I can do about this? I am certain that this is the step that is failing.
EDIT: I can duplicate this error at will. I am saving units at 11 characters per unit (2 characters for a unit identifier, 3 for experience, 2 for health, 2 for shields, 2 for energy), plus 2 characters for a unit count, and 4 for a hash (which is removed before the decompress). If I save 6 units (68 characters), everything works fine. As soon as I try to save 7, the save runs fine but it times out trying to load them. More correctly, it times out trying to decompress the 79 character string. I suppose I could make a new key for every 6 units, but I was hoping I could get the strings much larger than that. I'd like to be able to store upwards of 80 units in as few strings as possible.
Well, the problem is that Starcraft maps run on a somewhat slow engine. There's nothing anyone can do about that. At some point your computer just needs too much time to write or read the save codes.
What you can do to reduce the weight on each computer is to add a small Wait command inbetween each decompression:
This is how your trigger should look, I guess. Adding a small wait makes the engine "take a breath" before continuing so it doesn't give up as quickly.
However, at some point the sheer number of units you're trying to save will make the engine reach it's limit in only a single run. At this point you have to split it up further. As I wrote in the readme, I suggest save strings that don't become greater than 100 charaters.
I guess I misread the README, and I deleted it to save map space, so I actually thought the limitation was much higher than 100. Not a huge deal. Six units per string isn't bad. I was able to break up the unit group into multiple strings at 6 units each, enabling me to save more than 6 units. With the lifted bank limitation, I should be able to save a descent number of units. I'll have to try and push the limits once I have the multi-player truly working.
I really appreciate this library and the contribution you've made to the community. Your library makes compression and encryption so much easier, and it really removes any hurdles to implementation. I think STARCODE is just as important now as it ever was, even with the less strict bank limitations. More space means we can store more data, not that we can be beligerantly inefficient with our data storage.
I just wish Blizzard had let us save in true binary in the first place. Giving us limited storage space and then wasting most of it with XML markup and text-based saving is just... well... it's stupid. They're programmers, they should know better.
EDIT: I was able to save 120 units plus a hero in a 3.3kB bank. Not bad. I'm pretty sure 10 players loading 120 units won't work on BNet, but up to 80 might. Couldn't have done it without your library. For the record, saving 120 units takes about a second (I did have to add a wait of 0.0 seconds). Loading takes considerably longer, but I think most of that is unit creation.
If I might offer a suggestion, external hashes would make a nice addition. In my example above, I have 20 keys for unit storage. There's nothing stopping anyone from duplicating a key onto another one. The hash won't notice the difference because it's saved to the string itself and gets copied along with it. So if a player can figure out where his good units are in the bank, he could copy them over on top of his bad units. Even storing a unit count wouldn't prevent this as long as they are replacing one key with another.
I'd like to be able to save a seperate string of unit hashes, but right now STARCODE hashes only work as part of the string. This sort of protection would prevent key duplication, and would be very difficult to circumvent.
Since I need this feature, I'll probably be coding it in the next week or two. If you'd like, I can submit it back to you so that it can be made available to the public.
Now if I could only prevent bank duplication... That's another problem entirely.
Rollback Post to RevisionRollBack
Pocket Warriors - A pokemon-style game with SC2 units and full banking. New demo coming soon!
Instead you can also store an index with each string.
So infront of all the information of your units, you store a number between 1 and 20, and increase this number everytime you write a new code.
Then, when loading, you can check whether the numbers are unique or whether somebody tinkered around with it.
Or you can use a different encryption key for every code, this way you can easily spot dublications are they're exactly the same string, which doesn't happen with different keys.
Found some more. I don't know what it is. If you specify a maximum of 1.000.000.000 (which is totally in the signed int range) you get srewed up results. I am currently testing for the highest working value ;) I need to store the experience for my heroes and therefore need a really big maximum.
I just tried with 1,000,000,000 and it loaded fine.
Did you make sure to define all maximum values correctly?
How many of these (ridiculously high) values are you saving? Maybe it's the amount rather than the maximum.
Thanks for the answer, thats strange. If I change one of the maximum values of your example map to over 231.000.000 (for saving and retieving) its already screwed up.
EDIT: There is only one of those in each player bank.
Change the maximum for the last integer you read in the "Example Usage" trigger from 17500 to 232.000.000 and the maximum for the first integer you read from 17500 to 232.000.000 the final result is 466 and not 18500.
I just tried it again with a fresh downloaded version of the example map you added in the first post.
Hey guys, thanks so much for this library... we're using it with the multi-player version of YABOT to save space in the bank files. We also wanted to make an offline editor for our bank files, so we ported the Starcode library to .NET and I have attached it here if anyone is interested.
It is true that Starcode lost one of it's feats since Banks now don't have such a horrid limit anymore.
But it's still encrypting nicely (and less used space surely isn't too bad too) so it does bug me immensely that it doesn't work correctly sometimes.
@s3rius: Go
exactly, and its not only encrypting well but also it makes it very easy to work with banks because it automatically combines all the integers to a string and so on :D
Hmm, i think i've found a glitch or two, i'm using this in a map to avoid bank editing, but the number sometimes doesn't unencrypt properly.
This is basically how it works: Generate a random variable between 0 and 100 compress the saved values. Encrypt the saved values with the random variable hash the saved values with the variable mod 15 encrypt the variable hash the variable compress the variable add the variable to the end of the string after a space.
Now if you reverse that, about 60% of the time i get an enormous number, and i haven't changed the alphabet or anything. Any ideas?
Posted map.
maybe you should update to 1.4? :)
i think he fixed some bugs since 1.0 and you havent used it much till now, so its a quite good idea to replace it with the newer library
@ikillforeyou: Go
Yes, definitely update to 1.4 (get it from the first post in the topic)! You probably fell victim to the Encryption bug which I fixed in 1.3 or 1.4.
The checksum part of starcode is better security than any obfuscation or encryption. Plus the compression makes it so that you're having to convert bases on every cleartext value. The checksum part may be readable, but if its a good 1-way hash then thats all they'll be able to do, read it and NOT change it.
I've run into an instance where <StarCode> Decompress String is generating "Execution took too long". It is being run inside a loop (for each player), but the length of the string to decompress is only 79 characters? Is there anything I can do about this? I am certain that this is the step that is failing.
EDIT: I can duplicate this error at will. I am saving units at 11 characters per unit (2 characters for a unit identifier, 3 for experience, 2 for health, 2 for shields, 2 for energy), plus 2 characters for a unit count, and 4 for a hash (which is removed before the decompress). If I save 6 units (68 characters), everything works fine. As soon as I try to save 7, the save runs fine but it times out trying to load them. More correctly, it times out trying to decompress the 79 character string. I suppose I could make a new key for every 6 units, but I was hoping I could get the strings much larger than that. I'd like to be able to store upwards of 80 units in as few strings as possible.
@jaminv: Go
Well, the problem is that Starcraft maps run on a somewhat slow engine. There's nothing anyone can do about that. At some point your computer just needs too much time to write or read the save codes.
What you can do to reduce the weight on each computer is to add a small Wait command inbetween each decompression:
This is how your trigger should look, I guess. Adding a small wait makes the engine "take a breath" before continuing so it doesn't give up as quickly.
However, at some point the sheer number of units you're trying to save will make the engine reach it's limit in only a single run. At this point you have to split it up further. As I wrote in the readme, I suggest save strings that don't become greater than 100 charaters.
I guess I misread the README, and I deleted it to save map space, so I actually thought the limitation was much higher than 100. Not a huge deal. Six units per string isn't bad. I was able to break up the unit group into multiple strings at 6 units each, enabling me to save more than 6 units. With the lifted bank limitation, I should be able to save a descent number of units. I'll have to try and push the limits once I have the multi-player truly working.
I really appreciate this library and the contribution you've made to the community. Your library makes compression and encryption so much easier, and it really removes any hurdles to implementation. I think STARCODE is just as important now as it ever was, even with the less strict bank limitations. More space means we can store more data, not that we can be beligerantly inefficient with our data storage.
I just wish Blizzard had let us save in true binary in the first place. Giving us limited storage space and then wasting most of it with XML markup and text-based saving is just... well... it's stupid. They're programmers, they should know better.
EDIT: I was able to save 120 units plus a hero in a 3.3kB bank. Not bad. I'm pretty sure 10 players loading 120 units won't work on BNet, but up to 80 might. Couldn't have done it without your library. For the record, saving 120 units takes about a second (I did have to add a wait of 0.0 seconds). Loading takes considerably longer, but I think most of that is unit creation.
If I might offer a suggestion, external hashes would make a nice addition. In my example above, I have 20 keys for unit storage. There's nothing stopping anyone from duplicating a key onto another one. The hash won't notice the difference because it's saved to the string itself and gets copied along with it. So if a player can figure out where his good units are in the bank, he could copy them over on top of his bad units. Even storing a unit count wouldn't prevent this as long as they are replacing one key with another.
I'd like to be able to save a seperate string of unit hashes, but right now STARCODE hashes only work as part of the string. This sort of protection would prevent key duplication, and would be very difficult to circumvent.
Since I need this feature, I'll probably be coding it in the next week or two. If you'd like, I can submit it back to you so that it can be made available to the public.
Now if I could only prevent bank duplication... That's another problem entirely.
Preventing bank duplication would work if we could retrieve some sort of player information as a string :(
@jaminv: Go
Instead you can also store an index with each string.
So infront of all the information of your units, you store a number between 1 and 20, and increase this number everytime you write a new code.
Then, when loading, you can check whether the numbers are unique or whether somebody tinkered around with it.
Or you can use a different encryption key for every code, this way you can easily spot dublications are they're exactly the same string, which doesn't happen with different keys.
Thanks for the library. I was working on my own, but this works just fine! Makes saving rpg character propteries in bank files sooo easy :)
Found a bug:
Variable - Set Code = (<Starcode> Encrypt String((<Starcode> Compress String((<Starcode> Get Code()))), "key"))
If you nest the encrypt and the compress function, the result is screwed up.
Found some more. I don't know what it is. If you specify a maximum of 1.000.000.000 (which is totally in the signed int range) you get srewed up results. I am currently testing for the highest working value ;) I need to store the experience for my heroes and therefore need a really big maximum.
The maximum value is somewhere between 231.000.000 and 232.000.000! Is this something that can be fixed?
I just tried with 1,000,000,000 and it loaded fine.
Did you make sure to define all maximum values correctly?
How many of these (ridiculously high) values are you saving? Maybe it's the amount rather than the maximum.
Thanks for the answer, thats strange. If I change one of the maximum values of your example map to over 231.000.000 (for saving and retieving) its already screwed up.
EDIT: There is only one of those in each player bank.
Change the maximum for the last integer you read in the "Example Usage" trigger from 17500 to 232.000.000 and the maximum for the first integer you read from 17500 to 232.000.000 the final result is 466 and not 18500.
I just tried it again with a fresh downloaded version of the example map you added in the first post.
Hey guys, thanks so much for this library... we're using it with the multi-player version of YABOT to save space in the bank files. We also wanted to make an offline editor for our bank files, so we ported the Starcode library to .NET and I have attached it here if anyone is interested.