i found this weird behavior and i already don't know if i'm doing something wrong or script lies to me
If i do
r = (255 / 16)
i get 15,937744 with precision of 6 which is already even wrong because 4 should be enough. 255/16 should be 15,9375 and thats it. And to check i did 15,9375 * 16 and it gives back 255 in galaxy and win calculator....
Im not entirely sure, but I recall a very old forum post (by s3rius i think) that said that reals in sc2 are just integers divided by a very large number (four thousand and something), so 15,937744 is probably the closest possible number in sc2 to your actual result. Just my guess, I might be totally wrong.
Just like Maity said, reals/fixed are, as the name suggests, fixed point numbers, no floating point ones. So they have a minimum value they can hold. This value is 0.0625, or 1/16. Your calculation 255/16 should not be a problem you might think, but the limitation might affect the calculation internally somehow, providing false results.
Try to use 255*0.0625 instead, which should be the same results.
€ just trying it, Print(FixedToString((255.0/16.0),6)); was printing 15.937500 as supposed.
As said before, Sc2 is only approximating the number (so do a lot of programs and languages, just with a different degree of accuracy). But if I remember correctly it goes lower than 1/16th. I think it was something like 1/4000th.
Is it necessary for you to have even higher accuracy? Or was it just curiosity?
As said before, Sc2 is only approximating the number (so do a lot of programs and languages, just with a different degree of accuracy). But if I remember correctly it goes lower than 1/16th. I think it was something like 1/4000th.
Is it necessary for you to have even higher accuracy? Or was it just curiosity?
Well i was searching for bug and i found this but i believe i will find workaround
Hello
i found this weird behavior and i already don't know if i'm doing something wrong or script lies to me
If i do r = (255 / 16)
i get 15,937744 with precision of 6 which is already even wrong because 4 should be enough.
255/16 should be 15,9375 and thats it. And to check i did 15,9375 * 16 and it gives back 255 in galaxy and win calculator....
How i can help it?
Im not entirely sure, but I recall a very old forum post (by s3rius i think) that said that reals in sc2 are just integers divided by a very large number (four thousand and something), so 15,937744 is probably the closest possible number in sc2 to your actual result. Just my guess, I might be totally wrong.
@Nerfpl: Go
Just like Maity said, reals/fixed are, as the name suggests, fixed point numbers, no floating point ones. So they have a minimum value they can hold. This value is 0.0625, or 1/16. Your calculation 255/16 should not be a problem you might think, but the limitation might affect the calculation internally somehow, providing false results.
Try to use 255*0.0625 instead, which should be the same results.
€ just trying it, Print(FixedToString((255.0/16.0),6)); was printing 15.937500 as supposed.
hmm, alright i wasnt 100% correct here. I'm using exactly:
I run it and it shows 15,937744 . I'm doing something wrong then?
Edit: Ah now it seems that 100*2,55 = 255,004883 :s
Little more testing shows
200 / 10 = 20
20 / 1 = 20
2 / 0,1 = 19,9804687500
0.1 * 0.1 = 0,0100097656
it almost looks like rain of errors in process :p
As said before, Sc2 is only approximating the number (so do a lot of programs and languages, just with a different degree of accuracy). But if I remember correctly it goes lower than 1/16th. I think it was something like 1/4000th.
Is it necessary for you to have even higher accuracy? Or was it just curiosity?
Well i was searching for bug and i found this but i believe i will find workaround