• 🏆 Texturing Contest #33 is OPEN! Contestants must re-texture a SD unit model found in-game (Warcraft 3 Classic), recreating the unit into a peaceful NPC version. 🔗Click here to enter!
  • It's time for the first HD Modeling Contest of 2024. Join the theme discussion for Hive's HD Modeling Contest #6! Click here to post your idea!

Precise Numbers

Status
Not open for further replies.
Level 7
Joined
Jun 16, 2008
Messages
253
Anyone else have this issue when they're trying to do very precise maths in the program, and for some reason Starcraft 2 has forgotten how to add two and two?

For example, I can get it to store 0.008 and 0.008 and 0.016, but if I try to multiply this values together the program decides to return a value of 0.0000. I assumed it was was rounding up numbers, and just doesn't deal well with decimal points.

So I tried to be tricky by mutiplying each by 10000, and then dividing it appropriately at the end result, and it worked... sort of. It returned a really strange result, that does not agree with my calculator or rational math. I've literally gotten NEGATIVE results in the end calculation.

I've noticed this sort of thing before, when saving 'radio frequencies', 88.6 will turn into 87.578986 or something weird.

Is this anything to do with a minimum value thing? I know that the game itself operates on like a 0.685 (can't remember exactly) periodic rate, so is it that sort of minimum where's it's trying to round my figures to a multiple of some minimum number?

Please help. It's messing up my physics and playability big time. My guys are having trouble hitting planes 5 metres away.
 
Level 7
Joined
Jun 16, 2008
Messages
253
Oh, really? That's handy to know. :D But the above calculation only comes out to about 9 decimal places.

Some jokers answered my question on the offic forums, apparently it's some dumb stupid computer thing, that not all real numbers can be properly represented by a binary system, so it's gives back the closest thing it can.

It sounds completely incredibly gay to me, but there ya go. :D
 

Dr Super Good

Spell Reviewer
Level 64
Joined
Jan 18, 2005
Messages
27,198
Learn computer science.

Computers are finite state machines and thus can only compute numbers of a finite size. SC2 forgoes the range that floating point numbers give (which was what WC3 used) but instead goes for accuracy and speed.

1+1 is 2 using fixed points
1+1 does not yield 2 (neithor did the 1s yield 1) with floating point, instead rounding is used to get the value. For large numbers this means that lower value digits are inaccurate.
 

Dr Super Good

Spell Reviewer
Level 64
Joined
Jan 18, 2005
Messages
27,198
BlueBerryWizard, I advise you complain to the whole international community about that, it is the case in every programming language I know.

Integer division is meant to work like that. If you want to use it as a fixed you should probably store it as a fixed in the first place.

I do admit though that the language is stupidly strict in some ways. You can not map int to fixed directly, nor do opperators on mixed types. Almost every programming language does this automatically for you.

It could be worse though. It atleast has booleans (unlike languages like C89).
 
Status
Not open for further replies.
Top