Old 10-11-2017, 09:33 PM   #1
tack
Human being with feelings
 
tack's Avatar
 
Join Date: Jan 2014
Location: Ontario, Canada
Posts: 1,619
Default JSFX serialization and "slight loss of precision"

The JSFX docs say:

Quote:
Note when saving the state of variables or memory, they are stored in a more compact 32 bit representation, so a slight precision loss is possible.
When we talk about loss of precision, the implication is that this will affect floating point numbers, not integers. At least this is how I read it, thinking "ok, I'm using integers, not float, so precision isn't relevant."

Imagine my surprise to find that deserializing an array of integers look nothing like what I'd serialized. It turns out that the most significant 8 bits is obliterated. I was already avoiding the MSB (signedness oddities) but now it looks like I must avoid the entire high 8 bits.

Is this indeed the intended design? If so, can I suggest the docs are amended to make this clearer? I bet it will save someone else a multi-hour debugging session.

Thanks!
tack is offline   Reply With Quote
Old 11-01-2017, 03:34 AM   #2
geraintluff
Human being with feelings
 
geraintluff's Avatar
 
Join Date: Nov 2009
Location: mostly inside my own head
Posts: 349
Default

As far as I understand, JSFX uses floating-point numbers internally for everything. However, it's possible to 100% accurately represent integers in floating-point up to a certain limit.

A 32-bit floating-point value can accurately represent a 24-bit integer. A 64-bit floating-point value can hold a 53-bit integer. (In fact, that's the "unsigned" range - it can do negatives as well, so it can hold a 25-bit/54-bit signed integer).

Are you attempting to use/store 32-bit integers? If so, they will be accurate when using 64-bit floats, but you will lose accuracy in the convertion to 32-bit float.

I'm surprised that the 8 most-significant bits are being dropped - I would expect the 8 least-significant bits to be the ones that go (such that 395095664 -> 395095552 or similar).
geraintluff is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 03:57 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.