numerical display accuracy @gfx

[bezusheist]

i don't know if i'm doing something wrong or what but it seems the values displayed @gfx can be a little "off" from what they "should" be.
this meaning that if i have a value of "0.00001" and convert that to dB (20*log10(0.00001/20)) i should get -100 dBFS. which i do in the slider section. but in the @gfx section i get -99.99...to inf.
i attached a link of a screenshot with a sample JS code and the discrepancy i am talking about.

http://imgur.com/a/g5YMq

also, i'm pretty sure i'm doing something wrong w/ gfx_setfont because i can not actually set the font (i type different font names but still the same font displayed).
could someone tell me the correct way ? thanks.

[geraintluff]

I haven't looked into how it works for floating-point, but sprintf() doesn't do nearest-value rounding for integers ("%i"), it does truncation. I end up doing the rounding myself:

x_rounded = floor(x + 0.5);
gfx_printf(#x_string, x_rounded);

But I haven't actually tried it myself, but if you're looking for lots of decimal points, it might work to do something similar, multiplying/dividing by an appropriate constant:

rounding_multiple = pow(10, 8);
x_rounded = floor(x*rounding_multiple + 0.5)/rounding_multiple;

For the font: you've used the variable Futura (which is just the value 0) - you should use the string "Futura".

[bezusheist]

thanks for the explanation. it has cleared up the issue with the numerical values, but i am still a little foggy about setting the font. i guess i don't know what the "string" Futura is referring to. the section about that on the JSFX page is still confusing to me.

[Lokasenna]

You're giving it a variable named Futura that doesn't contain anything. gfx_setfont wants the font name as a text string:

gfx_setfont(1, "Futura", 16)

[bezusheist]

ah...ok. thanks for clearing that up.

[bezusheist]

ok, so manually rounding or truncating will work for many situations, but i ran into something now that has me stumped...
please take a look at my code and the attached /gif of what happens...
increments of "1" seem fine...increments of "0.01" don't work as good.
i really don't understand why i can't get this right...
help ?

options:no_meter

@init

ext_nodenorm = 1;

@gfx 200 200

gfx_getchar();

mouse_cap == 1 ?
(
in = out;
out = mouse_y;
out > in ?
( 
b -= 1;
);
out < in ?
(
b += 1;
);
);

mouse_cap == 9 ?
(
in = out;
out = mouse_y;
out > in ?
( 
b -= 0.01;
);
out < in ?
(
b += 0.01;
);
);
mouse_cap == 2 ?
(
b = 0;
);

gain = 10^(b/20);
out = mouse_y;

gfx_setfont(1,"Arial",24);
gfx_r = 1;gfx_g = 1;gfx_b = 1;gfx_a = 1;
gfx_x = 20;gfx_y = 20;
gfx_drawnumber(b,2);
gfx_x = 20;gfx_y = 50;
gfx_drawnumber(gain,8);

[Tale]

Here is an older thread about rounding issues, which might be of interest:

https://forum.cockos.com/showthread.php?t=168401

[bezusheist]

Originally Posted by Tale

Here is an older thread about rounding issues, which might be of interest:

https://forum.cockos.com/showthread.php?t=168401

Thanks for the link/headache
This behavior certainly makes it hard for those just learning how to code..
After exhaustive testing, I've come to the conclusion that base 10 sucks and I need to use base 2 for best results.
Ie...instead of incrementing by "0.01" and rounding, I use "2^-8" , which gets me close enough with little to no "error".
(Consistency is key)

edit: for real world (fader/gain) use, I honestly don't care how "accurate" the increments are, as long as I get some "fine" control when needed. This is more about learning/understanding how the code/jsfx works.