discuss@lists.openscad.org

OpenSCAD general discussion Mailing-list

View all threads

Inconsistent conversion of floating number to string at 7th significant digit

C
ctchin
Thu, Nov 12, 2015 5:17 PM

doug.moen wrote

... the export to ASCII STL, and the export to
AMF. In the export case, severely truncating each float to 6 significant
digits often damages the mesh, so that OpenSCAD can't read STL files that
it itself has generated.

Oh... I wasn't aware of that issue (do apologize I haven't find time to go
over all old threads.)

But I still... question if it's not a bit of a red herring.  If OpenSCAD is
really designed for and
by 3D printing hobbyists and professionals, then does it serve meaningful
purpose to reproduce
better than 0.1PPM precision?  I am not asserting it is red herring, just
curious if someone really
come upon a real world issue arising from such level of inaccuracy.

As for OpenSCAD can't read STL it generates... again is that a high priority
issue?  Unlike
File->Save and File->Load, I can tolerate, even expect some loss when I
Export... and than
Import... some data, this is true in many apps, even MS Office.  Over the
year (not quite 2
years yet) I've designed some moderately elaborate shapes and never had
Slic3r reject a
design that OpenSCAD deemed valid.

I imagine other apps/devices (laser cutter?) similarly have been accepting
the STL's generated
by OpenSCAD.  IEEE floats are imperfect representation of physical world.
And text string
representation are further corruption of accuracy of IEEE floats.  That's
unavoidable.

We are quite happy to accept 20-side polygon as a circle, 100-ish-faces
polyhedron for a sphere.
There's easily inaccuracy of 1% in almost all of our designs.

--
View this message in context: http://forum.openscad.org/Inconsistent-conversion-of-floating-number-to-string-at-7th-significant-digit-tp14350p14407.html
Sent from the OpenSCAD mailing list archive at Nabble.com.

doug.moen wrote > ... the export to ASCII STL, and the export to > AMF. In the export case, severely truncating each float to 6 significant > digits often damages the mesh, so that OpenSCAD can't read STL files that > it itself has generated. Oh... I wasn't aware of that issue (do apologize I haven't find time to go over all old threads.) But I still... question if it's not a bit of a red herring. If OpenSCAD is really designed for and by 3D printing hobbyists and professionals, then does it serve meaningful purpose to reproduce better than 0.1PPM precision? I am not asserting it is red herring, just curious if someone really come upon a real world issue arising from such level of inaccuracy. As for OpenSCAD can't read STL it generates... again is that a high priority issue? Unlike File->Save and File->Load, I can tolerate, even expect some loss when I Export... and than Import... some data, this is true in many apps, even MS Office. Over the year (not quite 2 years yet) I've designed some moderately elaborate shapes and never had Slic3r reject a design that OpenSCAD deemed valid. I imagine other apps/devices (laser cutter?) similarly have been accepting the STL's generated by OpenSCAD. IEEE floats are imperfect representation of physical world. And text string representation are further corruption of accuracy of IEEE floats. That's unavoidable. We are quite happy to accept 20-side polygon as a circle, 100-ish-faces polyhedron for a sphere. There's easily inaccuracy of 1% in almost all of our designs. -- View this message in context: http://forum.openscad.org/Inconsistent-conversion-of-floating-number-to-string-at-7th-significant-digit-tp14350p14407.html Sent from the OpenSCAD mailing list archive at Nabble.com.
AC
Alan Cox
Thu, Nov 12, 2015 6:29 PM

But I still... question if it's not a bit of a red herring.  If OpenSCAD is
really designed for and
by 3D printing hobbyists and professionals, then does it serve meaningful
purpose to reproduce
better than 0.1PPM precision?  I am not asserting it is red herring, just
curious if someone really
come upon a real world issue arising from such level of inaccuracy.

Probably not - and someone who understands the mathematics classes and
requirements of CGAL could change it to integer. I did look into it but
the requirements are complex and C++ is not my favourite language.

I imagine other apps/devices (laser cutter?) similarly have been accepting
the STL's generated
by OpenSCAD.  IEEE floats are imperfect representation of physical world.
And text string
representation are further corruption of accuracy of IEEE floats.  That's
unavoidable.

Text strings can be far more accuratethan IEEE float. The accuracy isn't
the problem however. The problem is that OpenSCAD does not follow the
accepted rules for creation of "correct" STL objects.

Properly written STL tools use a point dictionary to ensure that they do
not output an STL file with two different points with the same IEEE float
representation. If they find one they either juggle the points in the
dictionary the smallest amount possible to avoid a clash or cull what is
lost. Culling is hairy, juggling points far less so - especially if you
have the dictionary sorted in X Y and Z.

We are quite happy to accept 20-side polygon as a circle, 100-ish-faces
polyhedron for a sphere.
There's easily inaccuracy of 1% in almost all of our designs.

Quite a few of my designs have parts that won't work at 1% error, which
is fine because OpenSCAD can do more accuracy when needed. Please don't
assume just because you can tolerate 1% everyone can. When you have
moving parts or parts combined with etched metal pieces and/or mechanisms
the precision needed is often far higher.

Alan

> But I still... question if it's not a bit of a red herring. If OpenSCAD is > really designed for and > by 3D printing hobbyists and professionals, then does it serve meaningful > purpose to reproduce > better than 0.1PPM precision? I am not asserting it is red herring, just > curious if someone really > come upon a real world issue arising from such level of inaccuracy. Probably not - and someone who understands the mathematics classes and requirements of CGAL could change it to integer. I did look into it but the requirements are complex and C++ is not my favourite language. > I imagine other apps/devices (laser cutter?) similarly have been accepting > the STL's generated > by OpenSCAD. IEEE floats are imperfect representation of physical world. > And text string > representation are further corruption of accuracy of IEEE floats. That's > unavoidable. Text strings can be far more accuratethan IEEE float. The accuracy isn't the problem however. The problem is that OpenSCAD does not follow the accepted rules for creation of "correct" STL objects. Properly written STL tools use a point dictionary to ensure that they do not output an STL file with two different points with the same IEEE float representation. If they find one they either juggle the points in the dictionary the smallest amount possible to avoid a clash or cull what is lost. Culling is hairy, juggling points far less so - especially if you have the dictionary sorted in X Y and Z. > We are quite happy to accept 20-side polygon as a circle, 100-ish-faces > polyhedron for a sphere. > There's easily inaccuracy of 1% in almost all of our designs. Quite a few of my designs have parts that won't work at 1% error, which is fine because OpenSCAD can do more accuracy when needed. Please don't assume just because you can tolerate 1% everyone can. When you have moving parts or parts combined with etched metal pieces and/or mechanisms the precision needed is often far higher. Alan
MK
Marius Kintel
Thu, Nov 12, 2015 7:03 PM

On Nov 12, 2015, at 08:05 AM, doug moen doug@moens.org wrote:

I agree, but I'm not saying that. OpenSCAD uses the same float->string conversion everywhere, including the export to ASCII STL, and the export to AMF. In the export case, severely truncating each float to 6 significant digits often damages the mesh, so that OpenSCAD can't read STL files that it itself has generated.

We don’t - file exports are written with the default precision of C++’s stream operator. If you find any cases doing otherwise it would be considered a bug.

The primary reason why OpenSCAD sometimes cannot read back its STL output is that CGAL doesn’t support reading triangles without an area. This tends to happen when converting from CGAL’s internal representation to double, where some insanely small triangles get their coordinates shifted a tiny bit due to floating point accuracy limitations.
One solution, as mentioned earlier, could be to perform a topology-aware surface optimization to ensure all triangles have a valid area, but since CGAL is the most picky component and most other tools can handle zero-area triangles, this hasn’t been a priority.

-Marius

> On Nov 12, 2015, at 08:05 AM, doug moen <doug@moens.org> wrote: > > I agree, but I'm not saying that. OpenSCAD uses the same float->string conversion everywhere, including the export to ASCII STL, and the export to AMF. In the export case, severely truncating each float to 6 significant digits often damages the mesh, so that OpenSCAD can't read STL files that it itself has generated. We don’t - file exports are written with the default precision of C++’s stream operator. If you find any cases doing otherwise it would be considered a bug. The primary reason why OpenSCAD sometimes cannot read back its STL output is that CGAL doesn’t support reading triangles without an area. This tends to happen when converting from CGAL’s internal representation to double, where some insanely small triangles get their coordinates shifted a tiny bit due to floating point accuracy limitations. One solution, as mentioned earlier, could be to perform a topology-aware surface optimization to ensure all triangles have a valid area, but since CGAL is the most picky component and most other tools can handle zero-area triangles, this hasn’t been a priority. -Marius
C
ctchin
Fri, Nov 13, 2015 2:28 AM

Alan Cox wrote

Quite a few of my designs have parts that won't work at 1% error, which
is fine because OpenSCAD can do more accuracy when needed. Please don't
assume just because you can tolerate 1% everyone can.

I do not and I do not.

The level of accuracy I grew up with (very spoiled I was) was better than 1
mil
(25 um).  I'm constantly haggling with my 3D printer service provider to
squeeze more accuracy out of them.

But 25 um on parts of typically 5-50cm is only  50ppm, and that's about
the limit of a 20+ yr veteran machinist with a multi-millions dollar shop.

I challenge the notion that OpenSCAD number to string algorithm ever causes
a real world problem.

I have my pet peeve about OpenSCAD's numeric accuracy.  I feel strongly it
is
much more deserving of developer's precise time compared the
number-to-string
issue.  Perfect! there's already a thread for it, I will continue there...

--
View this message in context: http://forum.openscad.org/Inconsistent-conversion-of-floating-number-to-string-at-7th-significant-digit-tp14350p14429.html
Sent from the OpenSCAD mailing list archive at Nabble.com.

Alan Cox wrote > Quite a few of my designs have parts that won't work at 1% error, which > is fine because OpenSCAD can do more accuracy when needed. Please don't > assume just because you can tolerate 1% everyone can. I do not and I do not. The level of accuracy I grew up with (very spoiled I was) was better than 1 mil (25 um). I'm constantly haggling with my 3D printer service provider to squeeze more accuracy out of them. But 25 um on parts of typically 5-50cm is only 50ppm, and that's about the limit of a 20+ yr veteran machinist with a multi-millions dollar shop. I challenge the notion that OpenSCAD number to string algorithm ever causes a real world problem. I have my pet peeve about OpenSCAD's numeric accuracy. I feel strongly it is much more deserving of developer's precise time compared the number-to-string issue. Perfect! there's already a thread for it, I will continue there... -- View this message in context: http://forum.openscad.org/Inconsistent-conversion-of-floating-number-to-string-at-7th-significant-digit-tp14350p14429.html Sent from the OpenSCAD mailing list archive at Nabble.com.
PT
Philipp Tiefenbacher
Fri, Nov 13, 2015 11:47 AM

Hi?
In all cases given the number is not representable in base 2 (because base
2 has no prime factor 5 like base 10 which you gave your examples in) so
the ...5 becomes a ...4999smthng and that gets rounded down correctly.

This explains the "random" behaviour.

Greetings
Philipp
On Nov 12, 2015 6:13 PM, "runsun" runsun@gmail.com wrote:

Tim V. Shaporev wrote

Advanced rounding algorithms may cast 0.5 to either 1 or 0 depending on
the preceding bit in the binary representation of the number.

See e.g. "Round half to even" https://en.wikipedia.org/wiki/Rounding

Just my $0.02
Tim

Thx, Tim. But this doesn't explain those examples I raised in the first
post, where the rounding of 5 is handled differently even the preceding
number is the same.


$  Runsun Pan, PhD

$ libs: doctest , faces ( git ), offline doc ( git ),runscad.py( 1 , 2 ,
git );

$ tips: hash( 1 , 2 ), sweep , var , lerp , animGif

--
View this message in context:
http://forum.openscad.org/Inconsistent-conversion-of-floating-number-to-string-at-7th-significant-digit-tp14350p14406.html
Sent from the OpenSCAD mailing list archive at Nabble.com.


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

Hi? In all cases given the number is not representable in base 2 (because base 2 has no prime factor 5 like base 10 which you gave your examples in) so the ...5 becomes a ...4999smthng and that gets rounded down correctly. This explains the "random" behaviour. Greetings Philipp On Nov 12, 2015 6:13 PM, "runsun" <runsun@gmail.com> wrote: > Tim V. Shaporev wrote > > Advanced rounding algorithms may cast 0.5 to either 1 or 0 depending on > > the preceding bit in the binary representation of the number. > > > > See e.g. "Round half to even" https://en.wikipedia.org/wiki/Rounding > > > > Just my $0.02 > > Tim > > Thx, Tim. But this doesn't explain those examples I raised in the first > post, where the rounding of 5 is handled differently even the preceding > number is the same. > > > > > ----- > > $ Runsun Pan, PhD > > $ libs: doctest , faces ( git ), offline doc ( git ),runscad.py( 1 , 2 , > git ); > > $ tips: hash( 1 , 2 ), sweep , var , lerp , animGif > > > > > > > -- > View this message in context: > http://forum.openscad.org/Inconsistent-conversion-of-floating-number-to-string-at-7th-significant-digit-tp14350p14406.html > Sent from the OpenSCAD mailing list archive at Nabble.com. > > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >
TV
Tim V. Shaporev
Fri, Nov 13, 2015 12:45 PM

On 12.11.2015 22:03, Marius Kintel wrote:

We don’t - file exports are written with the default precision of C++’s stream operator. If you find any cases doing otherwise it would be considered a bug.

???

char buf[32];
sprintf(buf, "%.16f", x);
cout << buf;

Better approach would be to specify output accuracy like
sprintf(buf, "%.*f", n, f);

where n is something like

if      (x >= 1e15) n = 0;
else if (x >= 1e14) n = 1;
else if (x >= 1e13) n = 2;
else if (x >= 1e12) n = 3;
else if (x >= 1e11) n = 4;
else if (x >= 1e10) n = 5;
else if (x >= 1000000000) n = 6;
else if (x >= 100000000) n = 7;
else if (x >= 10000000) n = 8;
else if (x >= 1000000) n = 9;
else if (x >= 100000) n = 10;
else if (x >= 10000) n = 11;
else if (x >= 1000) n = 12;
else if (x >= 100) n = 13;
else if (x >= 10) n = 14;
else if (x >= 1) n = 15;
else n = 16;

or more accurately
if (x >= 1e16)
sprintf(buf, "%.15g", x);
else {
register int n = 0;
if (x < 10000000) {
if (x < 1000) {
if (x < 10)
n = x < 1 ? 16 : 15;
else
n = x < 100 ? 14 : 13;
} else {
if (x < 100000)
n = x < 10000 ? 12 : 11;
else
n = x < 1000000 ? 10 : 9;
}
} else if (x < 1e15) {
if (x < 1e11) {
if (x < 1000000000)
n = x < 100000000 ? 8 : 7;
else
n = x < 1e10 ? 6 : 5;
} else {
if (x < 1e13)
n = x < 1e12 ? 4 : 3;
else
n = x < 1e14 ? 2 : 1;
}
}
sprintf(buf, "%.*f", n, x);
}

Does this make any sense?

Tim

On 12.11.2015 22:03, Marius Kintel wrote: > We don’t - file exports are written with the default precision of C++’s stream operator. If you find any cases doing otherwise it would be considered a bug. ??? char buf[32]; sprintf(buf, "%.16f", x); cout << buf; Better approach would be to specify output accuracy like sprintf(buf, "%.*f", n, f); where n is something like if (x >= 1e15) n = 0; else if (x >= 1e14) n = 1; else if (x >= 1e13) n = 2; else if (x >= 1e12) n = 3; else if (x >= 1e11) n = 4; else if (x >= 1e10) n = 5; else if (x >= 1000000000) n = 6; else if (x >= 100000000) n = 7; else if (x >= 10000000) n = 8; else if (x >= 1000000) n = 9; else if (x >= 100000) n = 10; else if (x >= 10000) n = 11; else if (x >= 1000) n = 12; else if (x >= 100) n = 13; else if (x >= 10) n = 14; else if (x >= 1) n = 15; else n = 16; or more accurately if (x >= 1e16) sprintf(buf, "%.15g", x); else { register int n = 0; if (x < 10000000) { if (x < 1000) { if (x < 10) n = x < 1 ? 16 : 15; else n = x < 100 ? 14 : 13; } else { if (x < 100000) n = x < 10000 ? 12 : 11; else n = x < 1000000 ? 10 : 9; } } else if (x < 1e15) { if (x < 1e11) { if (x < 1000000000) n = x < 100000000 ? 8 : 7; else n = x < 1e10 ? 6 : 5; } else { if (x < 1e13) n = x < 1e12 ? 4 : 3; else n = x < 1e14 ? 2 : 1; } } sprintf(buf, "%.*f", n, x); } Does this make any sense? Tim
AC
Alan Cox
Fri, Nov 13, 2015 1:07 PM

Better approach would be to specify output accuracy like
sprintf(buf, "%.*f", n, f);

where n is something like

What is wrong  "%.*g" with FLT_DECIMAL_DIG (from float.h). None of the
other mucking around should be needed.

Alan

> Better approach would be to specify output accuracy like > sprintf(buf, "%.*f", n, f); > > where n is something like What is wrong "%.*g" with FLT_DECIMAL_DIG (from float.h). None of the other mucking around should be needed. Alan
TV
Tim V. Shaporev
Fri, Nov 13, 2015 1:16 PM

DBL_DECIMAL_DIG may be?

On 13.11.2015 16:07, Alan Cox wrote:

Better approach would be to specify output accuracy like
sprintf(buf, "%.*f", n, f);

where n is something like

What is wrong  "%.*g" with FLT_DECIMAL_DIG (from float.h). None of the
other mucking around should be needed.

Alan


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

DBL_DECIMAL_DIG may be? On 13.11.2015 16:07, Alan Cox wrote: >> Better approach would be to specify output accuracy like >> sprintf(buf, "%.*f", n, f); >> >> where n is something like > > What is wrong "%.*g" with FLT_DECIMAL_DIG (from float.h). None of the > other mucking around should be needed. > > Alan > > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >
TV
Tim V. Shaporev
Fri, Nov 13, 2015 4:44 PM

After some thinking I remembered why I imagined such piece of code :-)
to truncate trailing zeros (see at the end):

if (x >= 1e16)
sprintf(buf, "%.15g", x);
else {
register int n = 0;
if (x < 10000000) {
if (x < 1000) {
if (x < 10)
n = x < 1 ? 16 : 15;
else
n = x < 100 ? 14 : 13;
} else {
if (x < 100000)
n = x < 10000 ? 12 : 11;
else
n = x < 1000000 ? 10 : 9;
}
} else if (x < 1e15) {
if (x < 1e11) {
if (x < 1000000000)
n = x < 100000000 ? 8 : 7;
else
n = x < 1e10 ? 6 : 5;
} else {
if (x < 1e13)
n = x < 1e12 ? 4 : 3;
else
n = x < 1e14 ? 2 : 1;
}
}
sprintf(buf, "%.*f", n, x);
if (n > 0) {
for (i=strlen(buf); n && '0' == buf[n];) --n;
if (n>0 && '.' == buf[n]) --n;
buf[n] = '\0';
}
}

On 13.11.2015 16:07, Alan Cox wrote:

Better approach would be to specify output accuracy like
sprintf(buf, "%.*f", n, f);

where n is something like

What is wrong  "%.*g" with FLT_DECIMAL_DIG (from float.h). None of the
other mucking around should be needed.

Alan


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

After some thinking I remembered why I imagined such piece of code :-) to truncate trailing zeros (see at the end): if (x >= 1e16) sprintf(buf, "%.15g", x); else { register int n = 0; if (x < 10000000) { if (x < 1000) { if (x < 10) n = x < 1 ? 16 : 15; else n = x < 100 ? 14 : 13; } else { if (x < 100000) n = x < 10000 ? 12 : 11; else n = x < 1000000 ? 10 : 9; } } else if (x < 1e15) { if (x < 1e11) { if (x < 1000000000) n = x < 100000000 ? 8 : 7; else n = x < 1e10 ? 6 : 5; } else { if (x < 1e13) n = x < 1e12 ? 4 : 3; else n = x < 1e14 ? 2 : 1; } } sprintf(buf, "%.*f", n, x); if (n > 0) { for (i=strlen(buf); n && '0' == buf[n];) --n; if (n>0 && '.' == buf[n]) --n; buf[n] = '\0'; } } On 13.11.2015 16:07, Alan Cox wrote: >> Better approach would be to specify output accuracy like >> sprintf(buf, "%.*f", n, f); >> >> where n is something like > > What is wrong "%.*g" with FLT_DECIMAL_DIG (from float.h). None of the > other mucking around should be needed. > > Alan > > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >
DM
doug moen
Fri, Nov 13, 2015 5:04 PM

The best way for OpenSCAD to print floating point numbers is using the
algorithm from the paper "How To Print Floating Point Numbers Accurately".
That's what Python uses. That's what Javascript uses. That's what most
languages use, except...

...it's impossible to do this in C or C++ using only the standard library.
You need to use a separate open source library. The printf %g format is
bullshit, since it throws away most of the information in the number. You
can't do it with printf or std::stream:<<. The C and C++ standards are
rubbish for not supporting this important feature.

More details in my earlier post:
http://lists.openscad.org/pipermail/discuss_lists.openscad.org/2015-August/003078.html

Maybe I'll try implementing this on the weekend.

On 13 November 2015 at 08:07, Alan Cox alan@lxorguk.ukuu.org.uk wrote:

Better approach would be to specify output accuracy like
sprintf(buf, "%.*f", n, f);

where n is something like

What is wrong  "%.*g" with FLT_DECIMAL_DIG (from float.h). None of the
other mucking around should be needed.

Alan


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

The best way for OpenSCAD to print floating point numbers is using the algorithm from the paper "How To Print Floating Point Numbers Accurately". That's what Python uses. That's what Javascript uses. That's what most languages use, except... ...it's impossible to do this in C or C++ using only the standard library. You need to use a separate open source library. The printf %g format is bullshit, since it throws away most of the information in the number. You can't do it with printf or std::stream:<<. The C and C++ standards are rubbish for not supporting this important feature. More details in my earlier post: http://lists.openscad.org/pipermail/discuss_lists.openscad.org/2015-August/003078.html Maybe I'll try implementing this on the weekend. On 13 November 2015 at 08:07, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote: > > Better approach would be to specify output accuracy like > > sprintf(buf, "%.*f", n, f); > > > > where n is something like > > What is wrong "%.*g" with FLT_DECIMAL_DIG (from float.h). None of the > other mucking around should be needed. > > Alan > > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org > > >