[metapost] [mp-implementors] MetaPost 1.750 (alpha version)

[Dan Luecking]

At 04:22 PM 6/23/2011, you wrote:
>In article <4E03A18D.8000102 at wxs.nl>, Hans Hagen <pragma at wxs.nl> wrote:
>
> > On 23-6-2011 9:41, Troy Henderson wrote:
> > > Is there a built-in way with MetaPost 1.750 and with --math=double to
> > > "round" a number by using only a certain number of significant digits?
> > >
> > > For example, I'm getting an output of
> > >
> > > 7.999999999999998
> > >
> > > which is "begging" to be 8.

If it were exactly 8, it would be so displayed. Presumably it
was calculated. If it were _theoretically_ 8, the inexactitude
of either numerical representation or the calculation process
could still produce a result that looked like this. One should
accept this format as reminding you that computing has inherent
inaccuracies, and embrace it, not hide it.

> >
> > So maybe we need a round(n,precision) ?
>
>Maybe with a different name (roundoff, roundfloat, roundprec, ...)? Plain
>MetaPost already defines a "round" vardef...

I don't know how metapost stores math doubles, but the old
version apparently used a 32 bit integer. The plain.mp version
of round works correctly for rounding to an integer, and it is
easy to adapt it to round to other significance:
   round(x*8)/8
(barring numeric overflow) would produce x rounded to the nearest
1/8. This would produce exact rounding _only_ to powers of 2.
Rounding to a certain number of decimal places (powers of 10)
this way would not be exact. Thus, the only exact way to deal
with decimals is to convert to a decimal string and process
its characters. However, converting it back to an internal
numeric stored in binary could immediately introduce inaccuracy
and produce essentially the same problem again.

Of course, all this inaccuracy disappears if we are rounding to
integers, but essentially _only_ then.

If one is only concerned with appearences, what is needed is an
enhancement to decimal that accepts also a format and returns a
decimally formatted string. Perhaps also an option as to how to
format numbers for screen display.

Dan


Daniel H. Luecking
Department of Mathematical Sciences
Fayetteville, Arkansas
http://www-cs-faculty.stanford.edu/~knuth/iaq.html