[metapost] Units unaccuracy with numbersystem set to double

[Nelson H. F. Beebe]

Akira Kakuto <kakuto at fuk.kindai.ac.jp> writes today:

>> ...
>> MetaPost 1.900 shows
>>
>> *show cm;
>> >> 28.346450000000001
>>
>> Note that MetaPost does not calculate it from an exact formula.
>> It simply prints the value defined in plain.mp: cm=28.34645;
>> ...

A more accurate conversion from PostScript big points to cm is this:

    hocd128> 72 / 2.54
             28.346_456_692_913_385_826_771_653_543_307_09

Now that Metapost supplies arithmetic of higher precision, perhaps
input files and source code should be examined for truncated
floating-point constants, and those constants replaced by expressions,
such as the exact 72 / 2.54 above, that could then be evaluated at run
time in the prevailing precision, which might be higher than that in
classic METAFONT. 

Although there is unlikely to be a visible difference in the
appearance of font glyphs and drawings from such changes, at least
they might reduce user confusion.

-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe at math.utah.edu  -
- 155 S 1400 E RM 233                       beebe at acm.org  beebe at computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------