[metapost] Long numerical tokens with new number systems

[luigi scarso]

On Sat, Aug 15, 2015 at 9:52 PM, Toby Thurston <thurston at eml.cc> wrote:

> The syntax for <numeric token> that MP inherits from Metafont allows for
> "one or more" decimal digits in the fractional part of a number.  So it's
> legal to write:
>
> numeric pi;
> pi = 3.141592653589793238462643383279502884197169399375105820974944592;
> show pi;
>
> This works ok with all the number systems except "decimal" which complains
> about excessive precision.
> Here's what I get with each number system
>
> >> "scaled"
> >> 3.14159 )
>
> >> "double"
> >> 3.1415926535897931 )
>
> >> "binary"
> >> 3.1415926535897932384626433832795028 )
>
> >> "decimal"
> ! Number is too precise (numberprecision = 34).
> l.80 ...643383279502884197169399375105820974944592
>                                                   ;
> ?
> >> 3.141592653589793238462643383279503 )
>
> Doesn't this behaviour seem to be a bit inconsistent?
>
> Clearly, with decimal it would be possible to avoid the error message by
> writing this sort of thing:
>
>    if numbersystem="decimal": numberprecision := 64; fi
>
> before defining the constant, but it would be nice to have a consistent
> syntax across all number systems.
>
> One other thing that I notice is that the final digit rounding looks wrong
> for double and binary.
>
> Any comments?
>
> Thanks, Toby
> --
> http://tug.org/metapost/
>


Rev. 2070 has now a patch based on
"27 bits are not enough for 8-digit accuracy" by I. Bennett Goldberg



-- 
luigi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tug.org/pipermail/metapost/attachments/20151006/ca15f1a1/attachment.html>