[XeTeX] problem with discretionary

[Zdenek Wagner]

Hi,

please, notice that the number of character classes was increased from 256
to 4096, so 255 no longer works as a boundary but 4095 must be used. I use
the following code that I took from some other package:

\edef\CSat{\the\catcode`\@} % in order to work in plain XeTeX
\catcode`\@=11
\ifdefined\e at alloc@intercharclass at top
  \chardef\CSboundary=\e at alloc@intercharclass at top
\else
  \ifdefined\XeTeXinterwordspaceshaping
    \chardef\CSboundary=4095 %
    \def\newXeTeXintercharclass{%
      \e at alloc\XeTeXcharclass\chardef
              \xe at alloc@intercharclass\m at ne\@ucharclass at boundary}
  \else
    \chardef\CSboundary=255
  \fi
\fi
\catcode`\@=\CSat

Afterwards I use \CSboundary instead of a fixed number. It thus works both
with the old and new XeTeX.


Zdeněk Wagner
http://ttsm.icpf.cas.cz/team/wagner.shtml
http://icebearsoft.euweb.cz

2017-12-03 10:19 GMT+01:00 jfbu <jfbu at free.fr>:

> Hi,
>
> I need some help to identify which XeTeX release fixed
> that problem, the mwe is
>
> \catcode`@ 11
> \XeTeXinterchartokenstate=1
> \newXeTeXintercharclass\french at punctthin
> \XeTeXcharclass `\; \french at punctthin
>      \XeTeXinterchartoks 255 \french at punctthin = {\nobreak\thinspace}%
> \catcode`;\active
> \def;{\discretionary{\char`\;}{}{\char`\;}}
> a;b
> \bye
>
> In  real life it appeared in a Polyglossia+French context
> with the semi-colon make active to insert a \discretionary
> similar to the above. There is no issue in lualatex.
>
> It is currently seen at Python upstream (CPython) when
> they try to build French docs (via Sphinx)
>
> https://bugs.python.org/issue31589
>
> and it would be nice to pinpoint which XeTeX release
> precisely is ok. I know 0.99992 is bad and 0.99996 is good,
> but can't easily bisect.
>
> Best,
>
> Jean-François
>
>
>
>
> --------------------------------------------------
> Subscriptions, Archive, and List information, etc.:
>   http://tug.org/mailman/listinfo/xetex
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tug.org/pipermail/xetex/attachments/20171203/f48b80e5/attachment.html>