[XeTeX] First experiences with xetex and some bugs

[Jonathan Kew]

On 4 Jul 2004, at 10:59 pm, Steffen Prohaska wrote:

> If I get you right, you're basically saying that someone had to define 
> all the special character commands to map the input to unicode 
> (utf-8)? If one provided a complete mapping, this would solve all the 
> problems?

Yes, I think so. Look back in the list archives; someone (Ross, IIRC) 
has already posted examples of how to redefine these things in LaTeX.

>  Or at least I could do this for all the special characters I'm using. 
> But I never heard anything about "active character" coding and have no 
> idea how to proceed with this.

"Active characters" are individual characters that are given \catcode 
13 so that TeX treats them as macros in their own right (no \ needed); 
they can then do things like examine the next character and decide what 
to do. The TeXbook includes examples, e.g., where it talks about 
implementing hanging punctuation.

>  Isn't there some kind of active "character coding" already going on 
> in xetex to provide ligatures (ff, fi ...)?
>

No, this happens entirely within the AAT or OpenType fonts; at the TeX 
macro level, there's a simple stream of characters exactly as they 
occur in the source.

>>> I'd like to continue using Yandy's mathtimes or Thierry Bouche's 
>>> mathfont.sty,
>>> http://www.ntg.nl/maps/pdf/21_18.pdf . But neither work with xetex. 
>>> I read on
>>> the mailinglist that xetex is focused on typesetting text only. If 
>>> this will
>>> be the only focus in the future I think this would be a major 
>>> problem for most
>>> of the TeX users. I still use TeX especially because of the 
>>> simplicity of using
>>> math. But the math fonts have to follow the other fonts at least to 
>>> some extent.
>>> If xetex doesn't provide a way this will be a show stopper for me.
>>
>> I don't have these fonts/packages, so cannot try them. I suspect that 
>> if the fonts were converted to .otf format, similarly to the CM and 
>> other fonts bundled with XeTeX, it would be possible to use them just 
>> like the CM fonts.
>
> I agree. This might be true. The packages basically are telling that 
> they will not work with the present encoding. But I don't have an idea 
> how to proceed with this problem. Do you have a link were I can find 
> more details on 'converting to .otf' format?

It can presumably be done with Fontforge (formerly PFAEdit), though I 
used FontLab for the fonts currently bundled with XeTeX. Note that for 
XeTeX to be able to apply encoding vectors (.enc files) from 
psfonts.map, it is necessary for the .otf to include a full 'post' 
table with glyph names; I used Apple's font tools (see 
http://developer.apple.com/fonts) to add this.

I intend to post a description of all this, and the scripts I used, 
when I get time to work on it.

>  Right now, I would perhaps decide to stay with CM. If I got all the 
> encoding stuff mentioned above working, xetex could be useful for me, 
> nonetheless. It's great work and I think a pretty cool tool for text 
> only (no math) documents right now. Thanks Jonathan, for your work!
>
>>>
>>> ---+ Bug: Including pdf images
>>> ---++ Quick Summary
>>>
>>> Including certain pdf images breaks font rendering starting with the 
>>> next page.
>>> See the attached test document and the output files created using 
>>> pdflatex and
>>> xelatex.
>>>
>>
>> This problem seems to occur when using a PDF that was created by 
>> Quartz, but not with other PDFs such as from Distiller. Any PDF 
>> experts here who can inspect the test.pdf file and tell us if it is 
>> somehow invalid? It sounds like there may be a problem with Quartz 
>> PDF generation; if we can pin down exactly what's wrong, I'll file a 
>> bug report with Apple.
>
> I think you're right. All the pdf documents I encountered problems 
> with were created by Apple's Preview. I copied stuff to the clipboard 
> and used 'File - New From Clipboard', to get a pdf document, which I 
> saved to disk. I didn't have problems with other pdfs which were 
> created using, e.g. xfig. But nonetheless there should be a way to 
> deal with theses documents. At least pdftex is able to do so.

Yes, I see that pdfTeX handles them OK. I'm thinking the problem is 
with the PDF generation through the Quartz graphics, when a page from 
another PDF document is drawn into the context. I don't see anything 
further that I can do to try and solve this within my code, as I'm 
already wrapping the rendering of the embedded PDF with calls to 
CGContextSaveGState...CGContextRestoreGState. The fact that the 
particular PDF document drawn then disrupts font rendering on the next 
page looks like an Apple (Quartz) bug to me.

Jonathan