[XeTeX] Hyperref problem

[Jonathan Kew]

On 21 Jul 2006, at 6:52 am, jropers at freesurf.fr wrote:
>
> I have found that there is some interaction with the option  
> [unicode] given to hyperef (I probably read somewhere the  
> suggestion to include it but I don't remember where).
> .....
> I guess [unicode] to hypperef is a bad idea ?

Aha, it looks like you've found the problem!

I've never heard of the [unicode] option (though I expect it's in the  
hyperref doc, I'm just not familiar with this stuff). But what I  
think it's doing is trying to write bookmark information to the .out  
file in UTF-16, which is how it needs to appear in the final PDF.  
It'll do this based on the assumption that \write can directly write  
any byte value into the file. Meanwhile, however, xetex is  
automatically treating the character codes given to \write as Unicode  
characters and encoding them as UTF-8 byte sequences. And xdvipdfmx  
is taking the bookmark text from the \special where it ends up,  
reading it as UTF-8 and automatically converting to UTF-16 for  
inclusion in the PDF.

So it's a case of multiple levels trying to deal with the Unicode  
values in different, competing ways; and by turning off hyperref's  
processing (which is based on the assumption that TeX is working with  
byte codes), you allow xetex/xdvipdfmx to handle it all.

It's like \usepackage[utf8]{inputenc}; you *must not* use this with  
xetex, because all the Unicode encoding stuff is already handled by  
the engine, and a package like this will add an unwanted extra layer  
of "interpretation" to the codes -- treating actual Unicode  
characters as if they were the individual bytes of UTF-8 sequences.

JK