[tex-k] experiment: moving hash above eqtb_size

[Hartmut Henkel]

Hi,

an Easter exercitium: In the various TeX executables, the hash is in a
jail of fixed hash_size, which nowadays is 10000, and the hash_prime is
fixed to 8501. The texmf.cnf parameter hash_extra allows to extend only
the list part of the hash above eqtb_size, but the main hash scatter
area is not affected. As a result, if hash_extra > 0, the hash more and
more degenerates into a bunch of (slow?) linked lists. The tex.web says
that for a coalesced list hash the hash_prime should be about 85 % of
the hash size, which means 85 % of (hash_size + hash_extra).

The goal of my experiment was to get the hash back to this condition,
and to allow the setting of the hash_size and hash_prime through
texmf.cnf, by patching recent pdfetex-1.20a. The 1st try was to split
the hash scatter area up in two parts, and to move one part above
eqtb_size to make its size programmable, but this looked ugly. The 2nd
try, to make the hash_size of the hash in its original place
programmable, didn't work, as this memory area is fixed by lots of
constants through web macros. Finally it seemed the easiest to scrap the
original hash area altogether and put the complete hash above eqtb_size.
This also cleans up things a little, as the id_lookup() function then is
more like the original tex.web one. But now, happily, hash_size and
hash_prime are variables.

The hash_size is now also a texmf.cnf parameter, and parameter
hash_extra has gone. The hash_prime is (automagically) selected from a
list of exponentially spaced primes to be always close to hash_size.

The hash_size and hash_prime are dumped into the format file, and they
are undumped by the pdfetex working engine. If the parameter hash_size
from texmf.cnf or the derived hash_prime don't match their corresponding
values in the format file, the format needs to be regenerated. It's
currently not foreseen to increase the hash_size without making a fresh
format, but in principle one could also extend the list part of the hash
beyond hash_size. The values hash_size and hash_prime are shown if
\tracingstats=1.

You find this raw experimental patch for pdfetex (Web2C 7.5.2)
3.14159-1.20a-2.1-rc1 at:

http://www.circuitwizard.de/pdftex/patch4/tex.ch3.gz

Try on your own risk. The status here is, that it runs so that all
pdfetex based formats of the recent teTeX-beta can be generated and then
be used for pdfetex based PDF/DVI file generation. Speed comparisons are
yet to be done. And more testing and debugging. It should be possible to
adapt this patch also to the non-pdf TeX versions.

Comments welcome!

Happy Easter!

Hartmut


------------------------------------------------------------------------
H. Henkel, Oftersheim, Germany
------------------------------------------------------------------------