[tex-k] Speed of kpsewhich when querying the repertories of the TeX installation

[jfbu]

> Hi,

Hi Norbert

thanks for the detailed answer

I will only reply to the locations where you refer to
my experience of how long kpsewhich takes to complete

(sorry if I cut-out most of your reply, it was very
informative and now it is too late to paste it back
and I had no comments apart from thanking you for the info)

>

>> But in that cas TeXLive installation's manual should be explicit
>> about it. I am sure this brings other complications.
>
> For what? How many kpsewhich calls a user does in his *whole*life*,
> I am wondering ....

as mentioned below Emacs/AUCTeX 11.88 does it 9 times per launch

>
>> TEXMF_local = $(shell kpsewhich -var-value TEXMFLOCAL)
>>
>> # as kpsewhich is very slow (.5s) I want to evaluate once only.
>> installlocal: xint.tds.zip
>> 	$(eval $@_tmp := $(TEXMF_local))
>> 	unzip xint.tds.zip -d $($@_tmp) && texhash $($@_tmp)
>
> Scrapping .5s from an *install* routine that is called *once*
> every whatever, weeks, months?

well, as I said, the issue became really acute to me only since
I realized that Emacs/AUCTeX launch time was affected by it

>
> If you don't have *any* other problem you must be happy.
>

I have the Emacs/AUCTeX problem ! (currently being resolved by
the maintainers)

>> If I resuscitated this issue after some months, it is because I
>> discovered yesterday that Emacs/AUCTeX 11.88 on Mac OS X very
>> lengthy launch was entirely to be explained by its nine calls to
>> kpsewhich.
>
> According to your statement, 9 * 0.2 sec is about 1.8sec. Taking
> caching of the ls-R from the OS into account, my guess is that
> the actual time spent in kpsewhich calls is about 1.5sec.


from where do you get the 0.2 sec ? It is 0.5 sec on my Mac Book Air
with a 250G SSD and TeXLive 2014

95% of the 5 seconds launching time of AUCTeX is explained by the
nine kpsewhich calls

>
> IF you are so conservative about it, why not add to your .profile:
> 	TEXMFLOCAL=$(kpsewhich -var-value TEXMFLOCAL)
> 	TEXMFHOME=$(kpsewhich -var-value TEXMFHOME)
> 	export TEXFMLOCAL TEXMFHOME
> and all will be hell of a speed fast....
>
> Isn't that the simplest solution for all your problems?

It is a good solution.

But it would make me deal with issues I am not
to eager to go into, like how does the OS pass
the environment variables to GUI processes.

I never installed Yosemite (Mac OS 10.10) but as far as I know
there was a big bug in it regarding setenv/getenv

https://email.esm.psu.edu/pipermail/macosx-tex/2014-October/053065.html

I also recalled from earlier times that there was a
    
    $HOME/.MacOSX/environment.plist

which at some point disappeared.

http://apple.stackexchange.com/questions/126455/is-home-macosx-environment-plist-still-used-in-os-x-10-9-2-for-setting-global

Thus, regarding environment variables, things fluctuate.

However, it would instantly solved all my problems. But not
the ones for distributing packages with install scripts.

>
>> Earlier AUCTeX 11.86 did only 2, and furthermore it happened
>> during initial launch of Emacs, not on (first) opening a .tex file
>>
>> With 9 calls to kpsewhich, on my (comparatively quite fast, with
>> SSD) MBA with Mavericks and TeXLive 2014, this meant a 5 seconds
>> wait when first launching AUCTeX 11.88
>
> End after that? 0 .. right. Aehmmm, well.
>
>>
>> On an older Mac machine, as the one Adam Maxwell tested my October
>> 2014 findings about the slowness of kpsewhich --var-value
>> TEXMFHOME, it would have been 15 seconds.
>
> 15sec??? I would be surprised, that must be a very bad machine that
> loading a ls-R file into memory and parsing it takes that much
> time.
>

quoting from
https://email.esm.psu.edu/pipermail/macosx-tex/2014-October/053021.html

On a 2007 MacBook Pro with a hard disk, running
TL 2013 on Mac OS X 10.9.5, I get

$ time kpsewhich -var-value=TEXMFLOCAL
/usr/local/texlive/texmf-local

real	0m1.401s
user	0m0.926s
sys	0m0.038s


>> Let me explicitely mention to people reading this thread that this
>> issue does concern also Linux boxes, but may well be hidden to
>> most because their TeXLive installation has set-up environment
>> variables for TEXMFLOCAL, TEXMFHOME, etc...
>
> As the main dev of the TeX Live infra, I am running several installation
> of TeX Live, and I do NOT have *any* env variable. As a logician and
> mathematician I am editing TeX files in Emcas with AucTeX on a daily
> basis, and never had the feeling that *this* is the prbolems.

AUCTeX 11.86 did only 2 calls to kpsewhich

AUCTeX 11.88 does 9 calls.

If 1/10th of a second is not a lot, 9/10th of a second starts
being a nuisance


>
> As said above, there is no fast way unless you do something like
> a server. But that needs to monitor the file system for changes
> in the texmf.cnf files, and this is something that nobody wants to
> do for kpsewhcih calls ...


I have the lagging feeling that some other approach, perhaps with
a single file at some mandatory location, could bring some ways
to instantly recover the location of the main trees of a TeX installation

Best

Jean-François