[tex-k] Does TeX's Input Processor Tokenize The Entire Input File First?

[Jon Forrest]

I'm trying to understand TeX internals.

One thing I haven't found a definitive explanation of is whether the
input processor tokenizes the entire input file before the execution
processor gets the token list. TexbyTopic says "... all [processor]
levels are simultaneously  active, and there is interaction between
them.". Given the primitive  computing environment TeX was developed in,
it's hard to imagine how the  4 processors could run simultaneously. As
far as I can see, TeX has no multi-threading ability so it would be up
to TeX to "process" switch.

So, how and why do the processors run simultaneously? How much of the
input file gets tokenized before the token list is handed to the
expansion processor?

I wonder if Knuth were writing TeX today would he use multi-threading?

Cordially,
Jon Forrest