Disappearing NSA Files

[John K. Taber]

Mr. Jon D. Holstine asserted:
   The sort of work involved in breaking down the Soviet messages was
   not supported by computers.  The capacity to handle such work
   may now exist.  However, given NSA's current tasking with world wide
   terrorism, I can see a reluctance to devout the manhours
   necessary to set up  the raw, encrypted stuff, for computer treatment.

I have to ask what he bases his assertion on because I strongly suspect
that NSA indeed used computers to aid its cryptanalysis of the
intercepts.

I suspect that NSA's STRETCH/HARVEST specs were written specifically to
enable not just cryptanalysis in general but cryptanalysis of Venona
specifically. This was a supercomputer built by IBM to NSA's specs. Work
was begun in the 50s, and STRETCH/HARVEST was delivered to NSA in 1962.
It remained in service until the 1977 or so, when it was returned to
IBM. Supposedly the tape tractors had become defective and couldn't be
replaced.

According to an NSA 1968 report, cited in a talk by Fran Allen, STRETCH/
HARVEST did the following:

    "Recently HARVEST scanned 7,075,315 messages of approximately 500
     characters each -- examining every possible offset -- to see if
     they contained any of 7,000 different words or phrases. This ...
     required three hours and 50 minutes to complete -- an average of
     over 30,000 messages a minute."

What is being described is the equivalent of "dragging" a cryptogram
to site a tip. For Venona, this means dragging messages super-encrypted
with the same page(s) of one-time pads with probably code groups.

The size of the messages sound about right for the average Venona
message.

The 7000 words or phrases is about right for Venona. There were at least
five code books used, of 4-digit code groups arranged as a one-part code
book. In other words, there were 10,000 possible code groups, of which
some number were not known. 7000 sounds very good.

Seven million messages are far beyond what was released (roughly 3000+
msgs), but there is evidence here and there in the released messages of Soviet
messages that were never released. For example, there was presumably
Washington-London Naval GRU messages. They are nowhere to be found in
the released corpus. And, this specific computer most likely was used for
other traffic besides Venona.

Fran Allen was the lead designer on ALPHA, a programming language for
STRETCH/HARVEST, that was designed specifically as a cryptanalytic
language, which, from her talks, was used intensively by NSA. Imagine
that, an entire programming language specifically for code breaking!
I would have loved to have seen the manual for ALPHA, which I think is
still at IBM Research.

So, I think the work was computerized in the early 60s at latest,
remained in progress until its usefulness was at an end, and then
the work was put on the shelf about 1980.

I am very anxious to hear from Mr. Holstine what evidence he has for
supposing that Venona cryptanalysis was not computerized.

John K. Taber

Intelligence Forum (http://www.intelforum.org) is sponsored by Intelligence
and National Security, a Frank Cass journal (http://www.frankcass.com/jnls/ins.htm)