After a quarter-century chronicling the spook business, Allen Weinstein has learned that a bad archivist is often the historian’s best friend. Weinstein first realized as much in the ’70s, when he and the American Civil Liberties Union sued the FBI to open its files on Alger Hiss. After Weinstein & Co. won, forcing the first-ever release of secret historical files, he found that the agency’s historical filing dungeon neither ranked high on the career path for on-the-move G-men nor was staffed with trained archivists. “We benefited, because they did not always know what they were releasing to us,” Weinstein says. Out of that episode came his landmark 1978 book, Perjury: The Hiss-Chambers Case.

Much the same happened in the early 1990s, when Weinstein went to Russia to research The Haunted Wood: Soviet Espionage in America—The Stalin Era (Random House). As Weinstein found, the Russians kept their files in an even less orderly fashion than the Americans did. “We think of the Soviet Union as some massively systematic country,” he says. “Some of their intelligence files were kept systematically, but the documents from the 1930s and ’40s had been dumped in big packets, punched with holes, and sewn together with what looked like shoelaces.”

Weinstein, 61, got involved with Eastern Europe’s post-Cold War transition as head of the D.C.-based Center for Democracy, where he helped organize networking sessions between senior Eastern European government officials.

At one conference, Weinstein met Gen. Vadim Kirpichenko, the head of a group of ex-KGB agents. While he was squiring Kirpichenko around the U.S. dinner circuit, Random House offered Weinstein the opportunity to take over a project in which Russia was to open its intelligence archives in exchange for lots and lots of American dollars. The project became The Haunted Wood; eventually, Weinstein was teamed with KGB-agent-turned-journalist Alexander Vassiliev, who, under the agreement, had been hand-picked by the Russians as the book’s co-author.

After they started their research, the political climate between Russia and the U.S chilled, thanks in part to Aldrich Ames and Vladimir Zhirinovsky, and the authors received their last materials from Russia in 1995. Fortunately, it was around that time that the U.S. opened up some of its key Cold War archives from the period, enabling Weinstein and Vassiliev to fill in some of the gaps.

After 1945, he says, Soviet intelligence officials grew terrified that their entire network of Americans would be hauled before the FBI. “So they literally stopped all work,” Weinstein contends. “Knowing this, it’s ironic to think that four to five years before Sen. Joe McCarthy began his escapades, the spies had stopped their work.”;Louis Jacobson