Monday, September 17, 2007

Rewriting computing history with Colossus

What if I told you that a secret project conducted more than 60 years ago held the true origins of the modern computing era? And that the country behind this project did such a good job erasing its tracks that it did itself a disservice? And that many of the things invented during this project would only be realized with modern-day PCs?

It sounds like a made-for-TV movie, and it should be, and finally -- Now The True Story of Colossus Can Be Told (cue announcer and swelling music).

The world's first electronic digital computer wasn't developed on American soil but presaged by nearly two years during an intensive British top-secret wartime effort called Colossus. A team of engineers built the room-sized computer for a single purpose: to decrypt German military radio transmissions. The original designer died without ever receiving any acknowledgment or acclaim accorded to many of his American computing counterparts, yet he was responsible for the first implementations of program-controlled logic, parallel processing, variable programming, hardware interrupts, optical reading of punched paper tape, shift registers and other things that are now common to the PC lexicon. Some of these innovations took a decade or more to implement elsewhere by others.

And lest you think that Colossus was so old school slow, consider this: It was so massively parallel and efficient at what it did that a modern PC programmed to do similar code breaking tasks now takes about as long to achieve the same results!

There were actually ten Colossus models produced during the war by a team led by Thomas Flowers, who worked doing telephone switching research before and after the war at what was to become British Telecom. By a series of fortuitous circumstances, Flowers was originally hired for another project, and took it upon himself to develop Colossus using his own designs.

Why are we just figuring its place in history now? Several reasons: first, the Brits have finally declassified the key documents and lifted gag orders on the project, and as a result a wonderful book is now out by Oxford University Press.

The book is a huge 460-page tome with many details about the code breaking process directly from the people who worked at Bletchley Park and Dollis Hill, the government labs involved. This is a collection of first-person accounts and you get to see the enormity of the task and exactly how critical this effort was towards winning the war. If you got excited about crypto stuff in the DaVinci Code then you will have lots of hours of fun trying to work through the examples the authors provide. (And my thanks to my original writing mentor, Grant, for turning me onto the book.)

The book's author and organizer is Jack Copeland, who is a New Zealand professor and responsible for the archive. Turning, a name familiar to many of us, was intimately involved in many things Bletchley Park but not the brains behind the hardware of Colossus.

Second, owing to an odd set of circumstances, all but two of the machines were immediately destroyed after V-E day in 1945 on Churchill's command. Had more evidence survived, it might have become as notorious as the American ENIAC, which has gotten most of the credit for the early computing innovations. The two units were used by the British Government for several years for code breaking purposes, presumably. And recently, a project to rebuild one machine has been successful, and was tested with breaking some historic codes.

Third, the veil of secrecy around this project was colossal, too: despite the thousands of people employed during the wartime effort, there were few leaks. Maybe the Brits are better at keeping secrets than us Yanks, but the amount of containment and compartmentalization was austere. Thousands of people worked at close quarters yet never asked each other what they were actually doing on the project. Parents didn't have idea what their children were working on – indeed, one of the younger Colossus team members was so concerned that all of his contemporaries were off fighting (and he was just a mere geek-in-residence) that he asked to be reassigned because his father was beginning to wonder what he was doing. His superiors arranged to have an army officer visit his parents and reassure them that the son was involved in a major military project, and just because he wasn't shipped to the front lines he was still a key part of the war effort.

Despite the secrecy, engineers kept a few pages of design notes about the project and now a group of academic researchers is using this information to try to reconstruct Colossus and the German encryption devices it managed to break. There are exhibits at Bletchley Park as well, someday I hope to see those in person.

There are some modern lessons to be learned from Colossus:

"The only dependable way of protecting corporate and government computer networks today from the criminal trespasses of hackers is to hire from their top echelon on the principle of fighting fire with fire," ways Donald Michie, one of the younger code breakers at Bletchley park. He talks about how the code makers and the code breakers were never allowed to communicate and indeed were two separate government organizations, thus serious flaws were never fixed on early encryption machines. Many of these flaws introduced incredible human errors that allowed the Brits a way into decrypting the German transmissions.

A second lesson is how critical Colossus was towards the overall war effort. Many Americans like to think that we played a key role in winning the war, and certainly without our men and material things would have happened differently. Yet, the book describes the moment that Eisenhower changed his plans for the D-Day Normandy invasion based on the Colossus decrypts. Indeed, one of the Colossus machines was finished just hours before he had to make this decision. The decrypts made it possible for him to monitor German counterstrike efforts before he attacked the beaches and place his troops accordingly. There are several other examples throughout the book about other moments where Colossus saved lives – including German ones, too.

Third, reading the descriptions of the Colossus team and their exploits, I got the feeling that Bletchley Park was a very unique place in Britain's military history. People were addressed on a first-name basis, and civilians and soldiers worked side by side. Many of the people employed there were just out of high school, if that, and the team leaders had to find motivational ways to keep the place running 24x7. There is a lot to learn from in these sentiments for anyone who manages a team of people.

Finally, there is also an understanding the role of secrecy in an open society. I don't think anyone would argue that Colossus needed to be a secret during wartime. The issue is how it was revealed to the public, and how long it took before the exact details were openly available. Given that the events of the 1940s are just now coming to light, perhaps we need to keep this in mind when we think how the innovations we create during our modern era can find their way into the future public domain.

I highly recommend this book, and hope you enjoy it as much as I did.

No comments:

About Me

My photo
David Strom has looked at hundreds of computer products over a more than 20 year career in IT and computer journalism. He was the founding editor-in-chief of Network Computing magazine, and now writes for Baseline, Information Security, Tom's Hardware, and the New York Times.