Abstraction, Redundancy & Margins

 

Murphy's law, simply stated, is that anything can go wrong probably will. Not will, but probably. Therefore any simplistic reliance on one technological plank to underpin a complex system is, also quite simply, asking for trouble. And yet it would seem this is exactly what a significant number of corporate/government entities have been engaged in for some considerable time, and yesterday the chickens not only came home to roost, they crapped all over the coop and then trashed it.

Amazon, Ebay, Boots and even the bloody UK government's web presences were caught with their collective trousers down by an outage in the cloud server platform they all appear to solely rely upon for their services to function. The outage was short admittedly, but it was absolute, and no backstop appeared to be in place. How on earth can these people employ critical systems so bereft of one of the most basic cornerstones of engineering: redundancy. Never, ever put all your eggs in one basket. Critical systems 101: don't do it.

If NASA had taken this naïve approach back in the sixties, the poor sods engaged at the sharp end of the Apollo 11 mission would have ended up dying in deep space. Even though by today's standards, the flight computing power was infinitesimally small - a modern smartphone has more firepower than all of NASA's land-based computers at the time put together - they put several identical units on board, working in tandem: even if all but one failed - statistically very unlikely - the mission would continue safely.

Today we deal with and rely on systems of unimaginably greater power and complexity to simply run our everyday lives, and yet it appears that some of the corporates and institutions who we rely on for our most basic needs are operating zero-redundancy systems that rely solely on a single cloud computing platform for their operations.

As always, it seems that a lack of strategic oversight, reactive management and an inability to abstract out a system before deploying/employing/purchasing it hold sway: the PR spin, the advertising spiel and corporate hubris all conspiring to convince them that 'nothing could possibly go wrong'. It did. Always will do. The ability to sit back and abstract out an idea long before any specification, tender or purchase order are even thought of is essential. The great engineers of the Industrial Revolution could not have built any of the things they did without that prior thought. They were mavericks, no doubt, but their engineering was thought through and tested empirically, and most of their stuff still exists to stand testimony to that.

The picture above is a frontispiece from James Nasmyth's book on the Moon (1874). He reasoned that he could more accurately represent the geographical features of the Moon - even given that it was technically feasible with current technology to photograph it directly at the time - by abstracting his detailed telescopically-derived drawings, combined with time/shadow/trigonometrically-calculated topographical detail onto plaster realisations of the Moon's surface, and then photographing those models, rather than photograph the Moon directly. The results speak for themselves: no better representations of the Moon's features existed until the Apollo missions of the 1960s and early '70s.

There's lessons to be learnt from all of this. Don't put line-managers, HR, PR or bean-counters at the forefront of your decision-making process: please leave it to the professionals, and while you're at it, pay them handsomely for preventing the shit-storm you would otherwise have created.