Saturday, February 20, 2016

Distributed Systems and Orchestral Music

Writing for distributed systems and composing orchestral music

... too long for a tweet, too short for a real blog ...

Composing music for a single instrument, a la the Bach violin partitas or the Beethoven piano sonatas or the Berio sequenzas, requires expert knowledge in the targeted instrument. The composer must understand the dynamic range, the various timbres, the harmonic resonance, and many other facets of the instrument to create a truly compelling piece of music.

Similarly writing expert-level software for a single machine requires one to understand things like disk IO, CPU cache registers, kernel tuning, and the various ways in which those systems building blocks can be efficiently employed together (composed!) to create a useful piece of software.

Moving out to the next level of abstraction, whilst composing music for an orchestra, the composer not only needs to understand each individual instrument, but also how to assemble them together to contribute to the braoder musical goal. For example, listen to the Mahler symphonies or the Stravinski ballets (The Rite of Spring, and so on). In designing a distributed system, similarly, the engineer needs to understand how machines behave at the local, individual local, but also how the sum of them contribute to the overall distributed system algorithm. Faulty disks, long virtual machine pauses, or other failures at the individual machine level, if not handled properly, can cause the distributed algorithm to fail or misbehave in unexpected. . . .

To expand the subject space further, one could conceivably compare multi-datacenter distributed systems to something like Stockhausen's Gruppen or his Helikopter-Streichquartett (https://en.wikipedia.org/wiki/Helikopter-Streichquartett). Although, at this point the anaolgies start to break down, and there aren't enough data-centers-in-deployed-helicopters to have a fair comparison. Music rules.