Software Engineering and the Scientific Imagination: Past, Present, and Possible Futures, or, How to Stop Building Faster Horses

My forearms tingled. “Press your palms together with your fingers pointing up,” he said. This is a common test for repetitive strain injuries that my father was at this point well versed in, having spent nearly thirty years writing software professionally. As my palms struggled to meet, a searing pain shot from my pinky and ring fingers on both hands up the outside of my wrist towards my elbows. “FUCK! And I was just starting to really enjoy emacs!” I cursed.

This pain is the early stages of cubital (not carpal) tunnel syndrome, also known as ulnar nerve entrapment, and I had experienced it before. Long hours spent hammering away on essays at the last minute for a media studies class in my second semester at the University of Rochester had caused me to develop a mild case of ulnar claw in addition to the pain, numbness, and tingling. I spent several weeks sleeping each night with my right arm outstretched perpendicular to my torso, pinky and ring finger curled towards my palm involuntarily, trying to avoid compressing the ulnar nerve in my elbow and causing excruciating pain.

As I sat wondering whether my new favorite tool was going to eventually injure me so badly as to require corrective surgery, my dad mentioned that Xah Lee had put a lot of effort into making text editors (specifically Emacs) more ergonomic. But I wondered why it fell upon an individual who was disabled by work-related injuries to quest after fixing the problems that caused them and not the institution of software development and computer-based work more broadly that causes these injuries. It doesn’t have to be this way. As workers with college degrees who are often paid well to the extent that we ourselves sometimes do not think we deserve the sheer magnitude of our compensation, we are prone to ignoring those aspects of our jobs which should be considered unacceptable and need to change. Though not everyone shares such a pessimistic view, it is hard for me not to see the institution of software development as a massive farce, shambling forth on the limited strength of its own hubris and greedily consuming the energy and passion of those cursed to love it while crushing their bodies and spirits in its never-ending sprint for the novel and innovative.

Emacs is one of the few truly great things to come out of software development in the last 50 years. It’s an infinitely flexible environment for manipulating text of any kind, from emails to webpages to computer programs to chatrooms to newsfeeds and Twitter and poems and essays and love letters and so on. It is my belief that any serious typist or computer user could benefit from learning it, not just programmers. At the same time, it’s hard to sell the benefits of software dating from the latter half of the Ford administration and which is famous for requiring knowledge of arcane and sometimes literally painful keyboard shortcuts which themselves risk causing RSI in some users.

There is a broader point in this history and bellyaching that connects to some of my other passions for software correctness and verification. In the year 2018, we as software engineers and computer scientists of all levels find ourselves falling in love with tooling and practices which, in the relatively quick lifecycle of empirical science and especially the software development industry, can only be considered ancient at this point. Why is it that better tools haven’t been developed?

In what is seemingly a rare flash of insight into the actual failures of software development, Rob Pike needles the stagnation of systems software research in a presentation from February 2000. This piece, entitled “Systems Software Research is Irrelevant,” tracks the failure of computer science theory to generate useful ideas and tools for improving the practice of software engineering, and as such represents the first time I find myself nodding in rigorous agreement with Pike, whose language Go (in cribbing liberally from C) often seems a testament to the principle of tradition for tradition’s sake. Lambasting the homogenization of his field around a few-good enough tools and the monoculture that results from them, he writes:

In science, we reserve our highest honors for those who prove we were wrong. But in computer science…

There are a few broad categories of flaws I think are worth discussing when it comes to the failure of software to make use of empirical research that would help develop it into a true engineering discipline. The main two are the technological and social elements, roughly specified as the ways in which current tools and processes inhibit our ability to produce correct and reliable software of all kinds in a timely and repeatable fashion, and the management practices and structures and devices which bridge our access to these techniques. In the former category I would put things like programming language theory and type theory research, as well as concomitant research in verification of hardware designs and novel technologies in the field of human-computer interaction. In the latter falls things like project management strategies, communication systems, documentation, education, and tools for managing the whole process and lifecycle of software.

Condensed, software development is composed of the technologies that directly make up completed systems and the systems of human social interaction that surround and support those systems. I want to go into much greater depth in both of these areas, exploring the depths to which our tools, ideas, and ideologies fail us in not being able to keep up with the ever-increasing demands of the free market upon the bodies and minds of the working developer. While we may not be doing heavy manual labor, the structures of capitalism are still more willing to extract a devastating toll on our bodies and minds rather than treat us humanely and invest in solutions to ensure our safety.

to be continued…