In a recent talk by John Ousterhout called "A Philosophy of Software Design" he asked the audience what they thought was the most important concept in the history of computer science. It was the answer from Donald Knuth he quoted that struck a chord with me. Knuth answered, "layers". It took me a while but over the past years I came to realise that the concept of layers of abstraction has been one of the, of not the, most successful concepts in computer science. By abstracting hardware by machine code by assembly by imperative languages by OO languages, we were able to build information systems of truly mind-boggling complexity. It enables developers to stay within one abstraction level while designing and implementing an information system. Since lower levels were abstracted away developers had more brain cycles to spend on the abstraction level of their choice. It is the job of a compiler to turn higher level into lower level expressions. Although compilers will never be able to catch every error a developer introduces, their use will prevent the vast majority of mistakes they would have made if they had to write the lower levels by hand as well. There are a number of interesting things to note about this idea of layers.
First of all the it seems that we have been stuck at the current abstraction level way too long. Despite efforts such as 5G languages we got stuck at the OO-level since the early nineties. As people like Uncle Bob mentioned, it seems as if we got distracted by the internet and forgot to continue the great work our predecessors did in building all the abstraction layers we came to take for granted. Now that we realise that OO has its serious shortcomings as well, it seems we are kind of clueless where to go next. The most promising area I see is the domain of executable business process (see for instance Zeebe). Despite the fact that these efforts have been met with a certain dedain from part of the software community. The most important difference with previous abstraction levels is that this approach starts top-down instead of the regular bottom-up. This could very well be the only way to get us out of the current impasse.
Secondly I have noticed that the mixing of abstraction levels is one of the biggest reasons for confusion in any discussion involving software technology. And they often get mixed since people feel comfortable at a certain abstraction level and will, often unconsciously, direct the conversation to that level. Discuss a technical business challenge with a DevOp and you will be talking about Kubernetes, Terraoform and AWS VPC's in no-time. Convincing similar examples the other way around can be thought up just as easily. Realising this I have found that the best professionals in our trade are the ones who feel comfortable at every level of abstraction. From high level business and even philosophical topics to low level technical details. Professionals that can go up and down the stack without losing oversight or getting stuck in one abstraction level while debating certain aspects.
A third interesting aspect is what I think will be the most surprising revolution in the way we build information systems: AI as the compiler. I have written extensively about how I think AI is destined to become the compiler between the highest abstraction level and the highest implementation abstraction level. You can read about it elsewhere, but it is the notion that this is about creating abstraction layers that make it relevant for this post.
Thinking, designing and building with a clear notion of layers in your head is key to building efficient information systems. It is a lesson I found so important that I have based the name of my next company on it: Stekz.