Procedural CGI & the semantic programming parallel

One of my main interests within the realm of the digital revolution has been the progression in the craft of software engineering. The craft has moved to a higher abstraction level over the past, say, 70 years. Broadly speaking from machine code to assembly to C-like languages to OO-languages to the current crop of high level scripting languages. Although there have been many variations along the way, the main trend has been that the craft of programming moved to higher levels of abstraction. Instead of handwriting zeroes and ones software engineers now have languages and tools that allow them to specify for instance classes and inheritance and let the compiler turn that into machine executable instructions. As I've written before one of the questions that have been puzzling me over the past few years is why this progress seem to have stalled in the 90's. At that time the industry had quite a lot of experience with higher level languages such as COBOL (!) and there was a lively discussion surrounding 5th generation programming languages. But then the web happened and all attention was directed at writing as much software as possible to explore this vast new realm of undiscovered opportunities. Somehow that seemed to lure all attention away from fundamental progress towards the latest greatest in web development. Everybody was in a hurry to explore this new world and technologies were invented, and often reinvented, at neck-breaking speed. Today's most widely used programming language, JavaScript, was for instance developed in just 10 days at that time.

There is a problem with this stagnation, though. The demand for new software is rising so fast that there simply aren't enough humans to do the job if we stick to the labour intensive abstraction level the industry is stuck at at the moment. There is already a huge shortage of software engineers. It is actually one of the main limiters of growth in Western industries. 
But there is another problem. The low level work is prone to errors while at the same time the importance of the information systems for society at large is increasing. In other words, our lives depend on software in an accelerating rate while at the same time we ar developing the software in an archaic and error prone way. The only way out of this situation is to move the job of software engineering to the next level of abstraction. This means that domain experts should be able to express exactly what they want after which a compiler generates the information system that does just that. This has been called 'semantic programming' in the past, and it is in my opinion the only idea to move the software industry forward and prevent a lot of problems. Fortunately there is light at the end of the tunnel. In a recent post I mentioned my idea that AI seems to be the surprising candidate enabling the step towards the next level of abstraction. While in traditional information systems developers tediously wrote detailed instructions how to mangle given input to generate the required output, with AI developers just specify the required output based on a certain input and let it figure out the intermediate steps itself. This is in line with the ideas of semantic programming.
Interestingly enough there is also light at the end of another tunnel that has strikingly similar properties. After diving into the world of computer graphics (CGI), game development, AR/VR, and animation I realised that this was an industry where the exact same type of transformation was taking place. Early CGI work consisted of painstakingly typing out coordinates to, for example, draw a wireframe of a doll. Subsequently tools were developed to allowed artists to draw with a mouse and automatically tween between keyframes. Over the past decades the tools became increasingly more sophisticated, especially in the area of the notoriously complex world of 3D CGI. Each step gradually freeing the artist from tedious, repetitive tasks and allowing them to focus on expressing what they actually wanted. Interestingly enough one of the biggest trends in the CGI industry is the move towards proceduralism, where things like textures, geometry and even complete worlds are generated by procedures instead of by hand. Take a look at how the 3D modelling (if you can still call it that) software Houdini has been used to procedurally generate environments for the latest episode of the Farcry series. The artists of Farcry no longer have to draw every leaf or every road by hand. they specify the properties of their required worlds at a high level after which the algorithms of software like Houdini generate it. You could say that software like Houdini is becoming the compiler, just as AI is becoming the compiler for information systems (as previously discussed). The drive towards proceduralism in the CGI industry is the wish to focus on the high level picture and not on the low level details. But also by the need to create increasingly more complex worlds for which it would be impossible to build by hand.
I find this parallel intriguing. (Somehow my brain is wired to relentlessly look for parallels between separate phenomena.) Understanding the underlying driver enables us to see where our industry, work and tools ar heading. And it gives the creative mind a glimpse into the future.

Questions and ideas

I might have answers, but I sure have questions!
I've been thinking quite a lot about what I should trust to this blog and what not. I don't want to share too much personal stuff for obvious reasons (in case you wonder: we had social media for that awfully wrong, besides if you want to know me more personally I suggest we meet up in RL). Neither do I want the blog to be nothing more than a place where I only forward ideas of others (again, we had social media for that….). And about the topics, I like techy stuff, but as most of you know I'm a firm believer in the ecological approach towards handling a topic. Meaning that something without context has no meaning, so the context must be included. Even if the topic is highly technical. All in all I came to the conclusion that I want this blog to be the place where I share both my questions and ideas with regard to the broad field of the digital revolution. I honestly think that enforces a focus and keeps it interesting for readers while still being worthwhile for me as a place to ask my questions and dump my ideas. So from now on I'll ask myself the question if a new post is about one of my questions or ideas (or both). Ping me if I start to slack! 🙂

Run Jupyter notebooks on hosted GPU for free: Google Colaboratory

I've been refreshing my AI skills lately since they were a little rusty. After getting my master's in AI in 1998, during which the historical victory of Deep Blue over Kasparov took place, the next AI winter set in. For about 15 years AI was about as sexy as Flash is now. But the well known advance in processing power and the staggering price drop of storage AI, and specifically machine learning, became viable. Actually at the time of writing machine learning specialist is probably the most sought after specialism. So it was time for me to get back into the AI game. But mostly to cure my curiosity what the new state of the art was actually about and what was new that could be done. I mean, over the past years we've seen self driving cars, robots, algorithms for social media, all kinds of medical applications, etcetera etcetera come into being solely because of the advances in AI and the needed hardware/software infrastructure. In other words, the AI playground was revived and full of enthusiastically playing children. First thing that surprised me is that although there were a lot of refinements but not many fundamental new technologies. Of course, you run stuff in the cloud, there are more neural net variations, and the time it takes to train a network has decreased dramatically. But the underlying principles are still largely the same. Another thing didn't change were the hoops you have to run through to get your stuff up and running. It was a bit like the current web development situation (those in the know will nod their heads in understanding). Instead of focussing on the algorithms you spend most of your time trying to get your toolset installed and build/deploy street up and running. And that's a bad thing. Fortunately I stumbled across a hidden gem called Google Colaboratory. It's a free service that let's your run Jupypter notebooks on a hosted GPU….for free! If you want you can store the notebooks themselves on Google Drive, or, if you don't want that load them from elsewhere. That is quite amazing and an amazing boon for those that want to get up and running with machine learning as soon as possible. The amount of resources you get for free is, of course, limited, but it's more than enough the experiment and design your data processing pipeline and design, train and test your models. Once your content with your trained model you can take it to more beefy hardware (in case needed). Or to train it on huge training data sets. All in all quite an amazing service that will benefit the machine learning community a great deal. The nice thing about Jupyter notebooks is that you can take them elsewhere and run them there. You are in no way tied to Google, which is a good thing.

AI is the new compiler

What I think is one of the most interesting trends of this moment goes unnoticed by the general public, but surprisingly also by the majority of software professionals. Now that the latest AI winter is over (there have been a few in the past) and every self-respecting information system has at least a bit of AI in its bowels, the way we design and implement information systems is drastically changing. Instead of programmers writing down exact instructions that a computer must execute, machine learning specialists specify and modify both the input and output (I/O) of the system and leave it to AI to find the algorithm that does so on a training and testing set of data. (As a side note, the latter is important to realise, the exact algorithm is very hard to understand and so are predictions of it on unknown input data.) AI thus compiles the requirements into an executable algorithm.

I have been interested in ways we could improve the way we design and implement information systems for a long time. It is my belief that the current state of the art in software engineering is temporary and that we must move on to improve our profession for economic, safety, and societal reasons. Our current way of working is very cumbersome and leads to a lot of problems while at the same time software is controlling an ever increasing part of our daily lives. There is a long and very interesting history of software development from the 1940's up to the present day that I have been following for many years. If there is one continuous thread I would say that it's the fact that the software engineering profession keeps moving towards higher levels of abstraction. From machine code to assembly to C to C++ to Java to 'scripting' languages. But it seems the industry got stuck somewhere around the beginning of the 1990's. When the industry exploded with the advent of the web, so did the number of tools, but none of them was at a fundamental higher abstraction level than languages such as C. So what we got was basically more of the same. Unfortunately 'the same' turned out to be not good enough for our fast changing world. We simply don't have enough software developers to keep up with global demand while at the same time the stakes are rising for every piece of new software that gets released. Admitted, there has been research into more 'semantic' programming languages, but none of them left the academic realm to conquer the software industry.
With the advent of AI something interesting happened though. As said traditional software engineers writing down machine instructions are slowly being replaced by machine learning specialists selecting the right estimator and specifying the I/O. The latter work on higher semantic level, they are concerned with the properties they want the algorithm (system) to have, not about the instructions the machine should execute. This is a fundamental difference that fits neatly with the long historical trend of the software profession moving towards a higher semantic level. Of course the machine learning approach does not fit every use case, there will still be many systems that must be specified procedurally (or functionally if you prefer so), but the variety of use cases for machine learning surprised me (even as an AI veteran).
With machine learning specialist being the most sought after profession at the moment it is my guess that this trend is just picking up steam and we're only starting to scratch the surface of what I believe to be a fundamental change for the software industry at large.

Hype cycle considered harmful

As mentioned in my previous post the downfall in popularity of blockchains is considered to be perfectly in line with Gartner's well-known Hype Cycle. But there are so many fundamental problems with it that it's dangerous to assume a bright future of your technology of choice solely on applying the hype cycle 'theory'. The most obvious one, as pointed out here as well, is that by far the vast majority of technologies fail, despite the fact that they went through a period of hype (being aroused interest). After the hype they just die. It is incredible naive to think every technology will pick up steam again after it has lost interest from the general public. Besides that many successful technologies never go through a 'through of disillusionment'. Just as many technologies never become a hype while still becoming extremely successful in the long run. Sure, a few technologies followed Gartner's Hype Curve perfectly, but most didn't. So predicting every technology will follow the curve is misleading and could be harmful.

Blockchains: It’s not even funny anymore

Halfway 2017 I wrote quite an extensive article about our adventure in the world of blockchains. By that time my colleagues and me had spent over three years deep diving into both the technical and societal aspects of blockchains. How we went from awestruck by the sheer genius of Sakamoto’s idea to the disillusionment in the applicability, the engineering standards, and the cult of ignorance fostered by the ‘blockchain visionaries’. We built companies, products, communities and technologies to get this blockchain revolution going, but we ended up concluding that the only way you could earn a living with blockchains is by either talking about it or building proof of concepts. So we spent years trying to tell the truth about the true potential of blockchains but we were regarded as party poopers. The audience simply didn’t have the technical frame of reference to understand why most use cases would never work. Their response basically boiled down to, “All fine and dandy all this technical mumbo jumbo of yours, but look at all these billions and billions and billions and billions and billions and billions and billions of dollars spent on it. and all these incredible nerds, and all these successful entrepreneurs. They can’t all be wrong, can they?”. Read all the details in the article but in the end we abandoned ship and never looked back. Over the past years I haven’t been following blockchain news, I haven’t visited conferences, I haven’t talked to companies that wanted to ‘do something with blockchains’ (I spent my time reading books and learning skills in an area that actually does have a future, AI). In the meantime blockchains have apparently entered the mainstream. It’s on the news, it’s in the newspapers, it’s on the website of every big corporation and it’s in at least one slide of a presentation of some middle manager.


But a change has been brooding over the past year in the technically more advanced circles. As is often the case the ones with a thorough understanding of the subject matter are the only ones that can grasp the true potential of a technology. (This insight was for me personally the reason to complement my alpha master’s with a beta master’s and PhD.) The word ‘blockchain’ nowadays results in a quirky smile on their faces. The result of having heard so many stories from uninformed ‘visionaries’ that the revolution is neigh and everything will change that the only reaction to so much naïvety they have left is to laugh. I personally got so fed up that about a year ago I presented my tongue-in-cheeck game “Berco Beute’s Blockchain Bullshit Bingo”. On the bingo card were all the cliché statements every apostel of the church of blockchain uttered. For example:
–  It’s going to disrupt EVERYTHING. It’s following the Gartner hype curve, but it will be successful in the long run.
– We don’t need trust anymore.
– Nor trusted third parties.
– Bitcoin will replace fiat currencies.
– Etcetera…
So the word ‘blockchain’ and its cult following are slowly becoming the laughing stock of the industry. Mind you, I’m not talking about the technologists working on it simply because they love to work on interesting technical puzzles. I’m talking about the ‘visionaries’ not bothered by knowledge that fail to apply some modesty to their behaviour. At the time of this writing even mainstream media start to question whether blockchains could ever fulfil all the promises they have been attributed by the visionaries. Yesterday the Dutch newspaper De Volkskrant published an extensive writeup about the failure of all the blockchain projects to come up with truly viable solutions. And even the projects that lived past the ‘pilot’ phase are so simple that most experienced software professionals would agree they could be implemented much faster and cheaper with other technologies.
Although this recent trend makes for some good laughs and a few told-you-so’s, we shouldn’t dismiss it so easily. Not because the promise it still holds, but because the damage it has done. “Damage it has done!?”, you might react, but yes, let me explain. The blockchain is a hype that deserves its own category solely by the sheer amount of resources it has consumed of the past decade.
1. Energy. The proof-of-stake algorithm by now consumes about as much energy as Ireland. Or roughly twice as much as mining copper and gold. Combined. What we get in return is bitcoins, a mysterious entity that’s neither money nor gold, but has proven itself to be an effective way to buy nerds around the globe a couple of Tesla’s (each).
2. Money. Companies, investors and governments have invested billions and billions of dollars in blockchain technologies of the past years. A truly staggering amount of money has flowed to startups, ICO’s, consultancy companies, schools, etc. The awkward aspect of this is that blockchains were invented to create money (bitcoin), not vaporise it.
3. Brainpower. Likely even bigger than the energy consumption has been the brainpower that got sunk into the intellectual blockchain blackhole. Sure, there have been a few technological innovations but that pales in comparison to the amount of intellectual effort that led to nothing. The buzz, billions, technological marvel attracted most of the greatest brains of our era.
Think about it, these resources combined could have been invested in healthcare, fundamental research, battling climate change, fighting cancer, and many other truly important causes. But instead it was spent on a pipe dream. A troubling conclusion for which I think those responsible should be held accountable. All these so-called visionaries that got rich by misleading the general public with smoke, mirrors and visions of the promises land should repay the societal damage they have caused. Although I realise it is highly unlikely this will ever happen, I do think it’s important that this message gets out. As I said, it’s not even funny anymore.

Voice interface hype

Despite all the buzz surrounding smart home assistants such as Google Home, Amazon Lexa and Apple Siri, I still think the expectations with regards to voice interfaces should be scaled back. This Venturebeat article has some nice background reading. For instance, we’ve had voice-to-text for decades now and still almost nobody uses it. Who do you know that dictates his message? And if you know someone, is he/she the odd one out or one of many? While a few decades ago you could still blame the inferior technology for the low adoption rate, nowadays speech recognition has become so good that that’s no longer valid. And still almost nobody regularly uses a voice interface. And to be honest, I don’t think it will ever be really successful. And there is a very simple reason for that. When interacting with information systems users are mostly in a situation where talking is not very handy. They are in company of others that they don’t want to disturb, or they don’t want eavesdroppers, or they’re in a noisy surrounding (public spaces), or they simply want to take some time to order their thoughts before blurting them out. If you think about it there are actually not many situations where users would feel comfortable talking out loud to a computer.

Just as author Neal Stephenson had to point out in his insightful 1999 book “In the Beginning was the Command Line”, don’t just throw away old interface paradigms when a new one comes along. You might misunderstand what made the original paradigm so successful. The same goes for the envisioned switch from text to voice interfaces. To paraphrase an old saying, “A wise man once said….nothing…..but typed it in”.