It was this recent article AR will spark the next big tech platform — Call it Mirrorworld by Kevin Kelly in Wired that made me want to shout from the rooftops to everybody that ever had to listen to my ramblings about technology, "This is it! This is what I've been been talking about and working on all the time! This is the dream I've been pursuing!". This article tries to give you all the necessary background needed to understand why this is relevant for both you and me. It is a somewhat personal post explaining where my ideas about what is called a 'mirrorworld' come from. And how my professional career has been modelled around that idea. It should give you a better understanding of the idea itself and of my past and future personal journey. Albeit it being about my professional interest there is no denying that it is my passion for the frontier of digital technology and my relentless curiosity got me where I am now. So please forgive me if this reads as being too much about me personally. I think in this case you have to understand not just the idea, but also my personal journey and passion to assess the value of my opinion. In this post I will try to explain how I ended up where I am now and where I will be heading. I hope you enjoy reading it and become inspired yourself.
In augustus 1991 I started studying mass communication at the Universities of Groningen (and later Nijmegen), the same month Tim Berners-Lee posted a short summary of the World Wide Web project on the alt.hypertext newsgroup. I'm not sure if that is a coincidence, ideas almost always pop up around the same time (as Matt Ridley argues in his interesting 2015 book 'The Evolution of Everything: How New Ideas Emerge'). My main interest was how mass media worked. How do they get the message across? Do they color the message? What effect do they have on popular culture? Theories by visionaries such as Marshall McLuhan intrigued me deeply. Being a musician I was especially interested in the how the medium influenced the music it carried. This was before the web became widely available so the latter question was limited to media such as radio, television and tapes exchanges. Probably the most prevalent insight that occurred to me is that the influence of the medium on the message is most of the time grossly underestimated. To understand why this was the case and how this worked required a deep understanding of the inner workings of the medium as well. This became especially obvious while interviewing experts for my thesis who could broadly be divided in technical and non-technical experts. The former had a much better understanding of the medium, where it came from and where it was heading, then the latter.
Having been interested in computers since childhood I was one of the first to pick up on what were called 'electronic services' (think BBS, Minitel, videotext, teletext, etc) and this new buzzword 'internet' (or 'electronic super highway'). Hanging out on BBS' and learning about this new thing called 'the internet' I became intrigued by the idea of a super powerful, new mass medium. It triggered my curiosity and creativity in a way I seldom experienced before. Envisioning a fully digital network that connected everybody with everything I started wondering what messages this medium would transmit, and whether it would color the message, and how this medium would actually work. But there wasn't much interest in it at that time and place. I remember talking to a pop-culture professor in 1993 proposing to research what the influence of this new medium (the internet) would be on popular music. How would music be distributed through such a medium? Would it have influence on the music itself? What would it mean to the music industry. In an in hindsight shocking response the professor said he didn't see much fruit in such a research. It was still unclear whether it could ever take off at all. Remember, it was still very early days in internet time and most people had never heard of the internet. That didn't stop me though, I continued my studies and wrote a thesis about the possibilities of turning the broadcast television cable network into a bi-directional network fit for those newly found 'electronic services'. In other words, how viable was the idea of turning an old style mass medium into a new one. This was the research that convinced me that to truly understand the medium you had to grasp the underlying technology. A lesson I still hold dear, your understanding of a phenomenon is only as deep as the understanding of the underlying system that causes it.
With that in mind the best thing I could do after graduating was to spend the extra time I had on developing my technical/digital skills. Skills that I would need to better understand this new medium and be able to contribute to it. So in 1995 I started what was a new study direction at the University of Groningen called Artificial Intelligence. It was a technical study but with a broad view on intelligence. It offered courses in computer science, biology, language, physics and math. Basically everything needed to truly understand how the brain works, what intelligence is, and how that could be simulated in computers. A wonderful study in an era where nobody had any idea what to do with it, the world already had great chess computers. Then on May 11th 1997 IBM's Deep Blue computer beat the chess world champion Garry Kasparov. I still remember our class being all excited during a lecture the next day. This shocked the world and made the study AI hip again, although it would take over 15 years before there was enough data, processing power and memory to lift AI over the usefulness threshold and make it the runaway success it has become nowadays. Meanwhile my interest in what had become 'the web' only grew deeper. I took the technological deep dive to truly understand how this new medium worked under the hood. And, true to my own mantra, to understand meant being able to do. This meant writing lots of software, studying network architectures and protocols, talking to experts, joining open source initiatives, and so forth. Slowly it dawned on me that this new medium (the internet) was not just a medium between humans. Senders and receivers could be appliances as well. Or just software processes. Any process that could send or receive a message could use the medium. That made the medium infinitely more useful (and complex) than old mass media. The most important question this raised was inspired by two books I read. William Gibson's 'Neuromancer' from 1984 and David Gelernter's 'Mirror Worlds: or the Day Software Puts the Universe in a Shoebox…How It Will Happen and What It Will Mean' from 1991. While Gibson took the creative approach Gelernter, probably inspired by Gibson, took the scientific approach in playing with, and wondering about, a virtual world that exist next to our real world. Possibly overlapping each other. Gibson invented the term 'cyberspace' for this world while Gelernter coined the term 'mirrorworld'. Both books made me realise that this new medium connecting every possible process, whether software or hardware based, created some sort of virtual space where users could find and connect with every other possible user/service/process. Where Gibson envisioned a virtual world that was detached from the real world, Gelernter deepened the idea and talked about a virtual world in which the real world was represented. Hence the name 'mirrorworld'. Again, to understand what such a mirrorworld would look like and how it could be built, I forced myself to dive into all related technologies and become proficient in using them myself.
An additional passion and skill I picked up during the nineties was 3D computer graphics. This truly caught my imagination and I spent many midnight hours learning myself arcane 3D modelling applications. It literally opened up new vistas. This newfound passion and skill turned out to be very useful for my interest in cyberspace/mirrorworld. For humans understanding such a conceptual environment could become a daunting task in which 3D computer graphics, and especially virtual reality, could turn out to be useful. Although not at all user friendly, virtual reality appeared on the scene during those years and immediately convinced the technologically more advanced audience that one day it would become good enough to open up a whole new world of possibilities. My AI thesis was about the question what such a mirrorworld would look like, how it would work, how physically reality would be represented in the mirrorworld, and what role AI could play in constructing a mirrorworld.
The topic was so big that my research spilled over into a subsequent PhD project at the DTU (Danish Technical University). The question I addressed for my thesis was how something like a mirrorworld would work, could be built and could be made understandable for users. During my PhD project I got all the freedom I needed to explore every possible angle in thinking about and working on a mirrorworld. One of the most inspiring books I studied during those years was Steven Johnson's 'Interface Culture: How New Technology Transforms the Way We Create & Communicate' from 1999. Johnson argues that the interface we use to access virtual services deeply influence the way we create and communicate. Much in line with Marshall McLuhan's idea, and directly applicable to my interest, this new medium called the internet and the resulting virtual space called mirrorworld.
One of the most inspiring technologies I got involved with during my PhD period was Jini from Sun Microsystems. Computer science legend Bill Joy gathered some of probably of the smartest persons I've met and started working on 'a network architecture for the construction of distributed systems in the form of modular co-operating services' (a spot on quote from wikipedia). Jini's introduction in 1998 was accompanied by many references to the ideas of William Gibson and David Gelernter, but also to their predecessors (Vannevar Bush, Ted Nelson, Doug Engelbart, Alan Kay, and many others). But Gelernter's ideas of (the implementation of) a mirrorworld were undeniably the most influential on Jini. The Jini team had undoubtedly the biggest influence on my ideas on how a mirrorworld could be built.
It was a truly inspiring time in which I deepened my understanding and widened my skillset. Still, after finishing my PhD this expertise was of only limited use. There was simply no mirrorworld in sight since the technology wasn't ready, it could not live up to the initial hype. The VR hype was over and the enthusiasm about AI was slowly dwindling as well. The Jini project slowly disappeared from the stage because the hardware was nowhere near ready for the type of use cases it was meant to address. Besides, the industry had to recover from the dotcom bubble, the year-2000 'bug' made the general public weary of big statements about technology and the rise of mobile devices grabbed all the attention. I figured that since a mirrorworld was still a bridge too far, I'd better spend my time on technologies that were related. Since mobile devices seemed like a natural part of the trend towards a mirrorworld I decided to join a research institute (TNO) and focus on that. Starting with C on Psion's, going through J2ME up to the introduction of the first iPhone. Those projects involved a lot of innovative software, including the first versions of cloud computing and social media. But after the initial innovative phase was over those technologies became business as usual and I lost my interest. Been there, done that, and I started looking for something else to work on. Something a little less bureaucratic and with different technological challenges.
After leaving TNO I joined an online ticketing company called Paylogic. Not that I'm super interested in ticketing, but the technological challenges they faced were in line with my personal interests. But as I learned over the years, the human aspect of making software is almost as important ast the technical. Some of the most interesting challenges I had to tackle at Paylogic had to do with questions how to build a high tech team, how to choose the right technologies, how to grow a business, how to get VC, etcetera. After Paylogic was sold innovation became of secondary importance so it was time for me to move on.
Around 2014 some of the necessary technologies for a mirrorworld were becoming mainstream. Most notably everything related to the Internet of Things (IoT). So I started an IoT company called XIThing with a couple like-minded entrepreneurs. This kicked off yet another interesting period. Although the IoT predictions from established research firms were booming, the general market simply wasn't ready yet. As always short term predications were overrated while the long term predictions were underrated. We worked on some truly interesting and innovative architectures for networks of sensors and devices. It certainly was a booming technology area. But with this rise grew the first concerns related to privacy, security and safety. Everybody was rushing to connect everything to the internet, but the architecture and core protocols of the internet were not designed with security, privacy or safety in mind. As an avid reader about the history of computing I was aware of the inherit limitations of humans when it comes to designing and implementing highly complex information systems. Something pointed out by most of the computer science pioneers, especially Edsger W. Dijkstra. Looking at the IoT trend with this in mind made me realise that a change was needed to with regards to both the underlying technologies of the IoT as well as in the ways we built those. Else we would end up in dystopian version of a mirrorworld.
During those contemplations an intriguing technology called 'blockchain' appeared on stage. It promised to enable information systems that didn't require third parties to formalise and verify transactions. This could potentially solve some of the security and privacy issue that arose with the advent of the IoT. I spent a lot of time during those early years on figuring out how a blockchain works and how to make something useful with it. It provided me with valuable insights into the way information systems are built, how data should be treated to cater for privacy concerns, how distributed ledgers work and how smart contracts are programmed. Probably my most valuable insight was that if you want to build information systems that respect privacy they should be built around contracts. Contracts that govern the exchange of data and keep the data ownership where it should be. But, as I had learned before, it was only after going through all the effort to deeply understand the underlying technology that I began to see the limitations of a blockchain. Most use cases would simply never work. This meant I had to tell most customers that came to us in dire need of a blockchain solution that it would never work. Even though I could explain there were ways to solve their problem with existing technologies that actually worked, customers seldom listened once they heard 'no'. The best lesson from that period was that the software industry was both on the developer side and on the client side very much a human business. The company I've set up in this space, Contracts11, wasn't able to stay in business. Trying to cut through the hype, create a real working product and selling turned out to be very difficult. And with blockchain involved even impossible.
With this newfound insight and lessons in mind I rethought the concept of a mirrorworld. Mindlessly applying every new technology that comes along was naive and irresponsible. Newspapers spilling over with examples of data breaches, privacy issues, hacked devices, fake news and the unwieldy power of a few tech giants were ample evidence that this was a real and pressing issue. To prevent marching backwards into the future while looking into the rear-view mirror, as Marshall McLuhan used to say, we had to address these issues up front. For this purpose I started the Web11 Foudation, A movement for building the better web required for a non-dystopian mirrorworld. The movement aims to provide a platform for sharing ideas about ways to build a better web. Avoiding the horseless carriage syndrome 'the web' was not just the bunch of web pages for which the laymen would take it, but the web that connects everything. One of the first insights was that an all-connecting web would allow all kinds of entities to participate in ecosystems. Technology could also empower those entities so they could act independently and the ecosystem could reach an optimum by itself. A powerful idea that is applicable to many of the ecosystem challenges we face today.
While Web11 is an ongoing community effort I personally kept gravitating towards the concept of a mirrorworld and my passion for software engineering (especially AI) and creativity (3D, music). Where the rise of IoT showed progress related to the physical part of a mirrorworld, advances in the computer gaming industry were driving innovation with regards to the virtual part of a mirrorworld. It was also a realm where all my passions and skills would be of good use. I am not an avid gamer but I am deeply interested in how those virtual experiences are constructed. Because I realise those methods and technologies will be an important part of building a mirrorworld. And as the previously mentioned Steven Johnson convincingly argues in his 2016 book 'Wonderland: How Play Made the Modern World', almost all progress is driven by play. I expect this to be true for the implementation of a mirrorworld as well. So after selling my stake in XIthing and stopping Contracts11 I decided to submerge myself in the game industry and joined the Game Bakery cooperation with my company Media2B.
The game industry had been a long time driver in that other technology of my interest, AI. Since around 2012 the applicability of AI had gotten a tremendous boost due to the availability and dropping price of massive amounts of data, processing power and storage. Although AI had been successfully used in games before, this explosion of possibilities opened up completely new ways of both creating and running/playing games. The trend of applying AI for generating new worlds was especially interesting due to the obvious applicability for building a mirrorworld. Whether it was for constructing or running it. For me the final piece of the puzzle fell into place when bumping into cutting edge game development where scenes were described in normal English, an AI interpreted that and automatically generated a possibly infinite number of 3D worlds. Eureka! I realised that this was an instantiation of the general problem of designing information systems. That problem is the gap between the domain experts having problems that software could solve, but that requires them to explain it to software developers since they cannot program themselves. This translation gap is by far the biggest cause of problems with software, whether it is with regards to reliability, price, security, privacy, you name it. And now it turned out that the gaming industry had been running into this problem, it was becoming too labour intensive to create games. Modelling everything by hand was no longer viable for bigger games, and bigger games is what the market wanted. Modelling every small detail by hand would make a game prohibitively expensive. So the game industry first stepped up their game (pun intended) in the creation process. As in the general software industry tools were introduced that worked on an increasingly higher abstraction level. Instead of modelling every leaf of a tree, something I remember doing during the early years of 3D I described above, those tools allowed designers to specify a few paramaters and generate a tree. This methodology is called 'proceduralism' and is one of the most active fields in the area of game development. It is almost needless to say that AI plays an increasingly large role in proceduralism. On the other hand the game industry starting to experiment with tools that allow designers to express their intent in the language they know best and generated the content from that. An example would be turning a sketch of a landscape into a 3D landscape, or a moodboard of pictures being the source for the mood in a game, or dance to express the behaviour of game characters. Or turning a description in plain English into a virtual world, as PrometheanAI does. Again, it is AI that makes it possible to extract the intent from the semantic description. As I argued before, AI is becoming the compiler that turns semantic descriptions into experiences. It closes the gap providing complete freedom in expressing the intent and generating an unlimited number of variations of experiences from that. With that the game industry is solving one of the biggest problems of our times and provide a very large part of the puzzle for constructing a mirrorworld. Apparently John Hanke (author of the biggest AR hit so far, Pokémon Go) was right when saying, “If you can solve a problem for a gamer, you can solve it for everyone else”.
So here I am in my 25 -year old quest for a mirrorworld. You might question whether a mirrorworld will become 'reality' at all. It might just be another unrealistic pipe dream of a number of tech aficionados. But after all these years I have no doubt that this is where the world is heading. Of course, it will be different from what we envision with today's knowledge, it might take longer to get there, and the way we get there will probably be different, but the general direction is clear. And since we are going there we should take our responsibility and make sure we build something good. Now is the time to do so, and I'm surprisingly not the only one to say so. As I said at the beginning of this article, it was a recent article by Wired's editor Kevin Kelly called AR will spark the next big tech platform — Call it Mirrorworld that inspired me to write this post. Since you've come so far in this post I urge you to carefully read it. Although I don't agree with everything, for instance his big focus on AI and his view on the usefulness of blockchains, the article touches upon so many relevant topics that are in line with my experiences and ideas that it is too much work to quote them all here. Kevin Kelly has been one of the most respected thinkers in the realm of the digital revolution. In that role he started Wired, in my opinion the best magazine on the same topic publishing quality content since 1993. Kelly has been covering the march towards a mirrorworld for as long as I've been involved in it. And now, after 25 years, he finally dares to state the we are about to embark on the journey towards this next big tech platform, saying, "it will take at least a decade for the mirrorworld to develop enough to be used by millions, and several decades to mature. But we are close enough now to the birth of this great work that we can predict its character in rough detail". So yes, it is still early days but now is the time to act to ensure we build the best possible mirrorworld we can. There is a lot at stake, but also a lot of excitement, wonder and fun to be found. I know where I am heading. In the end, a mirrorworld will be the next big, possibly the ultimate, mass medium and that is where my story started. And that is why I called my company Media2B. Fortunately, in the mirrorworld, there is always room for more, so why don't you join me on this quest to build this new medium? As Kelly says, "There are no experts yet to make this world; you are not late".