Choosing a Computer Language

One of my favorite books is Gödel, Escher, and Bach: An Eternal Golden Braid by Douglas R. Hofstadter. It is a book about self reference in mathematics, art, and music. It is also about the challenge of creating a General Artificial Intelligence, although I don’t think Hofstadter ever explicitly says that. I had read portions of it several times before I managed to make it all the way through it from start to finish. That was partially because I was impatient and easily distracted but mostly because it was such rich fare. I would read a chapter or two and have to go think about it for a while and absorb it.

One of the things I learned from the book was Gödel’s theorem. Paraphrased it says that a sufficiently complex formal system can either be consistent or complete. I took that to mean a given programming language, which by definition is internally consistent, can not express all programs. This convinced me that there would never be a computer language that would be so good that we wouldn’t need any other.

Since then I have spent a lot of time learning new computer languages. I have tried to understand what kinds of processes they expressed well and what lay beyond their ability to express. Consequently when I sit down to start a new programming project, the first question that I ask is which language should I implement it in?

Some people will tell you that all sufficiently complex languages can be expressed as a Turing machine. This quality of a language is known in Computer Science as one that is Turing complete. The problem with such an assertion is that the Turing machine is so primitive that no one would ever write a complex modern application using only the operations provided by a Turing machine. Instead, they would write it in terms of more complex abstractions built on top of the Turing machine. It is these more complex abstractions that run afoul of Gödel’s theorem, i.e. being sufficiently complex.

This brings me to the point I was going to make about choosing a language. I often find that I can try to fit a square peg in a round hole by choosing a language that I am familiar with but that doesn’t support the abstractions of the domain that I am trying to model. Or, I can learn a language more suited to the domain. Or, I can use a language that is good at extension to support new domains.

Lisp is a language that falls into the latter category. From it’s conception it has lent itself to the creation of so called Domain Specific Languages (DSL). This is usually attributed to its simple syntax, the fact that Lisp programs are represented in the same fashion as Lisp data (a feature know as homoiconicity), and the availability of a macro facility that makes it easy to create new idioms within the language. The latter aspect, Lisp’s macro facility is at the heart of its extensibility. It is also one of the harder aspects of the language to master.

So, I find myself waffling between Java, a language that I have lots of experience with but that is very verbose and labor intensive to write, and one of the various dialects of Lisp that are much more concise, support incremental development, and have macro facilities that allow me to adapt them to the domain of my app.

One might think the choice would be obvious but it isn’t. Java has a very attractive aspect that it offers. There are libraries available to do just about anything you might imagine written in Java. This means that when you are writing an application in Java, you often find yourself spending your time first looking for a library that provides some functionality you require and then writing code to adapt that library to the rest of your application.

But wait, there is another way. Java compiles to an intermediate byte code that is implemented by the runtime on all the machines that run Java. This byte code interpreter is often referred to as the Java Virtual Machine (JVM). Other languages can be written so that they compile to byte code that runs on the JVM. These other languages then can make use of all the Java libraries that are available to native Java programs. Clojure, a modern implementation of Lisp, is one such language.

Now you must be thinking, “So what’s the issue here? Obviously Clojure is the language of choice here.” And you would be echoing what I keep trying to tell myself. The problem is, that Clojure is a very opinionated language. And while I agree with most of those opinions, the predominant one being the immutability of most data structures, the bad habits of my earlier career in programming make it difficult for me to embrace immutability and develop my apps in Clojure.

More precisely, when I am doing the exploratory part of developing my app, I spend a lot of time coming to terms with Clojure’s well intentioned restrictions. When I reach my frustration limit, I find myself going back to Common Lisp, the first Lisp that I learned, and trying to use it to prototype my ideas with the intent of porting my code back to Clojure when it is sufficiently mature.

Common Lisp has its own problems, chief among them, it is so big. I often find myself implementing some piece of functionality only to discover that there is a native implementation of the same functionality in the core implementation of Common Lisp. Not only that but there are a number (not quite as many as with Java but still quite a few) of libraries that also have better thought out and tested implementations of the functionality that I have produced on my own.

This sounds like a case of an embarrassment of riches and in fact it is. But it produces a kind of cognitive inertia that makes it difficult to get a project off the ground in the first place. I have managed to overcome that inertia to some extent and have found a collection of appropriate libraries to get me started on my latest app using Common Lisp.

Whether I wil be tempted to back port it to Clojure depends on how well it performs in Common Lisp. Steel Banks Common Lisp (SBCL) has a facility for compiling Lisp apps to native executables. That may end up being the way that I eventually ship the completed app and will make it so that I don’t need to put the effort into backporting.

If you’ve made it this far, thanks for listening. I hope I’ve expressed the issues that I’ve been struggling with clearly enough.

Stay safe in these dangerous times. Wash your hands, maintain social distance, stay home as much as you can, and wear a mask if you do go out. Tell your loved ones that you love them. I’ll talk to you again next week.