Old Programmer, New Idioms

Last night I talked my way through the analysis and design of a program to solve the Countdown numbers game. Tonight I have spent some time attempting to implement this software using Clojure, a modern dialect of Lisp. In doing so, I discovered a blind spot in my programming experience. Clojure is a functional programming language. I’ve spent most of my life writing software in imperative and object oriented languages. I have studied the functional paradigm but I haven’t practiced it to any great extent. Tonight I’m going to discuss what that means and what I will do to move beyond this temporary hurdle.

Functional programming languages are based on the concept of mathematical functions. In pure functional programming languages all data is immutable and functions transform their input into their output with no side effects. That is fine in theory, but the valuable part of most programs, that is reading input from various devices and writing output to other devices for instance, are inherently entangled with side effects. You can’t read a keyboard twice and expect to get the same character unless the user just happens to type the same character twice in a row. To put it simply, the input function depends on the side effect that is the key that the user has pressed on the keyboard.

So, languages like Clojure make clearly defined distinctions between the bulk of the language that honors the functional principles and the non-functional parts of the language that allows practical work to be accomplished. This should be easy to get your mind around but it’s not.

In traditional imperative languages, we deal with variables which serve as containers to store values. We can store a value in the container and retrieve it later and it will still hold the value that we stored in it. In fact, the only way that we can change the value that it holds is to store a different value in the container. Since the container can only hold one value at a time, that clobbers the original value with the new one. This is fine in the case where you have a single processor executing a single instruction at a time.

But in the modern world few computers have a single processor in them and even if there is only one physical processor the software simulates multiple processors and multiple threads of execution. The problem arises when one of those threads stores a value in one of those variables that it shares with another thread, and then another thread decides to read it just as the first thread decides to change it. The results are unpredictable and account for one of the more difficult bugs to find in modern software.

We have been doing this kind of coding for decades now and there are text book solutions to the problem but they are complicated and easy to get wrong and Murphy, the patron saint of programmers, has his way more often than not.

Clojure has made a crucial distinction between values and identities. A value is constant. The number seven has a value that we can count on never to change. Now we might have two containers that can hold a number. The first one we give the identity X and the second we give the identity Y. We assign X the value 7, using a special mechanism that insures that no one will be able to read the contents of X until we have completed writing the value 7 into it.

The key thing to know about the difference between values and identities is that values don’t change and identities don’t actually hold values but rather point to them. Thus, if one thread has a copy of the pointer to the value that an identity was pointing to at one point in time, that value won’t change unpredictably when the identity is changed to point to another value.

This calls for a very different way of programming. You spend most of your time creating temporary values that don’t ever get changed and only rarely give a value a longer lived identity. The idioms of this style of programming are different. I am having to figure them out as I go. I also have to analyze them to decide which ones perform better than others.

In short, you can teach an old dog new tricks but you and the dog have to want to learn and be patient. When I manage to implement this algorithm in Clojure I will devote a blog post or several of them to walking through the solution.

Sweet dreams, don’t forget to tell the ones you love that you love them, and most important of all, be kind.