Three a-ha moments

by Orestis Markou

I have recently had a minor enlightenment about what programming languages I want to use in my daily work, and I would like to capture them for the benefit of new and experienced programmers in search for an addition to their toolkit.

I want to preface this by saying that of course there are niches (small and large) and "killer libraries" for those niches that may render all of my arguments moot; I'm talking mostly in the context of doing "general-purpose" services and applications that run on a few servers somewhere. Certainly web services, but also other not-so-web services.

Consider Python in the context of web development. It has a few very mature web frameworks out there, a fantastic asynchronous networking library, an excellent and vibrant community — yet always there is a "ceiling" of load that a web application can handle before getting rewritten to something else, usually Go, perhaps Java; certainly a less dynamic and closer to the metal language that has "real concurrency".

Even by replacing the CPython VM with something like PyPy, you still have to the GIL and therefore you have to have some event-loop based mechanism to do concurrency (sadly other VMs like Jython and IronPython didn't survive the Python 3 transition, so we'll never know what a multi-core Python could work like in practice).

I don't mean to single out Python; Ruby and NodeJS also suffer from this ceiling; The moment you want your web service to do something more "interesting", you start to pull in the dependencies: Redis, RabbitMQ, Memcached and a host of similar tools that are capable of running for a long time, serve multiple concurrent connections in one process, maintain state and not leak memory.

My first a-ha moment: if you know one of these languages, there's no point in learning the others; you'll hit the same ceiling sooner or later. I think this is obvious by the perceived lack of lateral movement between the Python, Ruby and Javascript communities.

Even by taking those out of the counting, there's certainly no shortage of languages to consider. You can easily spend a week going over the TIOBE's 100-most-popular languages index, trying to compare strengths and weaknesses for each.

Assuming though that you need something cross-platform and general-purpose enough, here's my thinking:

  1. Pick the ones with a runtime (or VM);
  2. Evaluate the runtime (or VM) instead;
  3. Pick a language that targets the runtime that suits you.

Why runtimes are important:

  • A runtime usually only gains in performance, security, stability and you get all that for free, without having to rewrite your code.

  • Using a multi-language runtime means you benefit from existing libraries in other languages, assuming a sane interop strategy.

  • New languages will appear that target your runtime, and you can choose to write parts of your system that make sense in them, without ditching your entire codebase.

I think this whole exercise narrows it down to only a handful of contenders:

  • JVM
  • CLR
  • BEAM

My second a-ha moment: You want to focus on the runtime and not the language, because the runtime is what makes or breaks your software in production. The obvious point here is where are the languages that target the Go runtime? Does a Go runtime even exist?

All three are old and battle-tested enough across a whole range of companies and services that should be safe bets. Also all three have major corporations (Oracle, Microsoft, Ericsson) funding their maintenance and development. You really can't go wrong by picking one of those three. Perhaps you'll rule out CLR since it only recently (partly) became Open Source.

From those three, BEAM is quite interesting; It's the only VM I know of that provides pre-emptive concurrency, so that you don't have to write "cooperative" code. Of course, that comes at the tradeoff of performance and expressiveness (in the sense that you can never have shared memory, if you need it). But still you can go quite far.

Elixir has quickly gathered a community of developers that are giving back a lot of useful libraries; plus, as mentioned before, you get access to all the existing and battle-tested Erlang libraries. I personally think that if you're developing web-services without crazy number-crunching requirements, you should really look into Elixir and OTP.

If you want something more general purpose, you might want to look into Clojure; my tip for sticking with it is to keep counting the parentheses: it's always the same number as with your current language, it's just that foo(a, b) looks more familiar than (foo a b) - plus, there's fewer commas and semicolons :)

So finally we come to the elephant in the room: Javascript.

It's true that Javascript is the queen of languages - it can move everywhere. And indeed in has. Apart from being, of course, the only language that can run in the browser.

There was a point around a couple of years ago that ES2015 (aka ES6) had gained 100% support in most browsers, meaning you could ditch most of the transpiling and general complexity, at least during development. The future was bright; finally a modern Javascript dialect that can help us write more expressive programs without bringing in a billion dependencies and a convoluted build chain

Unfortunately, upon closer inspection, soon after ES2015 was finalised, Javascript became a "living" language. Meaning new syntax would be added all the time, and browsers and tool maintainers would need to play the catch-up game. And that would be OK if the focus was on actual filling in the Standard Library gaps, but instead the focus was mostly on new syntax sugar.

This was the point where I gave up and switched new development to Typescript. I felt that it was a more stable platform I could build on. Soon after though, Typescript started moving rapidly again, slowly gaining new features on its type system while also trying to keep up with Javascript (since it aims to be a superset).

At that point, I knew that the cycle of churn would soon begin anew; and I turned to look for other "compiles-to-JS" languages that treat Javascript as byte code and not as something that needs to be readable.

Out of many many contenders, I focused on two:

  • ReasonML/Bucklescript/OCaml
  • ClojureScript

I spent a month or so learning Reason and OCaml, but it felt that the ecosystem itself was much too young to be a robust platform, and even more crucially, the Standard Library story was just not there - it was not the goal of ReasonML to create and maintain a sane Standard Library for the web. Certainly though given the pace of development, it is worth another look in a year or so.

On the other hand, the ClojureScript approach is to build upon the existing Clojure Standard Library, and rely on the Google Closure compiler doing dead code elimination to ensure the delivered bundles are not bloated. I had a very pleasant experience in using the (Javascript version) of Google Closure: only 4MB of node_modules, ES2015 to ES3 transpilation (great for even very old browsers) and all the performance optimisations you can handle. In addition, the ecosystem has some top-notch developer productivity tool and has recently seen some major improvements in their interop story.

My third a-ha moment: For the browser, pick an advanced language that compiles to Javascript, and you'll never experience JS fatigue again.

Soppy note: While writing this post I felt a bit bittersweet because I realised I'll probably won't use Python again in a production setting. Python was the language that propelled my software career and it does feel weird to be moving on. I would like to thank all the wonderful people that work on Python, and all the fantastic, caring and thoughtful community. You are the best!