Web development with Clojure — an overview

After a few weeks of investigating various approaches to web development in Clojure, I'd like to write down a summarised overview to help me nail down some thoughts and potentially make a few decisions in the near future. I hope other people might also find it helpful.

Note: This is based on a considerable amount of research and thought, but very little personal practical experience. I will update this blog post based on suggestions from the Clojure community.

Web applications concerns

Most web applications have to deal with a large set of common use-cases that are inherent to the client/server, stateless, HTTP request/response model. Some web frameworks try to cover a lot of these use cases and provide some value on top, whereas other are more minimal and depend a lot on middleware and other plug-ins or libraries to cover the gaps.

On top of that, the more established frameworks also try to tackle data modelling, persistence and more. These so-called "Monolithic" web frameworks usually try to address most of the following concerns:

  • Architecture — there are pre-defined ways for you to write your code. Most, if not all architectural decisions are made by the framework creators, e.g. MVC.
  • Basic HTTP interface — request parsing, response generation etc. Most of the time there is an agreed-upon interface like WSGI, Rack or Plug that these frameworks conform to.
  • URL Routing — dispatching the correct action based on the URL structure.
  • HTML templating — creating HTML documents with common sections (headers, footers, sidebars), links, dynamic content and so on.
  • Persistence — reading and writing to some database, usually through some other abstraction, e.g. an ORM.
  • Sessions and Users — handling cookies, per-session data, authentication/authorization
  • Internationalization/Localization — showing the interface and data in multiple languages
  • File handling — accepting form uploads and storing them on persistent storage
  • Emails — sending emails via templates
  • Forms — rendering and accepting HTML forms, incl. widgets, validation and eventually persistence of data.
  • Security — ensuring that the web application isn't vulnerable to the most common security pitfalls, e.g. CSRF, XSS, SQL injection, timing attacks and so on.
  • Websockets — or compatible alternatives like long-polling, server-sent events etc.
  • Javascript and other Assets — packaging Javascript files, compiling SCSS, optimising images etc.
  • Job queues — scheduling jobs to happen out-of-band
  • Testing — writing and running tests at different levels of abstraction
  • Extensibility — allowing the addition of middleware and plugins that enhance or replace built-in functionality
  • Production — deployment, web servers, caching

On the other hand, the so-called "minimalist" frameworks will mostly deal with basic HTTP and perhaps one or two more concerns, while you pick and choose from various libraries to fill in the gaps.

The pros and cons of "Monolithic vs Minimal" depend on your use case, experience with the language and library landscape and your level of comfort when making architectural decisions — I won't go into a comparison here.

The Clojure situation

There is no such thing as a "Monolithic Clojure web framework". However, I'd argue that there quite a few frameworks that offer much more than other minimalistic frameworks in other languages. In addition, the JVM layer on which Clojure is based on is a rock-solid foundation for when the time comes to deploy, have logs, metrics etc.

Rather than going concern-by-concern, I'll present Clojure web development as a series of layers. The crucial different between Clojure and most other common web-dev languages is how easily one can combine and compose libraries to take care of some concerns (Phoenix/Elixir is also good at this).

NOTE: One major thing that Clojure also has is the toll-free interop with Java libraries — and also a lot of historical artefacts of Java Web development, such as servlets, web containers etc. However, most of the time you won't be dealing with those, and a lot of approaches don't rely on them at all.

Ring specification & Java servlets

This is not actually the top-most or bottom layer, but it's central to everything else so it needs to be introduced first.

Ring is a simple, server-agnostic specification that models HTTP requests and response to plain Clojure maps and functions. A Ring handler is a function that takes a Ring request map as an argument and returns a Ring response map. The request and response maps are plain Clojure maps with some well-defined keys like :body, :uri and :headers. There are synchronous and asynchronous flavours (the async version is trivial to implement once a sync version is written).

This abstraction, inspired by the WSGI and Rack specifications of Python/Ruby, supports writing adapters that are responsible for abstracting away the low-level networking and protocol aspects.

The rich I/O facilities of Java allow for some powerful features — for example, the response :body can be a string, a sequence, a file or a generic java.io.InputStream and the server will do the correct thing.

All the servers and frameworks listed below support Ring handlers (which are plain functions) and sometimes will add their own extra features on top. A Ring handler function is portable across all web frameworks and servers, though.

Java Servlets is a 1998-era specification for doing HTTP with Java, and there's a ton of Java web servers out there that support servlets. Ring has some tools that can convert a Ring handler into a generic Servlet, so that they can be embedded with any existing Java web-app.

Frameworks

Continuing this middle-out exploration, we go to frameworks. While you can very easily just use Ring to write a trivial web applications, there are some frameworks out there that are best suited for a more complex app. Some frameworks have an opinionated front-end story, whereas others focus solely on the back-end.

  • Luminus — less coherent than a framework, and more a project template that uses a curated collection of libraries. Has a number of options that can bring in databases, i18n, templating, etc. Extensive documentation and a published book.
  • Pedestal — originally a full-stack framework created by Cognitect, now focused on server-side APIs. Handles HTTP, Routing, Security, WebSockets, Deployment concerns. Production grade but documentation could be expanded.
  • Yada — A framework by JUXT, that aims for 100% HTTP compliance, including advanced or niche areas of the spec. Production ready but the HTTP focus may be limiting. Can only be run on Aleph (see below).
  • Duct — An ambitious "configuration-driven" framework, that brings in under its umbrella a number of compatible libraries for concerns such as routing, databases, migrations, front-end assets. The creator is well-respected within the Clojure community, however the project is still young and documentation is sparse.
  • Fulcro — a front-end first web framework. Extremely opinionated on the front-end, and somewhat prescriptive on the back-end. A special case since it's aimed 100% at Single-Page Applications, it dictates a server architecture that can be implemented by any of the above frameworks as needed.

There are quite a lot more libraries that can be integrated with any Ring-compatible framework (that is, all of the above). A lot of standalone Ring middleware are already reused by Pedestal and Luminus. Yada is somehow a lone example, as it suggests that Ring middleware go against HTTP semantics and data-driven architectures.

Networking stuff

At the bottom layer, we have the actual TCP/IP stack and the first layer of HTTP protocol implementation, plus web socket support. All the options here have more or less feature parity, so there's no clear winner in my eyes.

  • Jetty — Java HTTP/Websocket server, natively supported by Ring.
  • HTTP-Kit — Clojure/Java web server.
  • Immutant — Clojure HTTP library, based on Undertow. Standalone or embeddable in the WildFly container for cluster support.
  • Aleph — Clojure networking library, based on Netty.

All the servers are standalone and can be bundled into a JAR file. This means that the only dependency you have when deploying is a Java Runtime Environment.

Jetty and HTTP-Kit are quite stable and conservative choices. Both support Websockets and streaming, while Jetty also supports HTTP/2 and HTTPS. HTTP-Kit also includes an HTTP client.

Aleph is an ambitious asynchronous networking library, being able to handle TCP & UDP streams, while also exposing an HTTP layer.

Immutant is also far more than a web server — it adds support for Messaging, Scheduling, Caching, Transactions, built on established Java libraries. It also supports clustering when deployed in the WildFly container. You can start out by just using the standalone web server though and not even include features you don't use.

Deciding?

There is definitely a lot of choice, even for a seemingly simple concern such as dealing with HTTP — and then a ton of other choices that relate to persistence, front-end etc. I won't give any opinions here, but I'll try to articulate my thought process since I'm currently evaluating a lot of Clojure libraries for building a web application.

Start with the data

Perhaps the most important decision that will be the hardest to reverse is the data modelling and persistence layer. Perhaps you already have your data in some database, or perhaps you're only comfortable with one particular technology (e.g. SQL or NoSQL), so this narrows down your choices. Fortunately, the decision matrix for persistence in Clojure is much smaller — there are a couple of SQL libraries, and each major NoSQL library has one well-maintained client.

I'd first start by writing some data structures (start with plain maps, perhaps use records if really needed) that show how the data would like "in-flight", that is, when passed around inside a live system. You can use Clojure Spec for this, or take a suggestion from the libraries used in Luminus.

Saving the data

Next you have to decide how the data are going to look like when "at-rest" — I'd create an internal API (just a set of functions, really) that deals with persisting the in-flight data model to the database. Start with simple CRUD operations, then add other actions that might not model well to CRUD. It's a good idea to try and abstract this behind a protocol of sorts, so that it can be mocked out for tests or even entirely replaced.

Showing the data

The next biggest decision is actually how you'd get the in-flight data shown to your users — this usually means either rendering HTML (old-school style) or passing the data to the browser in some format like JSON. You might want to consider GraphQL as there's a really good library for that, or you may want to use a simpler REST-like interface. The possibilities here are quite large and it really depends on what you are aiming to do and what front-end technologies you'll use.

Tying everything together

I'd argue that the above steps are two thirds of the work needed to get a web application off the ground. When the above decisions are made and implemented, picking an HTTP framework should be an easy decision, and things like authentication and authorisation should be quite straightforward to add.

I hope this short overview has helped demystify the landscape of Clojure back-end web development. Please let me know of any errors or omissions on Reddit, Hacker News, Clojureverse or just drop me an email.