My Love-Hate Relationship With Clojure

rambling clojure

I've been working professionally with Clojure now since the spring of 2013 when I stumbled into a Clojure job without any prior knowledge of the language, or any exposure to Lisps, or even functional programming. I took a job that was advertised to be "a majority of PHP development with a little bit of Java and Clojure here and there" and the job instead turned out to be more like 90% Clojure, 9% Java, and 1% PHP.

Getting into Clojure was a hard adjustment for me. I can remember it took me a long time (probably a month at least) before I can say that I really started to "get" functional programming. And probably significantly longer before I think I could reasonably say that I was confidently able to write and architect complex and significant amounts of new code in a "good" functional or data-driven style.

I've now worked at four different companies that used Clojure primarily for development. I've been able to do some greenfield development work, but mostly, I was getting into an existing code-base. I've also authored a few different open-source libraries with Clojure. I don't think I'm some sort of "ultra Clojure guru", but I think it's quite fair to say that I am quite experienced in it.

Looking back at the past 9 years in which I've spent the majority of my working hours continuing to work with Clojure in some form, I think overall it has had a noticeably positive impact on my career and on my skills as a software developer. I think if I'd not been introduced to Clojure when I was, I likely would've continued down a very heavy object-oriented programming path in my career with probably some mix of Java or C#, using all kinds of "gang of four" design patterns everywhere. Today, I shudder at the thought of that, but I remember in early 2013, while I didn't love that stuff and would approach it cautiously, I didn't recoil in horror at the sight of it either. I just saw that stuff as another tool in my toolbox.

I know that nowadays, when I write code in a procedural or object-oriented programming language, I do tend to approach it from at least somewhat of a functional programming mindset. I definitely think this is a very good thing. So much so that I believe one of the best things that any developer can do today to grow their skillset, is to spend some serious time with a functional programming language. Even if you know you'll never be able to use it professionally, try to use one seriously for some side-project. It will really change the way you think about code.

But Clojure is not all sunshine and rainbows. At least, not in my personal opinion. As I find myself today at a bit of a turning point where I am seeking out a new job in earnest and am actually not specifically looking for Clojure work at all anymore (I wouldn't turn down the "right" Clojure opportunity if one was thrown into my lap, rather I just mean that I am no longer actively filtering my job searching for Clojure at all as I used to do), I wanted to take a moment to reflect on that. It's weird for me to say that given what I wrote above about how I think Clojure has had a fundamentally positive impact on my growth as a software developer. And this is why I wanted to capture my thoughts on the language, the ecosystem, and the community (to a certain extent) and what I think has led me down the path I now see myself headed in.

I want to stress here for anyone perhaps randomly coming across my post that the things I am writing about below are all my own personal opinions. You may or may not agree with me. You may have had very different experiences compared to mine. That's all fine. We are allowed to have different thoughts and opinions on these matters!

What I Like

The REPL

The Clojure REPL is such an integral part of developing with Clojure, and to me back in 2013, the experience was eye-opening. I'd not used any other programming language up to that point which had a REPL that was useful as anything other than a toy or simple tool to test out quick snippets in an isolated fashion.

In Clojure (and I guess most other Lisps), the REPL is normally directly connected to your running project and is a way for you to add new code, update/replace existing code, inspect the values or state of anything currently running in your project, run any arbitrary pieces of code against the running project, etc. One important distinction, as I understand it, as compared to REPLs found in many other languages, is that the Clojure REPL is not solely file-based. You can apply code changes against the running project independently of the source files on disk. But you can also reload namespaces from the files on disk too.

The typical way to do development with Clojure is to start up your REPL, and write code in your editor that will ideally be connected to that REPL session, so you can save code changes to your project's source files as you write it, but also so you can directly run your code changes in the REPL as you write it, and see your changes running against the same instance of the application without needing to do a compile and restart in-between.

This leads to a workflow that allows for a high degree of experimentation and iteration. In my REPL-connected IDE (IntelliJ with the Clojure Cursive plugin) I can easily place my cursor over any bit of code in my project and hit Command+Enter to run that snippet in the REPL. As I write Clojure code, my source files often temporarily end up looking like a sort of "scratch pad" of different bits of code snippets while I was trying different things until I finish up whatever it is that I was working on.

You can also connect to remote REPLs, so in addition to your REPL being useful as a local debugger of sorts (you can also use a more traditional debugger with Clojure too), you can also use it to inspect the state of a deployed application in real-time.

The Standard Library

The Clojure standard library is very fully-featured. Not much to really say here, just that it makes it easy to manipulate data in most any way you would want. I find that even years later I'm discovering functions in the standard library that do something which I'd otherwise be writing a special-case combination of map and reduce for, or something like that. There really is a lot of very useful stuff at your fingertips here.

Immutability First

During my transition towards thinking in a "functional programming" way, I found this annoying. But once it clicked in my head, it really clicked and was great.

Being able to write code that passes around values like lists, maps, etc (which you will do a lot in Clojure) and transform them in one function for some purpose, while not having to worry about things like "oh, I should maybe clone this first so I don't mess up the value for the caller" is very freeing.

Of course, this only applies to the built-in Clojure data structures. If you're working with a Java object directly, even in your Clojure code, then you of course do not get any immutability (unless that Java object is providing it internally).

Built-in Data Structures

The aforementioned data structures provided with Clojure are great to work with. The literal support for lists, vectors, maps and sets are all invaluable and you use them all the time. It's really a night-and-day shift to go from writing out a map definition in your code in Clojure vs Java or C#.

These are what really make the whole "code is data" thing such a joy to work with.

Interop With Java

The interop story with Java is pretty good. For most things you'd want to do here, I would say it works quite well and you won't encounter much difficulty. For some uncommon advanced uses it can get a little bit awkward at times, but I think writing Clojure code that utilizes some Java library isn't too difficult in the majority of cases.

And you will be doing this at some point as a Clojure developer. The situation today is even better than it was in the early 2010's where the community had not yet created all kinds of Clojure libraries to wrap Java libraries. Nowadays, there probably is a Clojure library available to make it a little bit easier for you to use whatever Java library you need in your project. But there still might not be. Or, the library that exists might not be maintained anymore. So in the event that you need to dig in for yourself, it's often not too difficult to write your own interop layer.

And because Clojure runs on top of the JVM, you of course are getting access to the entire Java ecosystem of libraries which is huge of course.

ClojureScript

ClojureScript is the single reason why my current JavaScript expertise basically stopped at jQuery with a very, very surface-level exposure to React directly.

This is because in 2013/2014 when I had to work on some front-end heavy web applications, I was introduced to the amazing Reagent library. This is a ClojureScript layer on top of React that really renewed my enthusiasm for writing front-end code after dealing with some disastrous jQuery mess and not particularly liking what I was seeing with using React directly with JavaScript (in particular, it seemed like a fair bit of boilerplate was going to be needed to create components).

Writing React components with Reagent and ClojureScript was just amazing in comparison to using React with JavaScript directly. Your React components were just functions! HTML could be written directly in your component functions through the use of Hiccup-style syntax for HTML, providing very similar functionality to React's JSX. And through the use of Reagent's "Ratoms" they would automatically refresh when application state changed. And throughout all of this, you were just using normal built-in Clojure data structures and functions. A Reagent "Ratom" is just a Clojure atom from the developer's point of view and you can update it just the same way you would any other atom with stuff like swap! and reset!. When combined with Figwheel to get live hot-reloading of code in your browser, this provided you with a really powerful development workflow. I think the JavaScript ecosystem has now finally caught up here with code hot-reloading tooling for the most part, but this was really something to experience in the early 2010's.

Leiningen

There are a few different build tools for Clojure projects available nowadays, but Leiningen was there from pretty early on and very widely used in the community. This seems to be slowly being replaced with Cognitect's own solution in the community, but I've still not ever used it myself. It was a (to me) curiously late effort by Cognitect who seemingly didn't care about providing a build tool to go along with Clojure and left it up to the community for many years.

Leiningen has never been a perfect tool, but it has been pretty solid overall. Some advanced, large projects can end up with project.clj files that are rather unwieldly, but for the most part I found it pretty easy to work with.

I particularly liked that, when combined with lein-cljsbuild, you had very easy entire project building for both server-side and client-side code all via Leiningen. Compared to most other programming languages and build systems where you'd be wrangling one build system for your server-side code, and then probably one or more others for your client-side JavaScript. Leiningen is just amazing in comparison to that.

What I Dislike

You'll note that the above sub-title is "what I dislike" and not "objective problems with the language" or something like that. If you get all hot and bothered by anything I write below (basically, if you turn into this while reading), then I would suggest you should step away from your computer for a few hours to get some perspective.

Dynamic Typing and Maps Everywhere

Oh dear. Dynamic typing vs static typing. Am I really going to go there? I hate discussing this. I hate discussing it because in my personal experience, most developers are incredibly immature when it comes to this topic. It almost always seems to devolve into people cherry-picking their favourite examples showing why such-and-such a thing isn't possible with whatever typing system they dislike. Or you get people making grandiose claims on either side about how "well, in my experience this doesn't happen, so that's not really a big benefit" etc, etc. And finally, in many cases it devolves into personal insult flinging. It really is sad.

I have my own opinions on this topic. You may too. It's all good.

My experience on large Clojure projects is that your code is, in most cases, wrangling un-typed maps all over the place.

Now, this makes sense when you think about it because Clojure is dynamically typed and has great, easy-to-use, built-in data structures like vectors, maps, and sets which are all quite natural to use to represent your application's data with. Whereas in Java, you would probably be writing classes for all kinds of things to represent your application data, in Clojure, you often just stuff your data into a map or vector. Even when you query for data from a database in Clojure, it is almost always being returned as a vector of maps, where each map is a row of data. This is a pretty natural representation of the data to work with.

Without getting into a huge amount of specific details, the problems I almost always see in large Clojure projects is that you end up with a large collection of functions all over your code that are operating on seemingly random maps, some of which may look similar to others at first glance, but this underlying uncertainty can make it difficult to make future changes or perform any refactoring work.

As well, it does absolutely nothing to help new developers get up to speed with a code base. When I'm reading a chunk of Clojure code operating on data, I will often have a difficult time understanding the exact shape of the data that any arbitrary function or collection of functions is working with. I can figure it out eventually, yes, but it is sometimes not trivial from looking at the code itself especially as compared to just looking at a type definition.

And even worse, over time I will be naturally building up a sense of that project's internal data structures and start encountering similarly named variables over a large amount of functions and then infer from that what that value looks like at run time. But I could then easily completely miss the fact that in maybe some cases certain transformations get done to the value before the code I'm looking at runs, meaning the value(s) I assume should be in this map, might not be sometimes. Or maybe the types of the values are different (keywords instead of strings, for example). In a statically typed language, you'd likely have specific types to represent these slightly different pieces of data and so the questions about the shape of the data at any point in time in the code becomes a bit clearer and easier to reason about at a glance. Maybe. It really depends on the quality of the code, as with anything!

Data Annotation Libraries

Here I'm referring to stuff like Schema and the more recent Spec which is Cognitect's own solution.

You'd think that after reading my previous thoughts on dynamic typing and maps that I'd actually love these libraries bringing the ability to annotate your Clojure code-base with type annotations and specifications. I thought I would too. I used Schema on quite a few projects starting from late 2014.

The problem in my experience is the optional nature of them. Most programmers are quite lazy and, I'm sorry to say, lack discipline when it comes to keeping a code base neat and tidy. It's hard, I get it. And the realities of the real world are almost always at odds with this and instead seem to reward rushing out quickly thrown together solutions.

Type annotations with Schema (and later, Spec) are almost always one of the things that gets left to languish in any project I've worked on. Or, developers take the "easy" way out and use schema.core/Any (which is a catch-all that matches anything, even nil). The only way I found to force developers to keep these maintained was to keep run-time validations turned on, and not everyone always wants to do that. And even if they did, they probably wanted to only have validations run in certain cases, not *everywhere*.

But the main reason why these data annotation libraries don't take care of my previously mentioned issues with dynamic typing is that most Clojure developers will not want to annotate every function in a code base. The common tactic here is to create type annotations at the boundaries of your code. For example, in your HTTP endpoints, or just at the public API of a library you're developing. That kind of thing. But then you rarely see anyone adding type annotations anywhere for anything internally in your code-base. And I guess I can't blame them, because when you start going down that road of annotating a whole lot of things, it kind of removes many of the benefits of Clojure being a dynamically typed language. It really is a double-edged sword!

"Choose Your Own Adventure"-Style Project Architecture

Clojure has nothing even approaching the Ruby on Rails framework. And the community doesn't even want it. The only initiative I am aware of that seemed like it was sort-of approaching that kind of scope was the ill-fated Arachne Web Framework which seemed to receive a ton of criticism in the community and just died out.

So that's fine I guess, the community would rather string together their projects like they were lego blocks by grabbing whatever libraries they need for things like HTTP routing, database access, security, JSON parsing, etc themselves directly. To make it easier, people have maintained project templates to help others get started with bootstrapping a project with a curated set of libraries and some starter infrastructure code.

At its core, this is kind of exciting if you've spent a long time dealing with some big framework in another programming language that you weren't a fan of. "Finally, I can set things up exactly how I want them!"

Of course, it opens up a ton of ways to shoot yourself in the foot. You now need to be well versed in a lot of different libraries and how they are pieced together. Even if you are using a project template to get your project started, the responsibility is still on you to keep things up to date as your project ages. Once you've created your project from a template, that template won't help you anymore to keep things updated. Libraries may change over time, the way that you integrate them together may need to change with them. It'll be up to you to stay on top of this. This problem of course does exist also when using a large framework in any other language. But with such a large framework, you usually have a single place to go for documentation and help with performing upgrades. With a collection of different libraries, you now have to hope that each of those library authors is diligent about documentation! And in more difficult cases, you need to hope that someone else has experienced the same upgrade difficulties with your chosen combination of libraries and has shared their solutions. Otherwise, you'll be on your own.

Some recent efforts have emerged to help fix this by attempting to modularize more and more of a typical Clojure project's architecture and in essence, turning the old project template idea into something much more similar to a real "framework." But it remains to be seen how (or if) this really improves long-term maintainability in practice, as it's still early days. I'm personally not sure if this is at the right level to really tackle the longer-term maintenance problems, but it does at least feel like a step in the right direction. But regardless, you will find projects in Clojure in the future that are not built with this particular "framework" because people have a choice, and I've observed most Clojure developers really do seem to love making the choices themselves when starting out a new project.

Also, hopefully one of the libraries you chose doesn't stagnate because its author lost interest in maintaining it. If that happens, well, good luck, hopefully it is easy for you to move on to something else. For what it's worth, this could also be seen as a benefit of this style of project architecture. If this situation does occur, then in theory it is easy to migrate only the affected portions of your code to an alternative. In theory ... Also for what it's worth, this is always an issue in any project, regardless of the programming language used. I just often feel like Clojure is at least a little bit more susceptible to it due to the "preferred" way to string together a project.

Anyway, this "choose your own adventure"-style of project architecture has, in my personal experience at least, lead to a reality where every single Clojure project of significant size is its own unique snowflake. There may be some loose similarities here and there, but even those similarities (if they even do exist) are nothing like the sort of regular code structure you might see across any number of Rails projects (for example).

This, to me, presents a huge double-whammy effect when it comes to getting familiarized with a Clojure code-base when combined with my aforementioned issues with dynamic typing and "maps everywhere." I guess this is really important to me as I rarely get to do greenfield development anymore and instead increasingly find myself cursing the names of the previous developers of whatever project I get dropped into.

With Clojure being not nearly as popular an ecosystem as other options out there, the resources available for new developers on anything like "best practices" for application architecture are not so abundant. The situation is better now then it was many years ago, but there is still not a great amount of resources available. So the likelihood of more and more of these "hodge-podge" code-bases emerging into the future is high, in my opinion.

Errors

If you knew anything about Clojure before reading this post, you probably expected some discussion about errors. And here it is.

Clojure really has terrible error messages. Stuff like Don't know how to create ISeq from ... or ... cannot be cast to class clojure.lang.IFn are favourites. The large Java stack traces that accompany these often just makes it more intimidating, especially when you're starting out with the language.

Over time you build up more familiarity, and at a certain point, you can glance at the message but then zero in on the stack trace, only paying attention to namespaces (packages) that are specific to your project, and figure out the offending bit of code in short order and fix the problem. But it takes time to get to that point. And I'd be lying if I said that after 9 years of Clojure development that these kinds of obtuse error messages still don't sometimes trip me up.

Clojure error messages have very consistently ranked very highly in the State of Clojure surveys on the list of things the community would like to see improved. But still not much improves year to year. At this point I doubt the situation will ever really significantly improve. I think if they (Cognitect) could easily have done it, then they would have long ago.

Documentation

This is not totally consistent across the board. Some projects have exceptionally detailed levels of documentation with plenty of examples. Some don't.

With an ecosystem as small as Clojure, the likelihood that you will run into one or more of the poorly documented libraries is probably a bit higher than it might otherwise be. Often, the best way to understand how such a library works is to read the code, which can take more time.

I think my biggest issue with many bits of Clojure documentation is a lack of descriptions or examples of the data that is expected as input or that is emitted as output. I often find if such a thing is even present, it is just not enough and/or only focuses on the simple cases. Again, this is exacerbated by my previously mentioned issues with dynamic typing. In the absence of documentation, static types to refer to really do help! This issue extends to both functions within your own code-base as well as third-party libraries.

Lack of documentation, or poor documentation, is not a problem unique to Clojure. But I do tend to notice it a bit more in this ecosystem for whatever reason.

The silver lining I suppose is that with access to an excellent REPL, experimentation is easy and relatively quick. So in the absence of good, thorough documentation, you at least have access to good tools to do your own self-exploration.

Clever Programmers

Fair warning: this one is highly anecdotal!

It's been my experience in Clojure teams more than any other that there seem to be more Clojure programmers who get lost in this thought process of "can I do this" versus what you'd instead hope to see "should I do this."

Whether this is overuse of macros, writing their own DSL, jumping on the bandwagon of whatever random new library is making the rounds, I often feel like I need to be really observant on whatever team I'm on to make sure we're not shooting ourselves in the foot by going down a new rabbit-hole because someone on the team got lost in some academic exercise, or some other pursuit to mitigate their own personal boredom.

I also have found in my personal experience more Clojure developers who get stuck in this endless pursuit of writing more "lean" code than in any other language I've worked with. "Less is not always more" is something I've had to repeat many times. Verbosity is not always bad. Sometimes an extra let binding helps improve readability. Sometimes an extra :as name in a destructured map function parameter (even if the name is never used) is useful to help the reader understand what the map is. Sometimes using an if instead of some fancy some-> line that saves you three keystrokes in the end is way more readable. Sometimes writing out an inline function body with named parameters is actually more readable than using partial or whatever other "leaner" thing you could've used.

It really gets tiring.

These types of developers are of course found everywhere and using any language that exists, but I swear that I see more of this type of thing in Clojure code-bases than any other. To me, it's on almost the same level as a Java developer who is getting lost in "gang of four" design pattern madness. Well. Maybe.

Conclusion

There's probably more I could write. And, indeed, I had to stop myself from going on a few tangents as I was writing this! But I think I've covered the main positives and pain points from my perspective.

I think a big theme for myself that I've found over these past 9 years is that as I find myself doing less and less greenfield development in my career, I care more and more about the maintainability of a code-base. And I don't think that Clojure gives you a lot of tools here. Conversely, I think it gives you a lot of ways to shoot yourself in the foot, much like many other dynamically typed languages. I suspect that a lot of the personal pain points I've described above I'd also find in any other dynamically typed language that I use for any significant length of time.

Today, I strongly believe that to be successful with a Clojure code-base over the long term (and, for the record, I do think you can absolutely be successful with it and build up a code-base that is reasonably maintainable), you really need to have a team of disciplined developers. If you don't have that (and, probably, even if you do) the likelihood that your code-base will devolve over time into some tangled mess of "map munging" is high.

As with anything in software development, there are no silver bullets. I don't claim to have any perfect solutions. I certainly do not believe that switching back to statically typed languages will solve all of these problems (I've seen my share of horrific Java code-bases too).

But when I weigh all of these things side by side, I keep coming to the same personal conclusion: all other things being equal, if I have to dive into a huge and ugly code-base, I'd much rather have types than not.