My Love-Hate Relationship With Clojure
I've been working professionally with Clojure now since the spring of 2013 when I stumbled into a Clojure job without any prior knowledge of the language, or any exposure to Lisps, or even functional programming. I took a job that was advertised to be "a majority of PHP development with a little bit of Java and Clojure here and there" and the job instead turned out to be more like 90% Clojure, 9% Java, and 1% PHP.
Getting into Clojure was a hard adjustment for me. I can remember it took me a long time (probably a month at least) before I can say that I really started to "get" functional programming. And probably significantly longer before I think I could reasonably say that I was confidently able to write and architect complex and significant amounts of new code in a "good" functional or data-driven style.
I've now worked at four different companies that used Clojure primarily for development. I've been able to do some greenfield development work, but mostly, I was getting into an existing code-base. I've also authored a few different open-source libraries with Clojure. I don't think I'm some sort of "ultra Clojure guru", but I think it's quite fair to say that I am quite experienced in it.
Looking back at the past 9 years in which I've spent the majority of my working hours continuing to work with Clojure in some form, I think overall it has had a noticeably positive impact on my career and on my skills as a software developer. I think if I'd not been introduced to Clojure when I was, I likely would've continued down a very heavy object-oriented programming path in my career with probably some mix of Java or C#, using all kinds of "gang of four" design patterns everywhere. Today, I shudder at the thought of that, but I remember in early 2013, while I didn't love that stuff and would approach it cautiously, I didn't recoil in horror at the sight of it either. I just saw that stuff as another tool in my toolbox.
I know that nowadays, when I write code in a procedural or object-oriented programming language, I do tend to approach it from at least somewhat of a functional programming mindset. I definitely think this is a very good thing. So much so that I believe one of the best things that any developer can do today to grow their skillset, is to spend some serious time with a functional programming language. Even if you know you'll never be able to use it professionally, try to use one seriously for some side-project. It will really change the way you think about code.
But Clojure is not all sunshine and rainbows. At least, not in my personal opinion. As I find myself today at a bit of a turning point where I am slowly starting to seek out a new job and am actually not specifically looking for Clojure work at all anymore (I wouldn't turn down the "right" Clojure opportunity if one was thrown into my lap, rather I just mean that I am no longer actively filtering my job searching for Clojure only as I used to do), I wanted to take a moment to reflect on that. It's weird for me to say that given what I wrote above about how I think Clojure has had a fundamentally positive impact on my growth as a software developer. And this is why I wanted to capture my thoughts on the language, the ecosystem, and the community (to a certain extent) and what I think has led me down the path I now see myself headed in.
I want to stress here for anyone perhaps randomly coming across my post that the things I am writing about below are all my own personal opinions. You may or may not agree with me. You may have had very different experiences compared to mine. That's all fine. We are allowed to have different thoughts and opinions on these matters!
What I Like
The REPL
The Clojure REPL is such an integral part of developing with Clojure, and to me back in 2013, the experience was eye-opening. I'd not used any other programming language up to that point which had a REPL that was useful as anything other than a toy or simple tool to test out quick snippets in an isolated fashion.
In Clojure (and I guess most other Lisps), the REPL is normally directly connected to your running project and is a way for you to add new code, update/replace existing code, inspect the values or state of anything currently running in your project, run any arbitrary pieces of code against the running project, etc. It's important to note that the Clojure REPL does not force you to write (or directly paste in) all the code to be run after starting it up. You can start a REPL session from your project, or you can just start a new one entirely from scratch. It's up to you. You can run arbitrary code snippets typed into the REPL session of course, but you can also load (and reload) code into the REPL session from your project's source files on disk.
The typical way to do development with Clojure is to start up your REPL, and write code in your editor that will ideally be connected to that REPL session, so you can save code changes to your project's source files as you write it, but also so you can directly run your code changes in the REPL as you write it, to test and see your changes running against the same instance of the application without needing to do a compile and restart in-between.
This leads to a workflow that allows for a high degree of experimentation and iteration. In my REPL-connected IDE (IntelliJ with the Clojure Cursive plugin) I can easily place my cursor over any bit of code in my project and hit Command+Enter to run that snippet in the REPL. As I write Clojure code, my source files often temporarily end up looking like a sort of "scratch pad" of different bits of code snippets while I was trying different things until I finish up whatever it is that I was working on.
You can also connect to remote REPLs, so in addition to your REPL being useful as a local debugger of sorts (you can also use a more traditional debugger with Clojure too), you can also use it to inspect the state of a deployed application in real-time.
The Standard Library
The Clojure standard library is very fully-featured. Not much to really say here,
just that it makes it easy to manipulate data in most any way you would want. I find that even years later I'm
discovering functions in the standard library that do something which I'd otherwise be writing a special-case
combination of map
and reduce
for, or something like that. There really is a lot of very useful stuff at your
fingertips here.
Functional Programming
As I've mentioned before, this took a while to finally "click" in my head. But when it did I found it really refreshing and eye-opening. I also strongly think it has improved my skill-set as a developer overall, even when returning to non-functional programming languages like Java or C#.
Clojure definitely is a functional programming language, though it does give you many escape hatches should you need to do things another way which can sometimes come into play, e.g. when doing Java interop. Or really, when you just have this realization that for some arbitrary piece of code you figure you can more clearly implement it by ignoring some functional programming practices. Clojure is mostly pragmatic in this way, but the language does "default" to a functional programming paradigm.
Immutability First
During my transition towards thinking in a "functional programming" way, I found this annoying. But once it clicked in my head, it really clicked and was great.
Being able to write code that passes around values like lists, maps, etc (which you will do a lot in Clojure) and transform them in one function for some purpose, while not having to worry about things like "oh, I should maybe clone this first so I don't mess up the value for the caller" is very freeing.
Of course, this only applies to the built-in Clojure data structures. If you're working with a Java object directly, even in your Clojure code, then you of course do not get any immutability (unless that Java object is providing it internally).
Built-in Data Structures
The aforementioned data structures provided with Clojure are great to work with. The literal support for lists, vectors, maps and sets are all invaluable and you use them all the time. It's really a night-and-day shift to go from writing out a map definition in your code in Clojure vs Java or C#.
These are what really make the whole "code is data" thing such a joy to work with.
Interop With Java
The interop story with Java is pretty good. For most things you'd want to do here, I would say it works quite well and you won't encounter much difficulty. For some uncommon advanced uses it can get a little bit awkward at times, but I think writing Clojure code that utilizes some Java library isn't too difficult in the majority of cases.
And you will be doing this at some point as a Clojure developer. The situation today is even better than it was in the early 2010's where the community had not yet created all kinds of Clojure libraries to wrap Java libraries. Nowadays, there probably is a Clojure library available to make it a little bit easier for you to use whatever Java library you need in your project. But there still might not be. Or, the library that exists might not be maintained anymore. So in the event that you need to dig in for yourself, it's often not too difficult to write your own interop layer.
And because Clojure runs on top of the JVM, you of course are getting access to the entire Java ecosystem of libraries which is huge of course.
ClojureScript
ClojureScript is the single reason why my current JavaScript expertise basically stopped at jQuery with a very, very surface-level exposure to React directly.
This is because in 2013/2014 when I had to work on some front-end heavy web applications, I was introduced to the amazing Reagent library. This is a ClojureScript layer on top of React that really renewed my enthusiasm for writing front-end code after dealing with some disastrous jQuery mess and not particularly liking what I was seeing with using React directly with JavaScript (in particular, it seemed like a fair bit of boilerplate was going to be needed to create components).
Writing React components with Reagent and ClojureScript was just amazing in comparison to using React with JavaScript
directly. Your React components were just functions! HTML could be written directly in your component functions through
the use of Hiccup-style syntax for HTML, providing very similar functionality
to React's JSX. And through the use of Reagent's
"Ratoms" they would automatically refresh
when application state changed. And throughout all of this, you were just using normal built-in Clojure data structures
and functions. A Reagent "Ratom" is just a Clojure atom from the developer's
point of view and you can update it just the same way you would any other atom with stuff like swap!
and reset!
.
When combined with Figwheel to get live hot-reloading of code in your browser, this provided
you with a really powerful development workflow. I think the JavaScript ecosystem has now finally caught up here with
code hot-reloading tooling for the most part, but this was really something to experience in the early 2010's.
Leiningen
There are a few different build tools for Clojure projects available nowadays, but Leiningen was there from pretty early on and very widely used in the community. This seems to be slowly being replaced with Cognitect's own solution in the community, but I've still not ever used it myself. It was a (to me) curiously late effort by Cognitect who seemingly didn't care about providing a build tool to go along with Clojure and left it up to the community for many years.
Leiningen has never been a perfect tool, but it has been pretty solid overall. Some advanced, large projects can
end up with project.clj
files that are rather unwieldly, but for the most part I found it pretty easy to work with.
I particularly liked that, when combined with lein-cljsbuild, you had very easy entire project building for both server-side and client-side code all via Leiningen. Compared to most other programming languages and build systems where you'd be wrangling one build system for your server-side code, and then probably one or more others for your client-side JavaScript. Leiningen is just amazing in comparison to that.
What I Dislike
You'll note that the above sub-title is "what I dislike" and not "objective problems with the language" or something like that. If you get all hot and bothered by anything I write below (basically, if you turn into this while reading), then I would suggest you should step away from your computer for a few hours to get some perspective.
"Map Hell"
Oh dear. Dynamic typing vs static typing. Am I really going to go there? I hate discussing this. I hate discussing it because in my personal experience, most developers are incredibly immature when it comes to this topic. Often people get emotionally involved and feeling like they're being attacked personally by these discussions. It almost always seems to devolve into people cherry-picking their favourite examples showing why such-and-such a thing isn't possible with whatever typing system they dislike. Or you get people making grandiose claims on either side about how "well, in my experience this doesn't happen, so that's not really a big benefit" etc, etc. And finally, in many cases it sadly devolves into personal insult flinging. Yes, really. I've seen it far too many times, even from people who are seen as industry or community "leaders." There are unfortunately a couple people in the Clojure world specifically who are really guilty of this but are also often looked up to. It really is sad and very unfortunate that we cannot discuss this topic rationally in many cases.
I have my own opinions on this topic. You may too. It's all good.
My experience on large Clojure projects is that your code is, in most cases, wrangling un-typed maps all over the place.
Now, this is blatantly obvious when you think about it because Clojure is dynamically typed and has great, easy-to-use, built-in data structures like vectors, maps, and sets which are all quite natural to use to represent your application's data with. Whereas in Java (or any other statically typed language), you would probably be writing types/classes for all kinds of things to represent your application data, in Clojure, you often just stuff your data into a map or vector. Easy-peasy.
Without getting into a huge amount of highly project-specific details, the problems I almost always see in large Clojure projects is that you end up with a large amount of functions all over your code that are operating on seemingly random maps which can often look very, very similar to the maps that other functions are working with at first glance. They'll often be named 100% identically too.
For example a map named patient
that is accepted by a great many different functions in your code base. And you start
seeing the code all over using these patient
maps and the :id
, :name
, :mrn
etc keys all over and you start
building up this intuition in your head that all these patient
maps are all the same. But then the inevitable
surprises happen (maybe early on, or maybe much later) when you discover some odd function or small collection of
functions scattered around where actually these patient
maps are not always exactly the same as the other patient
maps. They're like 99% the same in most cases, except these few times where the :visits
key is holding data that is
basically the same but it's maybe expected to have been pre-processed a little bit different first. Or some
:discharge-date
key is expected to have the same basic date in but having been converted to another datatype first
(e.g. in some cases it's expected to be a string and in a specific format (that may differ between different
functions! oh what fun ...), in other cases it's expected to be a Date
or another specific date type). Or a common key
is actually expected to not be there this time or else something else down the call chain blows up! Or ... etc etc,
you get the idea.
I wish I could say that this was uncommon. It's not like 50% of your average code base is going to be like this, but it does crop up in a not-insignificant amount of cases (in my anecdotal experience), and it is almost always a surprise that you waste a bunch of time debugging before realizing that "oh, it's because the data is supposed to be different in this one case, oops." And then you start thinking "well darn, I wish these odd-ball functions/parameters were named differently to better indicate this" but of course they never are. Or you start wishing that this section of code was just designed differently to begin with. "The data should flow differently here because it's actually not really the same type of data!" or something like that. This kind of thing happens, I think, often because dynamic typing makes it easy to quickly make these types of additions or updates to a code base. You don't have to think too hard when you're writing the code, you can often times just start experimenting with your REPL and bam! Feature done! That was easy!
I think this type of problem also negatively impacts the speed at which new developers can onboard onto an existing code base of significant size.
Static typing is not a silver-bullet here. I am not trying to make this claim. You can and will come across similar types of problems in a code base developed with a statically typed language (though, to a lesser degree, as with statically typed languages that have type systems that are not shitty like Java/C#/C++, you often have much more ability to catch these problems at compile time). The code in projects of significant size, regardless of static or dynamic typing, almost always becomes messy with poor design decisions leaking out all over the place. It happens.
But, my main take away from the Clojure "map hell" problem, is that we as developers spend most of our days reading code. And I strongly believe that types help to optimize for reading and understanding code, so why would you want to cut these out over the long-term? I think this is the seductive nature of dynamic typing that (in my experience) does come back to kick teams in the ass, especially so in the later maintenance years of many projects.
But, having said all that, I do also strongly believe that a highly disciplined team can absolutely produce a large code base that is reasonably maintainable in any dynamically typed language out there, including Clojure. I just unfortunately don't believe that the majority of developers are highly disciplined, heh.
Data/Type Annotation Libraries
Here I'm referring to stuff like Schema and the more recent Spec which is Cognitect's own solution.
You'd think that after reading my previous thoughts on dynamic typing and maps that I'd actually love these libraries bringing the ability to annotate your Clojure code-base with type annotations and specifications. I thought I would too. I used Schema on quite a few projects starting from late 2014.
The problem in my experience is the optional nature of them. Most programmers are, in my personal experience, quite lazy and, I'm sorry to say, lack discipline when it comes to keeping a code base neat and tidy. It's hard, I get it. And the realities of the real world are almost always at odds with this and instead seem to reward rushing out quickly thrown together solutions. "The client doesn't care about how pretty your code looks" and that type of thing. I get it. Though, the client will probably care when you're taking longer and longer to deliver new features down the road, but hey, let's just ignore that ...
Type annotations with Schema (and later, Spec) are almost always one of the things that gets left to languish in any
project I've worked on. Or, developers take the "easy" way out and use schema.core/Any
(which is a catch-all that
matches anything, even nil
) or find other ways to only write a partial definition, assuming they write one at all.
The only way I have found to force developers to keep these definitions maintained was to keep run-time validations
turned on, and not everyone always wants to do that. And even if they did, they probably want to only have validations
run in certain cases, not *everywhere*, for whatever reason. And to be honest, they might not even be wrong to
not want them turned on everywhere. And that's a part of what makes this so aggravating for me. It'd be easier if this
was much more "black and white", heh.
But the main reason why these data annotation libraries don't take care of my previously mentioned issues with dynamic typing is that most Clojure developers will not want to annotate every function in a code base. The common tactic here is to create type annotations at the boundaries of your code. For example, in your HTTP endpoints, or just at the public API of a library you're developing. That kind of thing. But then you rarely see anyone adding type annotations anywhere for anything internally in your code-base. This is unfortunate because the internal/private code that they often don't want to annotate is also usually the largest section of your code base. Oops!
And I guess I can't blame them, because when you start going down that road of annotating a whole lot of things, it kind of removes many of the benefits of Clojure being a dynamically typed language. It really is a double-edged sword!
"Choose Your Own Adventure"-Style Project Architecture
Clojure has nothing even approaching the Ruby on Rails framework. And the community doesn't even want it. The only initiative I am aware of that seemed like it was sort-of approaching that kind of scope was the ill-fated Arachne Web Framework which seemed to receive a ton of criticism in the community and just died out.
So that's fine I guess, the community would rather string together their projects like they were lego blocks by grabbing whatever libraries they need for things like HTTP routing, database access, security, JSON parsing, etc themselves directly. To make it easier, people have maintained project templates to help others get started with bootstrapping a project with a curated set of libraries and some starter infrastructure code.
At its core, this is kind of exciting if you've spent a long time dealing with some big framework in another programming language that you weren't a fan of. "Finally, I can set things up exactly how I want them!"
Of course, it opens up a ton of ways to shoot yourself in the foot. You now need to be well versed in a lot of different libraries and how they are pieced together. Even if you are using a project template to get your project started, the responsibility is still on you to keep things up to date as your project ages. Once you've created your project from a template, that template won't help you anymore to keep things updated. Libraries may change over time, the way that you integrate them together may need to change with them. It'll be up to you to stay on top of this. This problem of course does exist also when using a large framework in any other language. But with such a large framework, you usually have a single place to go for documentation and help with performing upgrades. With a collection of different libraries, you now have to hope that each of those library authors is diligent about documentation! And in more difficult cases, you need to hope that someone else has experienced the same upgrade difficulties with your chosen combination of libraries and has shared their solutions. Otherwise, you'll be on your own.
Some recent efforts have emerged to help fix this by attempting to modularize more and more of a typical Clojure project's architecture and in essence, turning the old project template idea into something much more similar to a real "framework." But it remains to be seen how (or if) this really improves long-term maintainability in practice, as it's still early days. I'm personally not sure if this is at the right level to really tackle the longer-term maintenance problems, but it does at least feel like a step in the right direction. But regardless, you will find projects in Clojure in the future that are not built with this particular "framework" because people have a choice, and I've observed most Clojure developers really do seem to love making the choices themselves when starting out a new project.
Also, hopefully one of the libraries you chose doesn't stagnate because its author lost interest in maintaining it. If that happens, well, good luck, hopefully it is easy for you to move on to something else. For what it's worth, this could also be seen as a benefit of this style of project architecture. If this situation does occur, then in theory it is easy to migrate only the affected portions of your code to an alternative. In theory ... Also for what it's worth, this is always an issue in any project, regardless of the programming language used. I just often feel like Clojure is at least a little bit more susceptible to it due to the "preferred" way to string together a project.
Anyway, this "choose your own adventure"-style of project architecture has, in my personal experience at least, lead to a reality where every single Clojure project of significant size is its own unique snowflake. There may be some loose similarities here and there, but even those similarities (if they even do exist) are nothing like the sort of regular code structure you might see across any number of Rails projects (for example).
This, to me, presents a huge double-whammy effect when it comes to getting familiarized with a Clojure code-base when combined with my aforementioned issues with dynamic typing and "maps everywhere." I guess this is really important to me as I rarely get to do greenfield development anymore and instead increasingly find myself cursing the names of the previous developers of whatever project I get dropped into.
With Clojure being not nearly as popular an ecosystem as other options out there, the resources available for new developers on anything like "best practices" for application architecture are not so abundant. The situation is better now then it was many years ago, but there is still not a great amount of resources available. So the likelihood of more and more of these "hodge-podge" code-bases emerging into the future is high, in my opinion.
Errors
If you knew anything about Clojure before reading this post, you probably expected some discussion about errors. And here it is.
Clojure really has terrible error messages. Stuff like Don't know how to create ISeq from ...
or ... cannot be cast to class clojure.lang.IFn
are favourites. The large Java stack traces that accompany these
often just makes it more intimidating, especially when you're starting out with the language.
Over time you build up more familiarity, and at a certain point, you can glance at the message but then zero in on the stack trace, only paying attention to namespaces (packages) that are specific to your project, and figure out the offending bit of code in short order and fix the problem. But it takes time to get to that point. And I'd be lying if I said that after 9 years of Clojure development that these kinds of obtuse error messages still don't sometimes trip me up.
Clojure error messages have very consistently ranked very highly in the State of Clojure surveys on the list of things the community would like to see improved. But still not much improves year to year. At this point I doubt the situation will ever really significantly improve. I think if they (Cognitect) could easily have done it, then they would have long ago.
Documentation
This is not totally consistent across the board. Some projects have exceptionally detailed levels of documentation with plenty of examples. Some don't.
With an ecosystem as small as Clojure, the likelihood that you will run into one or more of the poorly documented libraries is probably a bit higher than it might otherwise be. Often, the best way to understand how such a library works is to read the code, which can take more time.
I think my biggest issue with many bits of Clojure documentation is a lack of descriptions or examples of the data that is expected as input or that is emitted as output. I often find if such a thing is even present, it is just not enough and/or only focuses on the simple cases. Again, this is exacerbated by my previously mentioned issues with dynamic typing. In the absence of documentation, static types to refer to really do help! This issue extends to both functions within your own code-base as well as third-party libraries.
Lack of documentation, or poor documentation, is not a problem unique to Clojure. But I do tend to notice it a bit more in this ecosystem for whatever reason.
The silver lining I suppose is that with access to an excellent REPL, experimentation is easy and relatively quick. So in the absence of good, thorough documentation, you at least have access to good tools to do your own self-exploration.
Clever Programmers
Fair warning: this one is highly anecdotal!
It's been my experience in Clojure teams more than any other that there seem to be more Clojure programmers who get lost in this thought process of "can I do this" versus what you'd instead hope to see "should I do this."
Whether this is overuse of macros, writing their own DSL, jumping on the bandwagon of whatever random new library is making the rounds, I often feel like I need to be really observant on whatever team I'm on to make sure we're not shooting ourselves in the foot by going down a new rabbit-hole because someone on the team got lost in some academic exercise, or some other pursuit to mitigate their own personal boredom.
I also have found in my personal experience more Clojure developers who get stuck in this endless pursuit of writing
more "lean" code than in any other language I've worked with. "Less is not always more" is something I've had to repeat
many times. Verbosity is not always bad. Sometimes an extra let
binding helps improve readability. Sometimes an extra
:as
name in a destructured map function parameter (even if the name is never used) is useful to help the reader
understand what the map is. Sometimes using an if
instead of some fancy some->
line that saves you three keystrokes
in the end is way more readable. Sometimes writing out an inline function body with named parameters is actually more
readable than using partial
or whatever other "leaner" thing you could've used.
It really gets tiring.
These types of developers are of course found everywhere and using any language that exists, but I swear that I see more of this type of thing in Clojure code-bases than any other. To me, it's on almost the same level as a Java developer who is getting lost in "gang of four" design pattern madness. Well. Maybe.
Conclusion
Having written all of the above, I find myself asking the question: "Do I like Clojure?" And I can still honestly answer "yes!" I think it is a very productive language that has some downsides that any developer working with the language should be aware of. I think the language rewards disciplined teams and probably punishes (to varying degrees) those teams lacking in discipline over the long term. I also strongly believe that the ecosystem has a lot going for it, even despite some of the difficulties I wrote about above.
But I do really strongly wish the community would start looking at itself more objectively and start ridding itself of some of the "cult-like" behaviour you see. Honestly though, this is common behaviour for any Lisp-derived language.
There's probably more I could write. And, indeed, I had to stop myself from going on a few tangents as I was writing this! But I think I've covered the main positives and pain points from my perspective.
I think a big theme for myself that I've found over these past 9 years is that as I find myself doing less and less greenfield development in my career, I care more and more about the maintainability of a code-base. And I don't think that Clojure gives you a lot of tools here. Conversely, I think it gives you a lot of ways to shoot yourself in the foot if you're not careful, much like many other dynamically typed languages. I know that a lot of the personal pain points I've described above I'd also find in any other dynamically typed language that I use for any significant length of time.
Today, I strongly believe that to be successful with a Clojure code-base over the long term (and, for the record, I do think you can absolutely be successful with it and build up a code-base that is reasonably maintainable), you really need to have a team of disciplined developers. If you don't have that (and, probably, even if you do) the likelihood that your code-base will devolve over time into some tangled mess of "map munging hell" is high.
As with anything in software development, there are no silver bullets. I don't claim to have any perfect solutions. I certainly do not believe that switching back to statically typed languages will solve all of these problems (I've seen my share of horrific Java code-bases too).
But when I weigh all of these things side by side, I keep coming to the same personal conclusion: all other things being equal, if I have to dive into a huge and ugly code-base, I'd much rather have types than not.