Common Lisp, Clojure, and seriousness.

Brian Carper described a few days ago, how Clojure is better (for him) than Common Lisp (actually, SBCL).  I managed to dig through ensuing flame war, but it seems like nobody in the flame war realized (or it wasn’t stressed enough) that original post is actually comparing apples to oranges, a serious language to a toy language.

A language, to be considered serious, needs to be self-sufficient, a serious language can’t be a mere parasite on some host language or environment, and its bus factor can’t be finite.  That translates to just a couple of features:

  • A serious language has to have a defining standard, it can’t be implementation-defined.  A good standard bumps language’s bus factor to aleph null: after a nuclear catastrophe, archæologists of future generations should be able to re-implement the language on any hardware, including the Calculor;
  • A serious language has to be self-hosting, or at least have some self-hosting implementations.  Only then language stops being dependent on other languages.

Leaving aside other important traits (such as having multiple implementations, having a machine code compiler, extensibility, and so on), these two alone are necessary and sufficient for language to be serious; all programing languages not having these traits are just toys that can’t be guaranteed to last.

Toy languages may stay this way (as did Tcl and Perl, I think, and TeX is one-of-a-kind toy language which managed to grow enormously, staying implementation-defined and standardless).  On the other hand, starting a toy language may be just a way to bootstrap a serious language without starting from gathering a commitee, and I think that’s what is currently happening to Python.  I hope that will also happen to Clojure; I looked at it, and I liked what I saw (good concurrency support, decoupling polymorphism from data structures even more than in Common Lisp, a tiny bit more modern syntax, and it feels really well designed), however at the moment it is a toy language, feeding off Java, at very early stage of development.

Toy languages have short-term advantages (they are fast to create, usually simple, they leverage the host environment (libraries, operating system interface), they solve problem at hand quickly), and long-term disadvantages (lack of stability).  Serious languages have short-term disadvantages (multitude of slightly differing implementations to choose from, every single one of which has its own differences and extensions to the standard; implementation/library compatibility matrix; standard features are set in stone and it may be tricky to work around them), and long-term advantages (stability, one can rely on a serious language to be there in one form or another in ten years).

I have nothing against toy languages: they do their job, and do it well.  As a glue language, as a scripting language, as a solution to short-term problems, they’re good enough, and short-term advantages win over long-term disadvantages.  Brian was trying to solve a short-term problem and, not surprisingly, toy language worked better.  But: will the solution still work in ten, twenty years?  Heck, will it work with next release of Clojure, or next release of JVM, or if Rich Hickey (Clojure author) gets hit by a bus?  Brian didn’t need to answer these questions; do you?

When you get to choose a programming language, or any other tool, you may have some personal preferences, but don’t forget to ask yourself the single most important question: What is the problem you’re trying to solve?  The answer to this question determines all further questions and answers.

24 thoughts on “Common Lisp, Clojure, and seriousness.

  1. There are certainly some toy-like aspects of Clojure. At this stage its just starting to emerge from hobby-project like status for sure, but that doesn’t mean you should dismiss it.

    If the JVM happens to run on a platform that you’re interested in, then the fact that it runs on the JVM is a huge asset. You get about 500 man-years of work in a virtual machine on your side for free, amongst other benefits.

    The SBCL source includes a number of files written in C so maybe that doesn’t qualify as self hosting either? Just wondering.

    I think railing against Clojure because its relatively new, or because its based on the JVM is a mistake. There’s good stuff there, and its worth looking at. If not now, then maybe a year or two down the road if it continues along the path of goodness.

    Just something to think about.

  2. I agree on standard definition, preferably with standard test suite, but self hosting should no longer be a requirement.

    The JVM is standardized, widely deployed, and just as reliable a target for the future as the x86 instruction set is. The nature of “self hosting” has changed in the modern era, and running on a stable VM such as the JVM should not be considered a downside.

  3. Why so serious, dude?

    Really, if Common Lisp wants to be “serious”, it can sit in the corner looking all conservative and shit while Clojure gets all the pussy.

    Toy language or not, you can actually do stuff with Clojure. With CL you spend 95% of your time with implementation compatibility headaches, library integration issues, foreign interface crap, or getting basic stuff like threads and networking to work on multiple platform or even a single platform.

    In Clojure, you can run anywhere and use a huge code base without exerting a single neuron of yours. You have plenty of services such as multithreading and network access out of the box, which for a CL refugee is miraculous (I know). You can write concurrent programs that actually work without pulling your hair out. Oh, and you don’t need to read a whole red-tape book to understand what’s going on, what’s supposed to happen, or how to use it.

    Clojure is Lisp’s future. Common Lisp is for old serious bureaucrats. All these wrinkled serious guys do is maintain the status quo. Nothing innovative or useful will ever come out of them.

    Get with the times, man–Use Clojure.

  4. This article misses the point of Clojure. Hickey has made the case that serious language development shouldn’t be wasting time on trying to re-solve solved questions, neither should it be eschewing the work of others.

    From his perspective, it’s like insisting on programming without libraries and insisting on doing everything yourself.

    Rather than saying it sucks because it doesn’t self-host, it would be more useful to direct one’s attention to Hickey’s arguments about the JVM directly and then make the case for self-hosting in that light, if that is what one wants to do.

  5. Thank you all for your comments. Looks like I didn’t word my thoughts clearly enough, or like readers merged my thoughts with other lispers’ critique of Clojure. People are replying (here, on reddit, and on #clojure) to things I didn’t write (or that I didn’t intend to write).

    First, I don’t dismiss Clojure. I like the language, the ideas, it fixes many things that CL did wrong, or that would need a new standard to clean up (concurrency with STM, clean interfaces to emulate basic structures, laziness, and some more). I will certainly use it, and hope it gets just a little bit more serious (as in: reimplementable).

    I also didn’t write that anything “sucks”, I wrote only that CL and Clojure are not only different tools, they belong to different classes of tools, occupy different niches and have different uses. There are projects where short-term advantages of Clojure win, and there are projects where long-term advantages of CL win. I didn’t critique Clojure, I critiqued original post (and the Clojure/CL flame war itself) as comparing apples to oranges.

    Most commenters seem to assume that when I say about language being self-hosted, I’m critiquing JVM and, by extension, all kinds of FFI. Not true. Self-hosting does not mean being “turtles all the way down”, same language from top to bare metal. It means that core of the language is implemented in that language, nothing more. Using Java VM as a platform is not wrong at all (as long as it doesn’t exclude porting to other platforms); depending and feeding off Java the language for Clojure core implementation is. It doesn’t look to me like Clojure has means to be efficiently implemented in itself at the moment—implementation relies on Java’s interfaces, which need to be created at Java level. If I am wrong, correct me, please, maybe I didn’t understand the implementation or the language well enough.

    With this definition of being self-hosted (which I believe is the proper one), SBCL is a self-hosted compiler; C files are used to load up and start the Lisp image, in a C-based operating system. That’s just a bootstrap, not the language core. There’s also nothing wrong with FFI and using existing ecosystem (and I don’t remember writing that there’s anything wrong with it); I for one prefer FFIing to Java to FFIing to C.

    As for the standards and “red-tape books” needed to understand: one of things I appreciate most when working with CL is that I don’t have to pull my hair out when a bit of language doesn’t behave as expected (or I’m not sure what to expect, and I don’t believe Clojure is so intuitive I’ll never have that kind of problem; the only fully intuitive interface is a nipple, anything more is matter of learned expectations), because I have the definitive answer in CLHS. In most of other languages, edge cases aren’t definitively described anywhere (not even as having undefined results), and when I hit a problem, I’m into googling obscure forums and mailing lists, reading into implementation, and experimenting. With CL, I just look it up in CLHS, and in most cases answer is already there; if it’s not clear, I have a good starting point to asking a question.

    To sum it up, I like and appreciate Clojure, will certainly use it when it solves my problem, and will keep eye on it. Using the existing ecosystem and fitting into it is OK. I understand that Clojure is young, and I hope it will grow up to be a good, serious language.

  6. What’s self hosting useful for? Why would you want to not depend on other languages? I’ll leave aside romantic autarky. The main pragmatic uses I can see are self hosted ultra-light systems over the bare metal, like Forth – but those have very limited utility nowadays – or cross platform portability, where you want a defined thin interface to the outside world, and the bulk of the language built over that. That latter is the reason for SBCL’s design, I think. But Clojure is on the JVM, it only has one platform. Why then should its internals not be coded in the JVM equivalent of C?

  7. Thanks for the clarification Maciej. If you don’t mind, can you explain what you use CL for today? Is that work related or other?

  8. @Maciej: I think one thing that gets people worked up is pejorative words like “toy” and “parasite.”

    Two things are certainly true:

    1) Clojure is a very young language.
    2) Clojure directly targets a single execution environment (the JVM) and leans on it in a number of big ways.

    When Lisp was Clojure’s age, it was still so hard-coded to its implementation on the IBM 704 that two of its core functions were named car and cdr because of the machine instructions underneath. (That’s not a fair comparison, of course, because Clojure really is a Lisp). At this point in Lisp’s history a standard was still 25 years in the future.

    Rich Hickey has made a number of decisions to accelerate the development of Clojure by tying it more closely to the JVM. These decisions run counter to the “old” wisdom of how you build a language, but definitely seem more appropriate for the “modern” age.

    I fully expect that the kind of edge case details and other things that move you towards a standard will be nailed down as Clojure matures. Since it’s so closely tied to the JVM, much of the spec is “defined” by what’s in Java. The actual surface area of Clojure itself is fairly small.

    One of the fascinating developments of the last few years is languages targeting the JVM and CLR with the express purpose of interoperating in a tight way (Groovy, F#, Scala, IronPython, …). I’m waiting for the first Clojure-based distributed substrate that’s adopted in J2EE environment (though there’s still some work to be done on Java-to-Clojure calling environment).

    (As an aside, “Controlled Tango Into Terrain” is one of my favorite blog posts ever. Keep up the interesting stuff.)

    @Blue Phil: You made me laugh. Reminds me of the “I’m a Playstation; I’m a Wii” video: http://de.sevenload.com/videos/fRk72JPy-Playstation-3-vs-WII

    But in the end, I wouldn’t rely on Clojure getting you any. Everybody knows only Ruby can do that. :-)

  9. @Julian Morrison

    Self-hosting in and of itself has no direcly practical goals, but it ensures that language is expressive enough to hold its own definition, and that language’s definition doesn’t implicitly depend on other language’s semantics (e.g. Clojure depends heavily on Java (the language, not the VM) interfaces behaviour). Dependence on single language together with being implementation defined leads to unclear behaviour, where parts of semantics, esp. language core semantics, come from wholly different language.

    @Luc Préfontaine

    Congrats on putting Clojure in production! Seriously. Please, read my explanation above: I didn’t intend to dismiss Clojure, and I think other examples of languages I called “toy” (Perl, Tcl, TeX) should tell that clear enough. It doesn’t mean it’s unsuitable for production, or anything; it only means that convenience of single, Java-based implementation has its price. Simple trade-off, if Clojure (or any other tool) solves your problem, good for you. I just think Clojure has potential to actually become a hundred-year language, for which it would have to grow out of implementation-defined functional programming layer on top of Java.

    @Bill Robertson

    I used CL in my job: I hacked Embeddable CL (http://ecls.sf.net/) to be used in embedded Linux boxes, and later I developed and maintained UnCommon Web based (http://common-lisp.net/project/ucw/) middleware web-app, contributing to UCW itself. I also completed a Google Summer of Code project in CL: http://common-lisp.net/project/cl-openid/. At the moment, I don’t use Lisp professionally (only hobbyist/university stuff, mostly Web: Hunchentoot/Postmodern, with a vision of completing my own toolbox-framework on top of those), but I plan to go back to using Lisp (CL or Clojure, whichever is more handy for the problem at hand) as soon as possible.

    @Tom Faulhaber

    To get things straight first, you seem to be mistaking me with Maciej Cegłowski of idlewords.com, who wrote the “Controlled tango into terrain” post. I have the same name, but that’s all to it. The post, however, seems interesting (didn’t finish reading it yet), thanks for the link. :)

    I think you’re right, words I used may seem pejorative. Too much Usenet flame wars, I think, I got used to phrase myself in a direct way, which may be misread. The “toy” word was just first that I had on my mind, with no offense intended. And for “parasite” thing, the intended meaning was not pejorative: being a parasite is just an evolutionary strategy, and since we’re talking about languages/libraries/platforms ‘ecosystem’, it seemed natural way to say what I meant: a strategy as good as any other, having its merits and downsides, and having a potential to evolve into a commensalism, mutualism, or to grow out of strict dependence (which is what I hope will happen, Clojure is too good a language to stay hard-wired to a single host).

    A side note on parasite thing: when I was taking a university course in molecular evolution, professor said in one of the lectures that in these days cats can be considered parasites feeding off humans. Not literally eating people, of course, but cats held as pets provide no actual value besides being cute, yet we feed them, and keep them warm and healthy. Cats co-evolved with humans and some of them successfully employ this kind of strategy, and it doesn’t seem to me there’s anything wrong with it. I personally find this view fascinating.

  10. > A language, to be considered serious, needs to be self-sufficient, a serious language can’t be a mere parasite on some host language or environment […]

    To be considered serious by *you*, maybe–but that is an incredibly narrow-minded definition of “serious”.

  11. I agree with him in regards of the reliability, and the new languages should have “some sort” of backward compatibility as Java and C# does, however the JVM is stable enough, and with just a little coordination this problem should be solved.

    What i mean is, before the 1.0 relase all things should be tried, after that caution should prevail.

  12. The best things to do when there is a flame war is not to respond. If someone finds some lisp dialect better for solving his problem then let him use it. But there were, are and always will be other choices. Lisp is the language of liberty. If any lisp dialects succeeds to enter the mainstream it would be a great news for all the lispers.

  13. I might be projecting my own views onto you, but I suspect that you’re using a very idiosyncratic and almost fictional definition of seriousness. The kind that used to be glorified in the hard science fiction mostly — grownups doing important work, solving hard problems towards the long term betterment of human kind. When we send starships to distant galaxies, the ship support and navigation system will be written in common lisp ;) Or perhaps lisp is the language Hari Seldon, from Asimov’s Foundation Trilogy, would’ve used to model the psychohistorical development of the empire in the next few thousand years. Or the scenario that you brought up (for some reason I’ve heard that more then once from various lispers) — being able to reconstruct lisp code after this civilization gets replaced by the next one. I think Naggum in his more general rants laments the disappearance of that kind of seriousness. (this rant for example http://groups.google.com/group/comp.lang.lisp/msg/856bccf3eff6ab53 ) His posts made me appreciate common lisp from that perspective.

    As such I suspect that your notion of seriousness will be misunderstood (certainly one can write business oriented, “serious” code in clojure or jruby or whatnot). More importantly it’s not even common lisp specific– it has to do with the priorities of the “free market” instant gratification globalized society that we live in.

  14. It may sound rather dubious to say that the only spec for something
    is its implementation. It sounds like the sort of thing one might
    say about C++, or the Common Lisp loop macro. But that’s also how
    math works. If the implementation is sufficiently abstract, it
    starts to be a good idea to make specification and implementation
    identical.

    From Paul Grahams Arc tutorial. So he’s abandoned seriousness as well then?

    • C++ has a pretty comprehensive standard, and — as far as I know, if I’m mistaken, prove me wrong — no full implementation of it at all. LOOP macro is also pretty well defined by ANSI and CLHS. The “math” argument is *very* abstract and I have no idea what author actually meant — as I understand it, the “specification” (theorem — its hypothesis and conclusion) is pretty separate from “implementation” (proof of the theorem), and there are indeed multiple proofs for many theorems. I really can’t imagine what is Graham’s point here, since all examples are wrong, and “making specification and implementation identical” does not seem to be equal to “implementation-defined”. See, for example, literate programming, great example of which is TeX – reference implementation contained within a specification.

  15. (
    Whatever personal programming story I have and stories I saw … to the core concept, concerning me at the moment:
    )

    LISP IS A MYTH! Everything I have heard about Lisp is just a myth and no Lisp Development Environment (LDE?) exists out there; no stable enough implementation; no real success story; it is just PG and copies of that! After 50 years if really Lisp has anything “pragmatic” to offer, I (for one) should not programming in C# (Or Java, PHP, …) by now.

    (
    And I know: every cool idea is a Lisp idea and all people out there are thieves! It is just there is no stable Lisp implementation of the idea and Lisp was so busy with being elegant that could not waste time with being pragmatic.
    )

    Note: Maybe it never happens to me to develop in a dynamic language. But this harsh rant on Lisp (and it’s community) is an extremely obsessive and positive one. So to Lisp community: Clojure is not a toy language; it is a usable Lisp and that’s you do not like about it.

  16. As someone who uses Common Lisp to put grub on the table, I would say that using Lisp to do serious work works well. And I actually like CFFI. :) Boy am I glad to have left the C++ / C# / Java / Python etc world behind.

    I think the community that drifts from CL towards Clojure are those that deeply appreciate the JVM. Personally, I would have preferred Richs efforts to have taken place in SBCL, or one of the existing CL implementations. It definitely would have improved the state of Lisp VMs / Lisp libraries. Paul Graham also could have gone there – but I suppose his changes are purely syntactical anyway.

    Its funny because five years ago I had a lecturer who was addicted to C#, and when we spoke about Lisp he called it a Toy LanguageTM. His next question, which pushed his previous statement into the land of questionability, was: “But does it support interfaces?”.

  17. I realize this is old, but it came up at the top of a Google search. So here goes.

    First, I think your definition of “serious” language is somewhat flawed, and I think you place too much value on having a standard. Sure, CL has an ANSI standard. But it leaves so much undefined that there are still serious incompatibilities between the different CL implementations, and no guarantee code you develop for one will work in another. For example, the SBCL socket API, and the CLISP socket API, are very different. This has resulted in having to create compatibility layers such as usocket in order to write portable socket code in CL.

    Clojure, on the other hand takes advantage of the Java socket API. And the Java socket API is the same on all Java implementations running on all platforms. So whether you are using Sun Java. OpenJDK, IBM Java, or GCJ, you can be sure that the socket API will be the same on all of those platforms. So even though CL has a standard, Clojure still comes out ahead as far as guaranteeing portability.

    I also think you are greatly underestimating the value of Clojure being built on the JVM. The very fact that it is built on the JVM and can leverage existing Java code bases is what gives it a very good chance of becoming a “serious” language. On the other hand, many of the languages that have ANSI standards are basically academic toys that are difficult to use in the real world because of their inability to integrate with existing systems, as well as their lack of useful libraries to solve common day to day problems.

    When talking about Java, it’s important to distinguish between Java the platform, and Java the language. Java the platform is still very powerful, flexible, and offers a lot of advantages.

    Java the language, on the other hand, has run its course except for systems programming. It was great for it’s time. But by modern dynamic language standards, it’s verbose, very complicated, and no one really wants to write new code in it. But there are billions of lines of Java infrastructure already in place. Rewriting all of it would both be cost prohibitive, and time prohibitive.

    And this is where Clojure comes in, and why it is so powerful. With Clojure, I get a modern lisp that gives me all the benefits of functional programming that I can write new code in. That new code can integrate seamlessly into my existing Java infrastructure. And I’d say that makes Clojure a serious language ready to real work. Not a “toy”.

    I’ve got nothing against CL. But lets face it. It’s rather difficult to use CL in the real world. It doesn’t play nice with existing infrastructures built on Java or .NET. Despite having a standard, the standard is largely incomplete; and so different implementations are deeply fragmented anyway and often incompatible when it comes to things like sockets (as I mentioned above). And then there are the deployment issues. Sure I can write a a Web app in CL, but there is no standard way to deploy it. There are several CL Web servers out there, but once again, they aren’t compatible with each other as far as how you are expected to write your code. Clojure wins here as well, as Web apps written in Clojure can be deployed under any standards compliant Java Servlet container.

    Again, I have nothing against CL. but when it comes down to it, I really think Clojure is going to be the lisp that gets taken seriously–specifically because of it’s close relationship with the JVM (and now with the CLR).

  18. One more issue I forgot to mention. I think your concern about whether clojure will still work on the next release of the JVM is unfounded. At least unfounded as the concern that your CL compiler, or GCC won’t work on Intel’s next processor is. Both intel, and Sun / Oracle have made commitments to guarantee backward compatibility. So if you are concerned about clojure not working on the next version of the JVM, you also have to raise the same concerns about Intel breaking backward compatibility with their next CPU.

    With a CL compiler, the problem is even more insidious. Will your CL compiler still work on the next version of Linux? Especially if the Linux APIs have changed? With Clojure, as long as there is a compatible JVM for any given platform, Clojure will work. No matter how much the underlying platform changes.

    So again, I think when it comes to future-proofing your investment, once again, Clojure comes out on top.

    I’m not sure how you can claim, given everything I have pointed out, that Clojure is a toy. It’s clearly a serious language that is getting serious attention in the enterprise environment. And thats something that quite frankly, CL has largely failed to do.

  19. This article is over two years old, and while I still hold some points expressed here (having a standard is good; being implementation defined is bad and risky; piggybacking on somebody else’s standard and claiming to be a separate tool doesn’t seem right; etc), I admit I’ve overtrolled here a bit. Also, Clojure has matured a lot over these two years: it is (at least partially, I’m not sure) self-hosted and seems not to be as implementation-defined as it used to be.
    Michael: your points are completely valid. Clojure is a good, useful and every month more mature tool. Large part of its usefulness comes from it being defined in terms of a different tool; this is why I consider it a “toy language” (it’s an unfortunate name, but let’s stick with it): it does not define its own space, but it is merely a sugar coating over JVM the VM, Java’s runtime libraries and APIs. JVM as a platform is fine; dependence on Java’s runtime libraries, while it boosts immediate usefulness, seems to make Clojure just a nice frontend. I don’t argue whether it’s good or bad: at this moment, in terms of real-world productivity, Clojure is probably way more useful than CL. I was concerned about theoretical stuff like elegance, self-containedness, self-sustainability, etc.
    I was writing not from engineer’s point of view, but from computer science student’s point of view. From engineers point of view (especially now, two years ahead), Clojure seems a better tool to use in most cases; to a CS student, it seemed underspecified, not really defined, piggybacking on other tools and pulling its usefulness from their popularity. Immediately useful – maybe, but not nearly as elegant, independent and complete as CL. Engineer’s toy.
    Now – two years later and no longer a student for most of that time – I admit that my point of view had its flaws, and on top of that I have chosen quite inflammable and unfortunate wording. In real life and real work – right, Clojure is more useful, even if it’s not as beautiful and independent as CL.

  20. I’m aware the article is two years old. But it probably gets a lot of hits still. if you search “clojure vs. common lisp” on Google, it usually comes up in the top 5 or 6 results, so I imagine it will continue to be read frequently.

    I think characterizing Clojure as “sugar coating” over Java is really not accurate. Most of Clojure is written in Clojure, including its data structures such as hash maps, vectors, and lists. These are not Java’s hash maps, vectors, or lists. They are new ones that were written in Clojure. Also keep in mind that Clojure supports first class functions and software transactional memory–both things that you really can’t do in Java code without resorting to really deep black magic with the reflection and introspection APIs.

    As far as elegance, I think that is debatable as well. Clojure is somewhat more “functionally pure” than CL. And being a lisp-1, I think it’s possible to write more elegant code in clojure since you don’t have to use (funcall). Clojure’s support for STM also makes it possible to write some very elegant code.

    Of course, elegance is largely a matter of personal taste. And we aren’t likely to shed any new light on the lisp-1 vs. lisp-2 debate.

    But really, I think it’s unfair to call Clojure syntactic sugar for Java, given that Clojure can do a lot of things that most mere mortals wouldn’t even attempt to do in Java.

  21. Clojure is optimized for the slick one-page example. The basic idea is “Let’s create a Lisp-like language, but one which is a champion in programming language duels where the contest revolves around solving trivial tasks in half a page of code. A key requirement is beat functional languages using identical approaches.”

    Clojure is good for Lisp because most Blub programmers and their managers only understand programming language contests based on solving a task in a page of code.

    Go Clojure!

Comments are closed.