The one thing that Lisp programmers can agree on is how much better Lisp is than C and similar languages. I was talking last week to some programmers who use the Clojure version of Lisp and it made me wonder “If Lisp is so great, why did this guy have to build a slightly different version instead of building a popular application program in an existing version of Lisp, such as Common Lisp?”
What do readers think? We accept the proposition that C is feeble and yet there are only three major variants of C: C, C++, and Objective-C. Over the same period of time there have been at least the following: MacLisp, Interlisp, Lisp Machine Lisp, Common Lisp, Scheme, Emacs Lisp, AutoLisp, Clojure (perhaps readers can think of others). Yet Lisp has fewer programmers and completed programs. Thus the ratio of popular installed computer programs to versions of the language is vastly higher in C than in Lisp.
Who loves Clojure and why? And why hasn’t the C world turned into a similar Tower of Babel?
If you include Java and the JVM you get the same thing: a bunch more languages that are C like–Scala, Groovy, JRuby, or Jython. On the JavaScript side, the same thing CoffeeScript, ClojureScript, TypeScript, Dart, GWT (Java) etc.
In fact almost every other platform has more of the perceived problem than lisp.
Lazy loading and macros are cool features common to lisps any programmer should learn. I had eureka moments learning clojure that I’d of never had in Java land…
Why clojure? That’s easy. So you can have semi-decent (yes clojure is like the bastard son of lisps) that runs on the JVM. That way you can write new code in Clojure while still being able to use old code from java.
The whole “lisp is not popular” argument is completely irrelevant as any ad populum argument is. You can think the other way around too, Javascript was designed in 10 days and it’s in every browser today.
Python can’t even make lambdas work properly.
If you need more proof that Lisp is a great (family of) language(s) just look at every single language feature that you use today in your favourite language and go search where it was first introduced. And then look at how ssometimes it’s way harder to use in in your language. For example C++ that is such an abomination and any metaprogramming code is 10 times bigger than in any sane language.
“C like” means nothing and everything if it includes python, ruby, groovy, and scala. Java is hardly C-like except as a point of syntactic departure.
Common Lisp is too big and crufty – there are 2 or 3 ways to do the same thing, because the standard had to allow for legacy code that had already been written. New Lisp-like languages tend to be somewhat domain-specific – including those common pieces that everyone needs and uses, adding some custom stuff like Clogure’s interop with Java, and focusing on web related programming.
@Jonathon Graehl: You understand that the same argument holds for “Lisp like”, yes? And that really makes this entire conversation pointless, yes?
Java was in large part designed as an answer to C++, and for that and many other reasons was heavily influenced by C and C++ and is very much C-like (though it was also influenced by many other predecessors). Java is much, much closer to C or C++ than Clojure is to Common Lisp, and if those two are going to be grouped in the Lisp camp, then most Algol-based languages probably need to be grouped in the C camp.
And that then takes us back to the question of why we are having this made-up argument? To poke fun at Lisp, I guess? I can’t really see any other reason.
The people working in Lisp are more likely to be interested in the question “Why is Lisp better than C?”, and thus more likely to be interested in the question “How can I make Lisp even better?”.
The people working in C or Java are more likely to be building a system and are more likely to be interested in the question “How do I make this environment do X” and less likely to be interested in the question “How do I make this environment better?”.
Common Lisp *was* great, but it has not kept up with the times. From my perspective it’s been stagnating. Also, it was splintered with many different implementations of all the things the spec doesn’t cover. Tons of simple thinks like dealing with the filesystem just aren’t covered. There are SO many problems that computer science has addressed since the last major version of Common Lisp. So many lessons, learned. The world has changed too. There’s this “internet” thing and so many different (new) ways of working with other systems that are baked in to other modern languages, but have to be hand crafted from scratch in Common Lisp.
The Common Lisp ecosystem sucks too. The main wiki links to projects that don’t exist. Half the libraries are ancient tar files on some server with no version control history. The Quicklisp package manager is VERY new, before that there was no good package management (or none at all. I’m not sure).
Consider this: Clojure introduced a lot of concepts and abstractions that are simply not present in the Common Lisp standard (not sure about Scheme’s hundreds of incarnations). For example, treating various object types as if they are sequences, and introducing laziness in the mix. Yes, you can implement all for that in CL or Scheme, but it is not in the standard and therefore few people will ever use it.
That is the fatal flaw of CL and Scheme: they are standards-based. Most newer languages in general (e.g. Rust, Python, Scala) avoid this trap and are developed by communities that actually dogfood, guaranteeing a better language than a committee is or will ever be capable of designing (e.g. C++, but I count C in there too, as it is pretty much deficient at this point).
So to answer your question: new Lisps arise because of a mix of starting to use better abstractions on the one hand, while on the other being developed by committee/standards holds back the older Lisps significantly.
My guess is that the author of Clojure needed a Lisp for the JVM and instead of the boring task of improving an existing Common Lisp implementation, he preferred the fun task of Let’s Build It My Way.
As for users, I’m guessing that for most people who write Clojure today, Clojure is the first Lisp learned. Most users of Scheme and Common Lisp kept to Scheme and Common Lisp.
At least to me, Clojure is unattractive. Same goes for the JVM. I’ll keep using Common Lisp.
The major contribution of Clojure is not that it’s a Lisp; it’s that its model of immutability-by-default eliminates whole categories of errors due to shared mutable state. I personally also enjoy having non-parenthesized literals for things like maps and sets, but those are incidental benefits.
“If Lisp is so great, why did this guy have to build a slightly different version instead of building a popular application program in an existing version of Lisp, such as Common Lisp?”
A large portion of the business/enterprise world take Java, and hence the JVM, to be their lingua franca. If it does not run on top of the JVM then not only will ops be unwilling to support it but you will also be unable to integrate with any of the existing middileware libraries that the business has developed over the past 15 or so years.
So the question then becomes how do you get Lisp — and the benefits that come with it as a language — into such an environment? The answer is you write a dialect that is amenable to being hosted on top of the JVM. Hence Clojure. (Although, it is worth pointing out that Clojure can also target JavaScript through ClojureScript which is interesting in and of itself.)
From Clojure it is simple to write both call Java APIs and write code which can expose itself to other Java code. Correctly packaged the result can be nothing more than a single .jar file — or identical to what ops are used to deploying for your run of the mill Java application. Hence, it can be transparent from a deployment standpoint while simultaneously Integrating with whatever code happens to exist already.
Clojure as a language is also rather neat, especially when it comes to concurrency, sporting features such as software transactional memory from the get-go.
“And why hasn’t the C world turned into a similar Tower of Babel?”
To an extent it has. C, C++, Objective-C(++), C++/CLI, C++/CX, Cilk(++), C++ AMP, ISPC, CUDA, OpenCL, and that is before we get into the #pragma soup of OpenMP, OpenACC, OpenHMPP, OmpSs.
I don’t love Clojure or hate Clojure yet, just learning it after a long hiatus from Lisp.
My rationale for Clojure is that I’m torn. I really like Lisp, but I lust after Java’s libraries. We’ll see how it goes.
It depends on how you count. C-like includes Perl, Java, JavaScript, and in fact all the languages that use K&R style curly braces.
1. Why keep making new programming languages of any kind?
2. Why compare Lisp and C? What counts as a C variant?
https://en.wikipedia.org/wiki/Generational_list_of_programming_languages#Lisp_based
https://en.wikipedia.org/wiki/Generational_list_of_programming_languages#ALGOL_based
All great and mostly valid comments so far. One point I’d add: programming languages aren’t popular because of any sort of objective technical merit compared to other languages. They become and remain popular due to social inertia. Most programmers are not hardcore language nerds; they’re agnostic and choose a language to learn based on library and job availability rather than on any merits of the language itself.
New programmers check what language their favorite existing programs are written in. They check what languages have lots of tutorial and reference material available. They check what jobs are available in the subfield they want to work in, or if academically inclined they check what language is used for the type of research they want to do. Then they learn that language and use it. Their employers don’t want single points of failure, so they actively seek out additional programmers who use that language. Academics who build on existing work can do more than those who start from scratch.
Over time, these processes result in an ongoing positive feedback loop in the marketplace of ideas that amplifies and perpetuates certain languages. In other words, C is popular because C is popular, and many more people choose to learn already-popular languages.
There are certainly as many forgotten C-like variants as there are different Lisps. The difference is one of these many C-like languages had a huge breakout success (UNIX). It achieved a critical mass in the 1970s that formed the seed for its subsequent inertia in both industry and academia.
No Lisp-like language has ever yet had a comparable success story, for various reasons. Early Lisps were very slow — they were doing garbage collection in the 1960s. The Common Lisp standardization project resulted in a long series of internecine feuds and horse trading of pet features rather than a coherent centrally planned design. Lisp machines were really cool but extremely expensive. AI was supposed to be the big Lisp killer app in the 80s, but the actual product failed to live up to the hype.
Meanwhile, C-variants built on its initial breakout success in UNIX to become the de facto standard language on Windows and Mac OS, too. This resulted in a massive flood of new programmers taking up C-like languages in the 80s and 90s, and lots of corporate money to support them — just as Lisp’s “AI Winter” began. As a result, a huge body of public C libraries, many well documented, well maintained and debugged, developed for every possible function you can think of. Programmers who wanted to spend their time building high level applications rather than futz around with writing their own TCP socket opening routines (there’s no standard way to do this in Common Lisp) gravitated to the C-likes. Nor do companies want to pay someone to reinvent wheels every time; they just want to get some business process done, so they mandate the langauge that has approximately the most relevant libraries available.
Lisp’s public library space was comparatively tiny, fragmented, often poorly documented, and unmaintained — not due to any failing of Lisp itself as a language, but simply because so many fewer people used it and put time into building out the libraries. Few libraries compared to the C-likes resulted in differentially way more people and companies gravitating to the C-likes. Inertia.
Clojure is, among other things, an attempt to sidestep the library problem by being JVM-based. Java has a huge and robust collection of libraries, due in large part to its promotion as a “better C++” in the 1990s. It was very successful on the serverside in this role, resulting in a flood of companies spending money on building out its library ecosystem. Clojure was designed from the ground up for easy interoperaion with Java code so programmers can leverage all of those libraries with no hassle while still using a Lisp-like language.
Time will tell whether this strategy succeeds.
Well, to be honest D or Java are not much away from C than Clojure is from Common Lisp.
You should also consider complexity of building a C-like language; building a Lisp interpreter is probably easier (although both have actually been used as scripting languages – to each Elisp there is a Quake C). That may also be a reason for procreation. (And it may actually be a memetic disadvantage..)
Clojure has a completely different focus than other Lisps, namely simplicity. Have a look at Rich Hickey’s “Simple Made Easy”.
Paul Legato: “Programming languages aren’t popular because of any sort of objective technical merit compared to other languages. They become and remain popular due to social inertia.”
Absolutely! Flamewars are fun, but at the end of the day they don’t matter much. Hordes of people banging to their keyboards will workaround the biggest problems with a language in no time.
I have a coworker who is a big fan of Java (and long time ago was fan of C++) and he despises Javascript. I am telling him: Well, Java didn’t win because of objective merit, even though you probably started to believe that. So if Javascript wins, it won’t be due to objective merit either. 🙂
The HN thread is interesting: https://news.ycombinator.com/item?id=10375796
Sum Up:
– Situation is poorly stated, they are a lot of C-like language dynamic and non dynamic
– Lisp is general term encompassing several languages.
– It’s not as fragmented as it looks like, Common Lisp and Scheme implementations are in similar situations as C which has several flavors through several compilers gcc, clang, etc…
– Clojure vs Common Lisp is kind of in similar situations as Java/Scala and C/Rust/Go.
– Clojure fill a specific niche with a specific approach
Well, you first need to understand the history of C to fully get why it’s such a terrible language. I have link (below) with both a long video and a summary in numbered points. I think the arguments against using it *today* are solid and conclusive if you’re not forced to for some justifiable reason.
http://pastebin.com/UAQaWuWG
Nonetheless, C got a lot of use via network effects. There actually *have* been a ton of C dialects out there: compiler variants, extensions for specific systems, safe/secure versions of it, parallel/concurrent versions, hardware languages… the list goes on. Probably dozens of C variants appeared and mostly failed. The main reason is legacy: whatever replacement exists must support the existing C language usage and codebase. Any fixes that break apps or require rewrites are rejected. Marketing and community effects are another issue.
Now, we have to ask what is LISP. I think Graham said it best in that it’s an axiomatization of computation itself:
http://www.paulgraham.com/rootsoflisp.html
So, LISP is that concept and core. Its nature lets you express various constructs however you want to. You break the problem into smaller pieces while simultaneously modifying the language to better express their solution. What’s best way of expressing a problem naturally varies from programmer to programmer. The LISP variants therefore are the many ways different people (or communities) looked at solving a problem within the LISP core and philsophy. Some stay purely in it, some diverge wildly from it, and some are in the middle. The fundamental concept and approach, though, with many interpretations on expressing it are why there are so many LISP’s, though.
Clojure, a new expression of the LISP concept/core, has advantages expressed well in the first paragraphs of the rationale:
http://clojure.org/rationale?responseToken=894e644e3f4488b17953fe43bbd586c0
This philosophy and use of DSL’s for expression is no longer unique to LISP. One can see it in REBOL/Red quite directly with some takeup in languages such as Haskell and Julia. Forth also does this sort of thing with stack operations. So, we’re seeing uptake of the idea even in mainstream with them recognizing the benefits. All started with McCarthy’s reducing all computation down to some primitive, consistent elements that were shown to express or adapt to anything.
Lisp as a concept is great, it’s only natural that there would be many implementations of this concept and that you’d be able to improve on the existing ones going forward.
Also, not really sure how it’s different for the C family of languages. There are lots of dialects and Java is as much a version of C++ as Clojure is a version of CL. So, if we round up all C family languages we’ll see just as much variation there as well.
I’ve got 44 years of lisp experience, was marginally involved at the outer edge of the Common Lisp standard, and have served on the Board of the Association of Lisp Users.
Your question presumes that variants, such as Clojure, are competition.
Languages need variants because that’s how the language community expresses thoughts. That doesn’t mean the “main language” is flawed. It just means that a variant is a clear expression of the idea.
Clojure is interesting because it introduces a practical implementation of important ideas, such as immutable data structures. I spent a lot of time on it, attended a Conj, and gave a BOF talk. Hickey is nothing short of brilliant. I have learned a lot from him. I have learned a lot from his “lisp variant”, which is why language need variants.
The fatal flaws, at least for me, are that Clojure is a DSL sitting on Java and that both languages keep changing. I write programs to make the machine solve a problem. I want that problem to STAY solved. The Clojure test suite from the Conj no longer runs. So Clojure is a great vehicle for the expression of ideas but an unstable platform for long term work (insert shouting here).
Common Lisp is a standard. My code from the last century still runs unchanged. I expect that the code I write today will run unchanged 30 years from now.
There is no reason why the ideas in Clojure can’t be implemented as a DSL in Common Lisp. The only reason I haven’t tried is that I haven’t found the need. It’s the same reason I don’t care about the “library problem”. If I want to solve a vanilla problem using Java I’d certainly reach for Clojure.
Common lisp is my language of choice when the problem is really novel. Libraries don’t come to mind. Lisp is “clay for the mind”. I use it to think, to experiment, and to do things like self-modification, that other languages don’t support well. If I need an idea from another language, including a lisp variant, it is trivial to shape my lisp code to express the idea.
A language doesn’t have to look like lisp to be a “lisp variant”, in the sense that it impacts Common Lisp code. I’ve recently started writing Haskell-like signatures for my lisp code as comments. One of these days I may implement a “defsig” (defun with signature) which would be a macro to properly decorate the defun code with types, specialize “equal”, decorate fixnums, etc.
The great part about Common Lisp is that I don’t need anyone else to implement defsig as a language primitive. All I need to do is understand the idea in the variant, write some macros, and poof!
Variants are laboratory ideas, not competition. Keep them coming.
If you get this figured out let me know why we need Occitan, Eastern Catalan, Western Catalan, Central Catalan, Northern Catalan, Northwestern Catalan, Valencian and Balearic.
Great comments from everyone. Thanks. I do sometimes wonder “Why doesn’t a Lisp programmer ever walk down to the nearest car dealer and ask ‘What kind of software do you need to solve your problems?’ instead of looking in the mirror to ask ‘What new language features and development tools do I need today?'”
It seems to me that everyone here is over-thinking the matter. The real reasons are quite simple:
1) It is trivial to write an interpreter for a Lisp, and almost as easy to write a compiler, but writing one of either that preforms well is very, very hard. Conversely, writing a compiler for a language in the Algol family is a massive headache, but if you manage to do it at all, it ought to be reasonably efficient. This is especially true with C, which a horror to parse but once you are able to do it, it is almost easier to generate good machine code than bad because it is so close to the machine already.
2) Partly as a result of #1, Lisp has developed a culture of language hacking, whereas Algol language programmers think in terms of what can be done in the language without changing it.
3) Lisp’s flexibility emphasizes experimentation, including experiment with the language itself. This makes it very easy to write one-off programs, but hard to do anything that has to be maintained for very long (because discipline doesn’t come cheap). Algol family languages are very rigid, and nearly impossible to extend, but because they don’t change much over time the same code can be maintained for decades with minimal discipline or documentation.
4) Algol languages, especially C languages, have a model of computation that mirrors the common types of processor hardware almost exactly. Lisp doesn’t. By Lisp standards, improving the Algol languages would involve moving further from the machine, but because they are rigid and less easy to implement, doing so loses the facets that make them usable languages in the first place (by the standards of their users).
5) Lisp, at least to most programmers, is defined by s-expressions, so any s-expression language gets labeled as ‘Lisp’ regardless of what it is like. Algol languages, which have about the same amount of variance, seem radically different from each other if there is even a small difference in either syntax or semantics, so they tend to be seen as unrelated to each other. This means that people will talk of Scheme and Common Lisp as Lisps, but never speak of Pascal and C++ as Algols, even though Pascal and C++ bear roughly the same relationship to Algol that Scheme and CL do to MacLisp.
There are more reasons, but these seem to be the biggest factors in the apparent proliferation of Lisp dialects compared to the Algol family.
In the old days, it was about write once run anywhere & use libraries to add features. This drove down prices. Now, it’s write once run once & hire a new person for every language, driving up prices, creating the largest private equity boom in all history. It feels like every Summer brings a new crop of languages for no other purpose than keeping everyone employed & the corporate buyouts just so 1 corporation can understand someone’s new fangled Swift variant get ever bigger. If a new language ever had to have some intrinsic value or the buyout loans ever had to be repaid, it would be bad.
All computer languages are Turing complete, that doesn’t mean a program is equivalent whether it’s written in Assembly, C, Lisp, or Brainfuck. http://www.roesler-ac.de/wolfram/hello.htm
It is a problem of expressiveness in Lisp if the library doesn’t exist you make it, if a variant with a feature you want doesn’t exist you make it. Besides what has already been said by people pointing to the rationale web page, Clojure is brilliant because you get 11 DSL’s in one. http://blog.fogus.me/2010/03/23/clojures-mini-languages/
Clojure is a Lisp, but besides being a Lisp, Clojure is a way of thinking. I think of Clojure as a collection of applied ideas in software development, code as data as code…, immutable data structures, concurrency, parallelism, communication sequential programming, logic programming, simplifying vs completing.
This ideas will flow back into all programming languages that survive, and will form part of the new ones that will be created, some will be ALGOL and some will be Lisp, and some will be yet another uncharted way of programming.(Ruby is Lisp in C clothing, confirmed by Yukihiro Matsumoto).
The amazing thing about Clojure is that we have all this now. Non-trivial systems are being build everyday using Clojure, not just web applications (or as Linus Torvalds calls them “skins over databases”).
This talk by the creator of Clojure at JavaOne conference makes it clear why Clojure and whats so great and different about it.
Having great qualities isn’t the same as being great as a whole. Lisp has some great qualities, and some flaws, which empirically seem unredeemable.
There’s no consensus, among Lisp lovers, about what those serious flaws might be (nor even about their existence). Some of those who think they’ve understood Lisp’s flaws try to reinvent a dialect that circumvents them. If this dialect is homoiconic, it’s widely recognized as a Lisp, but there probably are many more languages which were partly envisioned as “fixed lisps”.
Even in this age of fast hardware, the more closely your programming language maps into machine language the faster it is going to run. On the other hand, the more abstracted you language is, the easier it is going to be for humans to work with it. This tension has existed for as long as abstracted programming languages have existed. Programmers tend to love highly abstracted languages because they make their life easier and bogging down the hardware is not their problem. Nor are programmers that interested in whether someone will be able to read and maintain their code years from now. Thus the languages that programmers love passionately (Lisp, Clojure, etc.) are not necessarily the best ones to actually implement in real life where issues like speed and maintainability also have to be taken into account.
Izzie: What about the success of SQL? It is as far away from the machine as a language can be and yet it dominates database management (which turns out to be the critical performance bottleneck in many real-world systems).
As I said before, these things exist in tension – if it was ALL about staying close to the hardware, programmers would still be writing assembly. If it was ALL about an English-like interface they would still be using COBOL. It’s true that SQL is more on the COBOL end of things, but the value of having the ability to do quick and easy database queries outweighs the need for speed (and by now the implementations of SQL are highly optimized so there is not as great a speed penalty as its abstract nature might indicate). This optimization is possible because SQL is very narrowly aimed at one set of tasks and is not meant to be a general purpose programming language. In fact it’s not even Turing complete in many versions (and/or only in a kludgy way).
you might like this guy’s blog
https://scottlocklin.wordpress.com/2012/09/12/not-all-programmers-alike/
philg,
That’s a good point.
(Because the thread is rich in tangents already, I’ll err on the side of being specific: the good point is that SQL is abstract and yet popular, while Lisp is abstract and unpopular.)
If is so great, why do we keep needing new variants?
I think it helps to generalize this question. Why do humans make new variants of things that, by some measure, are already “good enough”? Sometimes, it’s about specialization. Other times, it’s about aesthetics. From time to time, the state-of-the-art is quantitatively advanced.
Returning to your particular question, I don’t see how two Lisps could be further apart than Clojure and Common Lisp. And while Clojure is much more similar to Scheme or Racket, even among these cousins there is quite a bit of distance.
Lastly, as the underlying targets (hardware, platforms, etc) evolve, it’s natural for languages to evolve with them to take advantage of new features or designs. Clojure is a great example of this in the sense that much of its design makes it a great fit for the age of increasing cores as opposed to smaller transistors.
I’d like to add a few thoughts to Tim Daly’s comment above, which I think best expresses the particularity of Lisp as a “clay for the mind”.
Many issues (and debates) around computing can be traced back to the unique role of computing in the history of science and engineering. First, computer programs are both tools and notation for thought. Second, computing technology is still immature but also already a pillar that industrial societies rely on.
The first characteristic creates a tension between the uniformity of notation necessary for tools building by teams and the flexibility necessary to adapt a notation to an individual’s thought habits. The second characteristic creates a tension between technological progress (better tools, better notations) and application stability (solved problems stay solved).
The Lisp family as a whole leans towards flexibility rather than uniformity. Clojure in particular favors progress over stability. In both cases, people with opposite priorities aren’t happy.