What feature would you most like to see in Lisp?

Title says it all.

Your answer can be anything you would like to see relating to any (or all) the Lisp dialects.

Try to be as specific as you can in your replies.

If you're extremely lucky, someone might actually know what you're looking for.

Hello everyone,

Just so you know where I'm coming from, my company develops computer graphics tools for the special effects and computer game industries. We use C++, but it's like driving around in first gear all day...there's got to be a better way I am presently working through SICP and Practical Common Lisp and, so far, am liking Lisp very much.

What we need:

Speed - Common Lisp looks pretty good here and, from what I understand, you can use type annotations and compiler settings to squeeze out speed where you really need it.

Up to date OpenGL bindings - I think that there are some out there but am not completely sure.

Ability to generate executables/dlls - I know that this may not be the Lisp way, but it is a practical consideration for us.

Multithreading - using multiple CPUs works very well for many computer graphics algorithms and this is something we really need. From what I have seen, this seems to be hit or miss with Common Lisp implementations. Lispworks only allows a single thread to run Lisp code at a time but lets foreign functions run in parallel with the Lisp thread which might be enough of a workaround for us. It sure would be nice to have parallel Lisp execution though

Cross platform GUI toolkit (Windows/OSX) - we use Qt now, which has worked well for us.

Anyway, that's my short list. Parallel execution on multiple CPUs is what I'm most interested in.

August Swanson

Hi August,

For the OpenGL, the library at http://common-lisp.net/project/cl-opengl/ might help. Not sure how up to date it is, but it was committed to a week ago, so hopefully it is.

For executable compilation, check this recent topic on compiling to an "exe". There are a few snippets for each implementation and a couple of portable versions.
http://groups.google.com/group/comp.lan ... e89d1518e7

I'm not sure about multithreading or GUI though. If you don't find answers for those soon you could ask on comp.lang.lisp or IRC #lisp, but hopefully there'll be an archived conversation on each somewhere.

Since I dont use Scheme (or anything else) often, I can only talk about Common Lisp.

Some feature I am really missing for programming is some kind of automatical recompiling if you change some constant, i.e., if i write (defconstant bla 2) and again (defconstant bla 3) while developing, it would be nice if all functions who depend on that constant would be recompiled - it can be annoying to do this manually, just because some constant changes.

Defining Classes after having been used in Method-Declarations would be a nice thing too (as this will give you more freedom in code-arrangement).

Another thing would be an addition to have something like "java-interfaces" (abstract classes), i.e. to define a few Methods that must be defined for any sub-class, otherwise you will get an error (or at least a warning) if you try to create an instance of that sub-class - this helps programming, it doesnt add anything really useful to Common Lisp.

Real, native Continuations would be a nice thing, too.

Ali Clark wrote:
For the OpenGL, the library at http://common-lisp.net/project/cl-opengl/ might help. Not sure how up to date it is, but it was committed to a week ago, so hopefully it is.

For executable compilation, check this recent topic on compiling to an "exe". There are a few snippets for each implementation and a couple of portable versions.
http://groups.google.com/group/comp.lan ... e89d1518e7

I'm not sure about multithreading or GUI though. If you don't find answers for those soon you could ask on comp.lang.lisp or IRC #lisp, but hopefully there'll be an archived conversation on each somewhere.

Thanks much for the info. I have actually been watching the topic on creating an "exe" with much interest. Does anybody know if SBCL has multiprocessor support...Lisp running on multiple CPUs simultaneously? I downloaded the Windows install today and played around with it but that is a unithread build.

August wrote:Does anybody know if SBCL has multiprocessor support...Lisp running on multiple CPUs simultaneously? I downloaded the Windows install today and played around with it but that is a unithread build.

Looks like a qualified yes.
http://www.sbcl.org/manual/Threading.html

Paul Donnelly wrote:

August wrote:Does anybody know if SBCL has multiprocessor support...Lisp running on multiple CPUs simultaneously? I downloaded the Windows install today and played around with it but that is a unithread build.

Looks like a qualified yes.
http://www.sbcl.org/manual/Threading.html

Although its possible that the threading doesn't work on Windows yet, since Windows isn't mentioned on that page and the port is behind the Linux version (http://www.sbcl.org/platform-table.html).

For the GUI, McCLIM (http://common-lisp.net/project/mcclim/) should work on Windows and appears be maintained and used in a few apps. There was a Qt binding library, but it appears to have stopped development at v0.2 some time ago.

Apart from a well-established, high-quality, and cross-platform GUI library, which, apart from Java with its JFC/Swing framework, doesn't really exist for any language, what I'd most like to see is something like this:

#include <stdio.h>

(defun mulk ()
  (c::printf "Hello %s!" "world"))

And for ridiculous bonus credits:

#include <iostream>

(defun mulk2 ()
  (c++::<< c++-std::cout "Hello " "world" "!" c++-std::endl))

A pipe dream, I'm sure.

Certainly not a pipe-dream. You need a C header parser like cparse (http://common-lisp.net/project/cparse/), and the FFI ofcourse. From there you should be able to do something like (printf ...).

The #include ... statement can be implemented through a reader macro. The < ... > path delimiters could also be implemented. However its probably better just to use write a (c-include "stdio" :global t) function or similar.

Inside the c-include function, you would check to see if the FFI bindings for that header already exist, and if not you would search for the header file, cparse it, save the bindings somewhere, and then load them.

The second example seems a lot more difficult because it uses weird operator overloading stuff, but you could try rolling your own with fprintf I guess.

McCLIM is probably the closest thing we have to that GUI

I'd like to have standardized support for threads. Having threads is a must for many things: for web-apps, for GUI apps (actually, for almost everything except command-line apps). There is threading support in many implementations, but API and semantics differ. Having a supported threading API would be a great improvement.

dmitry_vk wrote:Having a supported threading API would be a great improvement.

Doesn't Portable Threads work? Granted, I've never used it, but it looks nice, and hey, it's even documented! What more can one wish for?

Kompottkin wrote: Doesn't Portable Threads work? Granted, I've never used it, but it looks nice, and hey, it's even documented! What more can one wish for?

It works. But it is just one of a couple of portability layers. And I don't feel too well mixing libraries that use different portability layers for the same thing. They might have some things implemented differently, slightly different semantics (e.g., do newly created threads inherit dynamic bindings?), slight incompatibilites, etc... Like, e.g. if I create a thread with clim's threading, can I use it with bordeaux-thread?
Having an agreed spec would resolve this.

Not sure which has the best support, but from the cliki threads page at http://www.cliki.net/thread

Bordeaux-threads seems to support a couple of implementations and was last updated in July.
http://common-lisp.net/project/bordeaux-threads/

dmitry_vk wrote:I'd like to have standardized support for threads. Having threads is a must for many things: for web-apps, for GUI apps (actually, for almost everything except command-line apps). There is threading support in many implementations, but API and semantics differ. Having a supported threading API would be a great improvement.

Or perhaps threads are the wrong answer :)

The Problem with Threads

That link about threads is interesting, because it's exactly the problem Clojure claims to (try to) solve. I'm not sure I've drunk the Clojure Kool-Aid yet and I don't have enough experience in concurrent programming to form a very well-informed opinion, but based on what I do know and what I've seen, it looks good.

I'm finding that most of the other things I've ever wanted from a Lisp are already in Clojure too...

• Libraries in abundance. It's hard to beat Java in that department. As Kompottkin said, a well-established, high-quality, cross-platform GUI library already exists for Java, thus it also already exists for Clojure. Clojure also has officially-supported Qt4 and OpenGL bindings. And a mature standard threading library and socket library and so on. And it's all as well-documented as you could ever hope for.
• Cross-platformness and an avoidance of the "which implementation should I pick?" problem in Common Lisp. Clojure runs the same anywhere the JVM runs. This also helps with the "how do I deploy this app?" problem. I don't have to worry about whether ADSF works right on Windows, because Java packages essentially work the same everywhere.
• A benevolent and wise dictator. It's highly arguable whether this is a good thing, but I think it is. Rich has an overall vision and it helps the language end up more unified and directed. Clojure isn't being written by a committee, and that's a really good thing in my mind. There's no widespread "implementation-dependent" or "undefined behavior".
• Modernity. "It's there for historical reasons" is such a common phrase when you talk about Common Lisp, and it's understandable, but Clojure doesn't have to care about backward compatibility. A lot of the warts and sharp corners of Common Lisp can be sawed off when you redo it all from scratch. New features can be added, crappy features can be removed, broken features can be fixed. This is a benefit of any young language but it's a nice one.
• More Ruby-ish/Perl-ish features like pretty literal arrays and hashes and regexes. The collection system in Clojure is awesome. Common Lisp has arrays and hashes but they aren't as first-class as lists. Clojure arrays/hashes/sets/lists are all equal members of society.

And so on. Much of the above is subjective, but Clojure hits most of the things on my list.

Unne wrote:...Clojure hits most of the things on my list.

I think Clojure gets a lot of things right, but I'm not totally sold. In particular, using the JVM was, IMO, the wrong way to go. I know Rich has strong reasons for doing so, and getting libraries in play was good, but I have never seen another language take off when it's tied to the runtime of yet another language. Jython and JRuby are interesting, but not the dominant versions of those languages, even with some ability to use Java libraries, etc. You can see some of the negative aspects of using the JVM when it comes to the loop syntax. Because Rich doesn't control the underlying runtime, he had to create something that's a bit twisted, IMO, to deal with the limitation of not having proper tail recursion.

It's a question of Clojure + JVM, or Common Lisp + Windows/Linux/OS X/others + a C compiler. Either way you're tied to and limited by your platform. How many Lisps have threading on Windows right now? How many even work cross-platform at all? How many libraries do? Look at ASDF for a good example of the bad side of using an operating system as your platform rather than a VM. The JVM has a very good compiler and garbage collector; how much time has been spent re-inventing those things for all the various Lisps? They're free in the JVM, and already mature and fast and stable and work on any OS.

There are tradeoffs though, yeah. Clojure is tied to a product that's largely under the control of a mega-corp. (Java is open source nowadays, but I would imagine forking Java just to run Clojure on it might be a bit impractical.) But the admittedly nasty loop/recur is a small price to pay for all the benefits of a nice VM, in my opinion. It's not much worse than throw/catch or labelled-block returns in many languages, and in Clojure you could hide loop/recur with macros if you wanted. The JVM may be getting tail-call optimization, in which case Clojure will get it too. Sun seems to be moving to make the JVM better for dynamic languages, so maybe your concerns are more valid in theory theory than in practice. For now.

Jython and JRuby are bolted on to the JVM as an afterthought, and yeah, they'll never be as popular as the canonical C versions of themselves. They have to go through contortions to get the JVM to work like the C versions. Clojure is designed for the JVM, so it's not competing with itself, and it can work more smoothly and integrate more fully with Java than Jython/JRuby ever could.

Unne wrote:It's a question of Clojure + JVM, or Common Lisp + Windows/Linux/OS X/others + a C compiler. Either way you're tied to and limited by your platform. How many Lisps have threading on Windows right now? How many even work cross-platform at all? How many libraries do? Look at ASDF for a good example of the bad side of using an operating system as your platform rather than a VM. The JVM has a very good compiler and garbage collector; how much time has been spent re-inventing those things for all the various Lisps? They're free in the JVM, and already mature and fast and stable and work on any OS.

Whether Lisps have threading on Windows right now is purely a function of their implementations and whether that's important to them. Remember that many of the Lisps out there are based on very old implementations that have literally been in existence for decades (CMUCL -> SBCL, KCL -> ECL, MCL -> ClojureCL, etc.). In many cases, the primary implementations were built for Unix which have had very scatter-shot threading support. Put all that together and you have a mess that's going to take a while to work out. Threading support is starting to happen in standard Lisps, but it's admittedly a mess. Nobody screams about this fact more than I do.

A better comparison might be between other dynamic languages like Perl, Python, and Ruby, which all run on Windows with threading today (typically with green threads today, but threads all the same). Given these other languages, I'd just propose that any lack of features on Windows for Lisp is a self-inflicted problem, nothing inherent to not using the JVM as a foundation.

Actually, however, Armed-Bear Common Lisp (ABCL) does use the JVM.

There are tradeoffs though, yeah. Clojure is tied to a product that's largely under the control of a mega-corp. (Java is open source nowadays, but I would imagine forking Java just to run Clojure on it might be a bit impractical.) But the admittedly nasty loop/recur is a small price to pay for all the benefits of a nice VM, in my opinion. It's not much worse than throw/catch or labelled-block returns in many languages, and in Clojure you could hide loop/recur with macros if you wanted. The JVM may be getting tail-call optimization, in which case Clojure will get it too. Sun seems to be moving to make the JVM better for dynamic languages, so maybe your concerns are more valid in theory theory than in practice. For now.

Actually, I think you have that backwards. The concerns are valid in practice now. There is no debate that Clojure had to compromise functionality and syntax in order to fit within the limitations of the JVM. If Sun fixes the JVM (and I'm sure they will, but I'm making the point), then Clojure has the opportunity to fix some of that. Over time, I have no doubt that Clojure will get better, but it will always be subject to the limitations that Sun puts on the system. Whether that's a problem in the long term remains to be seen. Again, I have nothing fundamentally against Clojure or the tradeoffs made in creating it. Indeed, I think it's a splendid piece of work and there is much that Lisps in general can learn from its new and interesting features. The JVM thing just gnaws at me.

Jython and JRuby are bolted on to the JVM as an afterthought, and yeah, they'll never be as popular as the canonical C versions of themselves. They have to go through contortions to get the JVM to work like the C versions. Clojure is designed for the JVM, so it's not competing with itself, and it can work more smoothly and integrate more fully with Java than Jython/JRuby ever could.

So, two things... First, it isn't competing with itself, but it is competing with other Lisps. It will have to distinguish itself to gain long term momentum. Second, it can "work more smoothly and integrate more fully with Java" because it accepts the limitations of the JVM as a given and re-labels them "features." In other words, it's a two-way street on JVM compatibility.

Boiling this down some... Clojure is a (Faustian?) bargain: if you're willing to accept some compromise in functionality to deal with limitations of the JVM, you'll get a language with Lisp syntax that runs the same everywhere the JVM runs and can take advantage of Java libraries. That's it. Whether that excites you or not depends on whether you're cool with that compromise.

All that said, if it was me working in a Java shop and I wanted to slip in a little Lisp, I'd probably reach for Clojure or ABCL to get the job done. That would be an externally-imposed limitation, however. I don't think I'd set that limitation for myself if I had the option.

findinglisp wrote:Whether Lisps have threading on Windows right now is purely a function of their implementations and whether that's important to them. Remember that many of the Lisps out there are based on very old implementations that have literally been in existence for decades (CMUCL -> SBCL, KCL -> ECL, MCL -> ClojureCL, etc.). In many cases, the primary implementations were built for Unix which have had very scatter-shot threading support. Put all that together and you have a mess that's going to take a while to work out. Threading support is starting to happen in standard Lisps, but it's admittedly a mess. Nobody screams about this fact more than I do.

But that's what I mean by theoretical concerns vs. practical concerns. Any implementation COULD have all the things I mentioned on my list, in theory, but how many do right now? I don't have a while to wait for things to sort out, so what are my options if I want to quickly and easily write a cross-platform multithreaded GUI app this afternoon? Not 2 or 10 years from now? That's why Clojure is exciting to me. Lack of those features is a self-inflicted problem in CL that could be fixed in theory, but it's still a problem, and Clojure solved it. Maybe it's not ideal, but at least it's a solution that exists right now.

Actually, I think you have that backwards. The concerns are valid in practice now. There is no debate that Clojure had to compromise functionality and syntax in order to fit within the limitations of the JVM.

What are some other specific examples of functionality and syntax compromises besides loop/recur that are due to limitations of the JVM? Maybe there are some I'm not aware of. I don't know the JVM itself that well. I know that most of the many horrid limitations of Java don't exist in Clojure at all.

So, two things... First, it isn't competing with itself, but it is competing with other Lisps. It will have to distinguish itself to gain long term momentum.

I think Clojure has potential to attract people from Java as much as attract people from other Lisps. The set of Java developers is a lot larger than the set of Lisp developers, so maybe Clojure has a chance even if it's rejected by Lispers. People using Java right now are the ones really in need of dire help anyways. It also has potential to attract people like me who desperately want to use Lisp but who find Common Lisp to be showing its age and lacking in many areas.

Unne wrote:It also has potential to attract people like me who desperately want to use Lisp but who find Common Lisp to be showing its age and lacking in many areas.

Are the problems in CL really too much to solve retrospectively? Surely the language itself wouldn't prevent CL from adopting things from more modern languages? By that I mean, if CLOS could have been added inside the language, then why couldn't the same be said for modernisation?

Why doesn't someone do it then? It's not that these problems CAN'T be be solved in Common Lisp, it's that they haven't been and aren't being. I have to write programs today, I can't wait a decade while these things are fixed, assuming they ever will be.

A fair response is "Hey buddy, why don't YOU?" I don't have the time or the ability or the desire (from an "it'd be fun" perspective) to fix many of my perceived problems with CL, even though the language is powerful and flexible enough to let me do it, surely. One other possible motivation for me to do it would be necessity, but given Clojure (or even Ruby/Python which get you most of the way there) I don't even have necessity, because they already give me most or all of what I need, for which I am eternally grateful. Another motivation would be some kind of aesthetic or nostalgic devotion to Common Lisp that makes me solve the problem in spite of necessity or desire, but I don't have religious feelings about any language, and even though many people do, it seems not to motivate enough people to get us all the way there.

Lack of motivation and resources and effort in the community is the only problem, and it's a surmountable problem, but it's still a real show-stopper problem in the meantime.

This thread was started in part to try to give an avenue to let off steam for exactly that. Its an informal note on the internet for the things that a modern lisp needs but doesn't currently have.

People should draw their own conclusions from the thread, but what I gathered was that there are more than one people out there who want:
a) GUI.
b) Threads.

On an optimistic note, CL appears to have outlets for both efforts. From what I've seen, McCLIM appears to be a fairly portable and supported GUI library. Threads unfortunately don't appear to have much appearance in open source implementations. But if SBCL threads can work on Windows as well as Linux, at least that could provide one free and cross-platform implementation to use threads on, which is a start.

Any replies with pointers to a standardised GUI library are welcome.
Any replies with pointers to a standardised threading library are welcome.
General replies on either of those are welcome.

And of course, if you can see other modern features that are missing from lisp, however small and however large, please post them here!
I (probably) won't write a library for any myself, but its good to make a note of it anyway.

Unne wrote: But that's what I mean by theoretical concerns vs. practical concerns. Any implementation COULD have all the things I mentioned on my list, in theory, but how many do right now? I don't have a while to wait for things to sort out, so what are my options if I want to quickly and easily write a cross-platform multithreaded GUI app this afternoon? Not 2 or 10 years from now? That's why Clojure is exciting to me. Lack of those features is a self-inflicted problem in CL that could be fixed in theory, but it's still a problem, and Clojure solved it. Maybe it's not ideal, but at least it's a solution that exists right now

...

What are some other specific examples of functionality and syntax compromises besides loop/recur that are due to limitations of the JVM? Maybe there are some I'm not aware of. I don't know the JVM itself that well. I know that most of the many horrid limitations of Java don't exist in Clojure at all.

....

I think Clojure has potential to attract people from Java as much as attract people from other Lisps. The set of Java developers is a lot larger than the set of Lisp developers, so maybe Clojure has a chance even if it's rejected by Lispers. People using Java right now are the ones really in need of dire help anyways. It also has potential to attract people like me who desperately want to use Lisp but who find Common Lisp to be showing its age and lacking in many areas.

Okay, fair enough. I'll grant you that if you want to write a cross-platform GUI in a Lisp-like language today, your options are pretty limited. Certainly, there are implementations such as Lispworks that will allow you to do it in CL, but not the open source implementations today. Again, that's the taking advantage of the libraries thing and making the trade-offs thing.

The only thing I know about today in terms of a "wart" is the loop/recur thing. I honestly haven't done anything with Clojure other than read all the docs on the web site and all the various presentations that Rich has done that have been posted to the web. My real experience is admittedly limited. There may be performance issues that surface because of things about the JVM that can't be implemented as efficiently as possible (the way values are represented, boxed vs. unboxed, etc.). I don't know of anything, specifically, but every other Lisp I have seen implemented on the JVM seems to grumble about those things. Whether the performance is too poor for a given application is really the question. I'm generally not hung up on performance, per se, as I'm a big proponent that good enough is good enough and programmer time is much more valuable than machine cycles. For instance, I'm sure that Clojure is fine for GUIs and web applications which typically spend most of their time executing NOP instructions waiting for users to send them events. Whether Clojure would work for scientific computing, I have no idea.

Finally, whether Clojure survives or not is a very complex question to answer. I won't prognosticate one way or another ("It's way above my paygrade." ). As I said, I think either way Clojure is very interesting and Rich has done a great job. There are a lot of really cool ideas to learn from at a minimum. And I'm a big proponent of seeing where we can take Lisp in a post-standard-CL world. The main hang-up for me on Clojure is the insistence on the JVM. Certainly, it's a benefit, but it also brings limitations.

Ali Clark wrote:Certainly not a pipe-dream. You need a C header parser like cparse (http://common-lisp.net/project/cparse/), and the FFI ofcourse. From there you should be able to do something like (printf ...).

Another approach might to load a DSO and look for C++ functions; with C++ name mangling, you could probably get enough information to call them safely from Lisp. Name mangling is compiler-specific, of course, but it has the advantage of being a lot more restricted than C headers—no macros to worry about.

Sorry for the late answer. I discovered Lisp forums quite late this month

Unne wrote:It's a question of Clojure + JVM, or Common Lisp + Windows/Linux/OS X/others + a C compiler. Either way you're tied to and limited by your platform. How many Lisps have threading on Windows right now? How many even work cross-platform at all? How many libraries do? Look at ASDF for a good example of the bad side of using an operating system as your platform rather than a VM. The JVM has a very good compiler and garbage collector; how much time has been spent re-inventing those things for all the various Lisps? They're free in the JVM, and already mature and fast and stable and work on any OS.

C is not a limitation by any means. If anything, C is as low level as it gets without doing assembler and JVM's are written in C, btw. Then the other questions do affect me personally, so let me answer some of them.

Lisps do have a history and do have priorities and these do influence what features you find on them. Not many lisp programmers use threads or care about them, but that does not mean they do not exist. SBCL was one of the first free implementations to provide native threads, Clozure has them, and ECL as well.

Regarding portability, some lisps do have it easier than others. Those that implement their own native compilers have it more difficult, as they sometimes have assumptions about the OS and chips. ECL on the other hand is pretty much as portable as it gets, being completely built on top of C or C++. The only dependencies are the garbage collector, which is the Boehm-Demer-Weiser library, and the GMP bignum library, which can be compiled in pure C mode and is thus absolutely portable.

You talk about lisp implementations reinventing the wheel because we provide garbage collection? That is funny, because lisp implementations have had garbage collection way before the JVM ever did. SBCL and CMUCL have an impressive garbage collector, fast and accurate. ECL uses a well known and widely ported, the BDW, an example that a garbage collector can be reused. Others have different alternatives. All of them are much older than Sun's.

Java is definitely not the panacea. In my free time I do some stocks and derivatives trading and there you have two alternatives: Windows and Java. I must say the second one is not that nice in any platform other than Windows. Even some platforms do not work in some operating systems at all, such as OS X, because of the JVM they ship and the integration with the operating system that those programs have. Talk about portability.

Maybe not many Lisp programmers care about threads, but I do, and so does anyone who writes a GUI or web app. Which is a pretty huge slice of programming nowadays, in my world anyways. When we have 32- or 64-core CPUs in a few years, are Lispers still not going to care? Who's doing the work to keep all of the Lisps up to date with modern hardware and practices? Few have done it up to this point, and it's just going to get worse over time.

The JVM is written in C, so people building new languages on top of the JVM don't have to mess with C. That's the whole point. In SBCL you have this sad state of affairs: http://sbcl.sourceforge.net/platform-table.html Versions for some platforms lagging behind others, versions for some platforms non-existent because no one ported them. That is the limitation of using C and targeting an OS rather than a VM.

On the other hand for Clojure, most or all of that chart would be green, and all versions would be up to date. I don't have to wait and hope for someone to port Clojure to my OS. The hard work has been done in the VM and no one has to do it again.

I know that Lisp had garbage collection for a long time. My point was more about Lisps competing with each other. Does the SBCL GC do something substantially different than the ECL one or the Lispworks one or the CLISP one or the Allegro one? If not, why are they all still around, being maintained in parallel? Perhaps some give slightly different performance than others, but that doesn't matter nearly as much nowadays as it did in the 1980's when all of these Lisps were put together. Maybe this was a necessary state of affairs two decades ago. Today it's a waste of time. All you need is one good GC that everyone can use everywhere. This is debatable of course.

There are good and bad things about VMs. But the good things are relevant to me and to apparently most programmers in the world, given the immense popularity of .NET and Java.

(Please note, I'm not a Java fanatic. I use Ruby in my day to day life. But I can recognize the benefits of the JVM.)

Unne wrote:The JVM is written in C, so people building new languages on top of the JVM don't have to mess with C. That's the whole point. In SBCL you have this sad state of affairs: http://sbcl.sourceforge.net/platform-table.html Versions for some platforms lagging behind others, versions for some platforms non-existent because no one ported them. That is the limitation of using C and targeting an OS rather than a VM.

On the other hand for Clojure, most or all of that chart would be green, and all versions would be up to date. I don't have to wait and hope for someone to port Clojure to my OS. The hard work has been done in the VM and no one has to do it again.

I know that Lisp had garbage collection for a long time. My point was more about Lisps competing with each other. Does the SBCL GC do something substantially different than the ECL one or the Lispworks one or the CLISP one or the Allegro one? If not, why are they all still around, being maintained in parallel? Perhaps some give slightly different performance than others, but that doesn't matter nearly as much nowadays as it did in the 1980's when all of these Lisps were put together. Maybe this was a necessary state of affairs two decades ago. Today it's a waste of time. All you need is one good GC that everyone can use everywhere. This is debatable of course.

I understand the point you're trying to make and to some extent I agree with you, but the facts actually undermine you here.

Specifically, Java has had just as rocky of a time getting JVM support everywhere. For years, the top Java enhancement request was "Port Java to Linux." Literally, that was it. It had thousands of votes from developers in Sun's bug database. Yes, that has gotten better since that time (Java obviously runs on Linux, even Sun's implementation these days, in addition to other open source versions now from Red Hat), but there are still many environments where Java won't run. And Clojure won't run there either until somebody ports Java. The people who port Java are not Clojure people, so you have this problem where you have to wait until an unrelated community decides that they want Java before you can get Clojure. Obviously, if they get excited about doing a Java port, there is benefit there because Clojure doesn't have to do the port, but you're stuck waiting until it happens.

Second, many of the Java implementations running on those different platforms use different technology. IBM's implementation of Java is different than Sun's. Specifically, the GCs are often a big source of difference in the implementation since you might want to trade off performance and memory usage for various operating environments (server vs. desktop vs. embedded, for instance). Thus, if you want to attack multiple CL implementations for reinventing the wheel (and there is some reason to call that out), don't use the GC as an example of that, particularly vs. Java.

That said, I won't argue with you further about Clojure. As I have always maintained, there is a lot to like in Clojure; my own opinion is that the use of the JVM isn't one of those things, however. I like the fact that Clojure exists. It's good for Lisp. It gets a lot of things right.

Unne wrote:Maybe not many Lisp programmers care about threads, but I do, and so does anyone who writes a GUI or web app. Which is a pretty huge slice of programming nowadays, in my world anyways. When we have 32- or 64-core CPUs in a few years, are Lispers still not going to care? Who's doing the work to keep all of the Lisps up to date with modern hardware and practices? Few have done it up to this point, and it's just going to get worse over time.

I am afraid the problem is in pessimistic views like this.

Unne wrote:The JVM is written in C, so people building new languages on top of the JVM don't have to mess with C. That's the whole point. In SBCL you have this sad state of affairs: http://sbcl.sourceforge.net/platform-table.html Versions for some platforms lagging behind others, versions for some platforms non-existent because no one ported them. That is the limitation of using C and targeting an OS rather than a VM.

No, you are completely wrong about this. The fact that SBCL is lagging on some platforms is because you have to write a whole lot of things in assembler and C that the Sun people are giving to you for free. But that does not mean what you get from Sun is the best of the worlds. Or from IBM. Or from the GCJ people. You cannot blame an implementation for lack of developers and blame that on the fact of using C and porting things. Who writes your VM anyway?

Unne wrote:On the other hand for Clojure, most or all of that chart would be green, and all versions would be up to date. I don't have to wait and hope for someone to port Clojure to my OS. The hard work has been done in the VM and no one has to do it again.

Note that I am not arguing about Clojure here. I do not know about this dialect, and right now I do not feel the need to learn it myself, but I just want to make it clear that you are spreading some misinformation about CL itself.

Unne wrote:I know that Lisp had garbage collection for a long time. My point was more about Lisps competing with each other. Does the SBCL GC do something substantially different than the ECL one or the Lispworks one or the CLISP one or the Allegro one? If not, why are they all still around, being maintained in parallel? Perhaps some give slightly different performance than others, but that doesn't matter nearly as much nowadays as it did in the 1980's when all of these Lisps were put together. Maybe this was a necessary state of affairs two decades ago. Today it's a waste of time. All you need is one good GC that everyone can use everywhere. This is debatable of course.

Garbage collection is not a 100% solved problem, even though people may argue the opposite. Different implementations have different approaches. Have you heard about the difference between conservative vs. exact garbage collectors? Or between those that do not move and those that compactify? There are many choices. ECL cannot use a compactifying garbage collector without breaking the programs that embed it. SBCL does use a more sophisticated approach because it is in control of the memory.

Unne wrote:There are good and bad things about VMs. But the good things are relevant to me and to apparently most programmers in the world, given the immense popularity of .NET and Java. (Please note, I'm not a Java fanatic. I use Ruby in my day to day life. But I can recognize the benefits of the JVM.)

I only entered this discussion to make your statements about CL more precise. I do think VM are nice and the fact that many mobile phones out there live out of java is a proof of that. I also do recognize the benefits of a VM, but this is nothing new and we still have to learn on that field. I myself program a lot in C/C++ for "a living" (if science can be considered so), and I do it with massive parallelization, and not only multithreading. Currently, VM will not help in this respect. And besides this, remember that just like somebody has to write your CL implementation, so does someone have to write the VM. The fact that one or another have more supporters does not make it neither the only alternative nor the best one.

jjgarcia wrote:No, you are completely wrong about this. The fact that SBCL is lagging on some platforms is because you have to write a whole lot of things in assembler and C that the Sun people are giving to you for free.

That is entirely my point. On one side, you need developers to port something. On the other you don't, because the work is already done and you can just use it for free. How can you say that requiring more developer time and effort is not a limitation?

People seem to be far too quick to dismiss manpower as a cost. In reality it's a huge cost, and if it's already been paid and you can take advantage, you are way ahead. This isn't pessimism, this is reality. If only SBCL had tons of developers with lots of time, it would exist on all platforms with libraries to rival any other language. If only I had a few million dollars, my house would be a mansion.

Garbage collection is not a 100% solved problem, even though people may argue the opposite. Different implementations have different approaches. Have you heard about the difference between conservative vs. exact garbage collectors? Or between those that do not move and those that compactify? There are many choices. ECL cannot use a compactifying garbage collector without breaking the programs that embed it. SBCL does use a more sophisticated approach because it is in control of the memory.

Yeah, it's an interesting subject, but it's largely academic. I know that it isn't a solved problem and there are different approaches, but any approach that works is good enough for me. Fast hardware and lots of memory means I don't have to care. I get by with Ruby after all, which is orders of magnitude slower than everything. ECL seems to be a sort of special case, given that it tries to interoperate with C. Rich has stated that the JVM's GC is pretty impressive nowadays anyways. He discusses it a bit here (video), I think perhaps at the end of part 2.

Unne wrote:That is entirely my point. On one side, you need developers to port something. On the other you don't, because the work is already done and you can just use it for free. How can you say that requiring more developer time and effort is not a limitation?

No, I think you do not get the point. Your argument is to jump on the bandwagon and leave everything aside when there is a project with a greater manpower. I strongly disagree with this. Diversity is the source of creativity. I furtherargue that there is room for everything and I will leave it here. I am happy that you are pretty excited about Clojure and all that, but please understand it as well that JVM is not everything and other people have other needs.

Unne wrote:People seem to be far too quick to dismiss manpower as a cost. In reality it's a huge cost, and if it's already been paid and you can take advantage, you are way ahead. This isn't pessimism, this is reality. If only SBCL had tons of developers with lots of time, it would exist on all platforms with libraries to rival any other language. If only I had a few million dollars, my house would be a mansion.

Yes, but what is your point? That SBCL developers should stop doing what they are doing because there is the Java Virtual Machine? To promote other dialects? I sincerely hope you do not see the arrogance on that. You are asking them, and probably me, to stop supporting Common Lisp because you found something that makes you happy. I have seen this excitement too many times (C, C++, Sheme, Java, Haskell, Javascript...)

Unne wrote:

Garbage collection is not a 100% solved problem, even though people may argue the opposite.

Yeah, it's an interesting subject, but it's largely academic.

No, it is not. It is academic for you that get a very high level implementation, do not have serious needs other than threading and GUI, and do not worry about low level details. But perhaps somebody who works on a tight memory environment, or needs to port things to other OSs, or needs many languages to interoperate has to worry about this. Please understand that there are two levels of programmers: those that only work with high level libraries which are luckily supported everywhere, and those of us that have to deal with more complicated things. If I read all your posts, you are telling us we should be doing something else. But if all systems programmers disappear who will write your VMs? Who will research on the next big language? Do you realling think that a single dialect will settle everything? Or a VM for that matter?

I just want to warn you from your discourse of "do not waste effort on other things", "let's go with the right thing" because it can be harmful. I myself have experienced it on my own pet project, ECL, with GCL people asking to drop all effort on it and more recently Richard Fateman trying to convice Maxima developers to stop working on any port of Maxima to ECL. This is a discourse that can too easily become hate and in my own view it is not far away from that of people disregarding other lifestyles or beliefs.

Please can we stay on topic?

There are many implementations of Lisp in C, and there are implementations of Lisp in JVM and .NET . We can all have our cake and eat it.

The discussion on this raises interesting facts, but I'm not sure it is going anywhere.

I would like to see a common lisp based on llvm, with llvm as back-end. (more)

gmlk wrote:I would like to see a common lisp based on llvm, with llvm as back-end. (more)

ECL is aiming at that right now (http://tream.dreamhosters.com). The goal is to replace the current bytecodes compiler and the lisp to C compiler by a common frontend that optionally produces either the same bytecodes, or C, or uses llvm as a backend. The only thing is that currently I am busy fixing some real problems with the lisp runtime, most of them related to the limitations of C interrupts and to thread safety. Sorry for trolling again

Ali Clark wrote:Please can we stay on topic?
There are many implementations of Lisp in C, and there are implementations of Lisp in JVM and .NET . We can all have our cake and eat it.
The discussion on this raises interesting facts, but I'm not sure it is going anywhere.

Sorry for trolling. I get too emotional sometimes In case I am permitted, my 2cts
- Real in-browser support for CL, with access to the DOM and so on.
- Serializable and persistent CLOS classes done right and transparently, with little overhead.
- Sealed classes, where types are unboxed and slot access can be compiled efficiently.
- A better definition of the multithreading package, with threads, locks, etc. Desperately needed by implementors
- Network transparent pathnames that really allow you to access remote machines. ECL has the syntax, but we lack the real code.
- Sandboxed CL processes, where execution is restricted to certain sections of the file system, limited resources, etc.
- Not only multithreading, but also transparent execution on clusters, with process migration, etc.

I may have missed many of the libraries out there for these tasks, but I tend to be quite out of date, anyway O:-)

jjgarcia wrote:Sorry for trolling. I get too emotional sometimes

Juanjo, I'm happy you found LispForum. I'm using ECL right now for a project because of its small size. I'm happy that it exists and that you're SO responsive to all things that surround it. (Honestly, folks, if you haven't used ECL yet and seen how quickly Juanjo knocks off the bugs that are reported with it, you would be amazed.)

In case I am permitted, my 2cts
- Real in-browser support for CL, with access to the DOM and so on.
- Serializable and persistent CLOS classes done right and transparently, with little overhead.
- Sealed classes, where types are unboxed and slot access can be compiled efficiently.
- A better definition of the multithreading package, with threads, locks, etc. Desperately needed by implementors
- Network transparent pathnames that really allow you to access remote machines. ECL has the syntax, but we lack the real code.
- Sandboxed CL processes, where execution is restricted to certain sections of the file system, limited resources, etc.
- Not only multithreading, but also transparent execution on clusters, with process migration, etc.

I may have missed many of the libraries out there for these tasks, but I tend to be quite out of date, anyway O:-)

Any thought to creating a "real" mod_lisp that would actually embed CL into Apache, similar to mod_perl, mod_python, mod_ruby, etc? I have thought for a while that ECL is perfect for that.

jjgarcia wrote:

gmlk wrote:I would like to see a common lisp based on llvm, with llvm as back-end. (more)

ECL is aiming at that right now (http://tream.dreamhosters.com). The goal is to replace the current bytecodes compiler and the lisp to C compiler by a common frontend that optionally produces either the same bytecodes, or C, or uses llvm as a backend. The only thing is that currently I am busy fixing some real problems with the lisp runtime, most of them related to the limitations of C interrupts and to thread safety. Sorry for trolling again

Get ECL to work with CLX and StumpWM people will love you forever.

TheGZeus wrote:Get ECL to work with CLX and StumpWM people will love you forever.

Oh, that would not be difficult. Portable CLX used to build in ECL. The only problem is that I lost my account in the project and got disconnected from it. There have been reports that the current library loads with some patches, but there are some compiler problems. I guess I should update the source tree with the most recent darcs tree.

findinglisp wrote:Juanjo, I'm happy you found LispForum. I'm using ECL right now for a project because of its small size. I'm happy that it exists and that you're SO responsive to all things that surround it. (Honestly, folks, if you haven't used ECL yet and seen how quickly Juanjo knocks off the bugs that are reported with it, you would be amazed.)

I am blushing right now You would not be amazed about the speed if you really knew how simple those bugs most of the time are, hehe.

findinglisp wrote:Any thought to creating a "real" mod_lisp that would actually embed CL into Apache, similar to mod_perl, mod_python, mod_ruby, etc? I have thought for a while that ECL is perfect for that.

Wow, I had a wonderful dinner discussion with some guys at the European Common Lisp meeting precisely about this. I must admit I did not work on that because of ignorance of the Apache API, but I think it would be really easy. Let's assume that Apache provides the module with a file descriptor to read from, another one to write to and the name of the file that contains the code, then I could write you three black-box functions: one for setting up ECL, another one for shutting it down, and another one for doing the work. It would probably look as follows (not checked), but perhaps with some additional lines for handling signals, and so on. Again, what scares me is all the Apache stuff

void mod_ecl_open() { cl_boot(0, 0); }

void mdo_ecl_close() { cl_shutdown(); }

int mod_ecl_run(int in, int out, const char *file)
{
   cl_object in_stream = ecl_make_stream_from_fd(Cnil, in, smm_input);
   cl_object out_stream = ecl_make_stream_from_fd(Cnil, out, smm_output);
   cl_object pathname = cl_parse_namestring(1, make_constant_base_string(file));

   cl_object code = c_string_to_object("(lambda (in out file) (let ((*standard-input* in) (*standard-output* out)) (load file :verbose nil :print nil)))");
   cl_object ok, fun =  si_safe_eval(3, code, Cnil, OBJNULL);
   if (fun == OBJNULL) return 0;
   ok = cl_funcall(4, fun, in_stream, out_stream, pathname);
   return ok == OBJNULL;
}

Disclaimer: I'm currently studying various lisps, but I'm a beginner. And, as any other lisp newbie, I'm dreaming of the perfect lisp dialect
This is how I'd like it to be:

• Lisp 1 (like scheme, funcall & sharpquote are too verbose for functional programming)
• Functional (like clojure and erlang, i prefer functional stile to OO: OO feels like imperative++)
• No mutable state (again like clojure and erlang, also PLT scheme)
• Purely functional data structure are first class, no list centric, sequence types are abstact and lazy (like clojure)
• Vast functional library, with maps, filters, zippers, folds etc (like haskell)
• Concurrency via green threading and message passing (a la erlang, highly scalable, no mutable state so there's no need of locks)
• Files are modules, implict namespacing (like python and otter)
• Facilities for logic programming (oz & prolog), functional reactive programming.
• Standard libraries for network, db access, gui creation (clim), opengl & vector+raster graphics & sound (processing).
• Support for continuations (scheme)
• Both interpreted and compiled (like Common Lisp)
• Should be easy to create standalone executables, maybe not image based but with a default mechanism for persistance
Syntax details
• Small function names for most used functions, still readable (like arc and otter, NOT perl)
• def for definition, fn is lambda, (+ _ 4) is (lambda (x) (+ x 4))
• Hashtables are functions of their keys (like Clojure, arc), maybe list and arrays are functions of their index (like arc)
• Facilities for both Hygienic and Non-hygienic macros
• Reader macros should have a limited scope (maybe only the current module)

Can't think of anything else for now.

A (setf if) macro would be nice, so you can say

(let (goods bads)
  (dolist (x xs)
    (push x (if (goodp x) goods bads))))

MrSelfDefstruct wrote:def for definition, fn is lambda, (+ _ 4) is (lambda (x) (+ x 4))

This one isn't lispy, the first symbol in a list must determine the function/macro, unless it is interpreted by a macro, and then what is lispy gets blurrier. I think better versions of what you mean are discussed here.

As for lisp-1 i personally prefer different namespaces for functions and variables. Maybe you should blame quantummechanics :p
<x|X = x<x|, Integral(x over space, |x><x|)=1 (x is variable x, |x>,<x| is state and X is position operator. Jokes aside, it seems really arbitrary which too choose. Anyone know something to make it less arbitrary?

As for features i desire, i have a project; lang-lisp(I still have to improve the explanation on that website); with which i plan to create them. Here some of them are:

Better types, ones that allow for having different types, like C++'s templates. Functions that are overloaded using that. Types that are 'functional' written like this: (integer) is an integer, and a pair is written like (pair type-1 type-2), a pair of a float and an integer:
(pair (float) (int)), basically, lose symbols not at the start of list are variables of the type, and can (usually) be filled with anything, that is why just 'integer', is an arbitrary type, but '(integer)' is the particular integer.

Some would be special, like (any) being an unspecified type, and (eql to-thing) for something eql to 'to-thing' (eql* to-thing), same but it is tested for whenever some value is not (eql to-thing) but a subset. (eql (number x)) being something equal to some unknown x. (or ...) being one of a set of types.

As an extension of better types, macros could use types as well, types choosing a macro just as a function would be choosen. Also, the types that it actually is can be passed on via the variables in the types. (defmacro name ((pair q r) (array s (eql (integer n)))) (a b &rest rest) ..something dependend on q,r,s,n, and a, b, rest.

References is another feature i miss. Don't see any reason not to let how functional the code is up to the programmer. In my project references will be 'special types', and would work in a few different ways; (let ((c 2)) (meh (ref c)). Then c is (integer), as 2 is, and (ref c) is (ref-var (integer)). (let ((c (ref 2)) (meh c)) should work too, ref should then create the space for the 2, and then make c a pointer to it, making c itself (ref-var (integer)) ref-var is for when it is a variable, and doesn't always need pointer-indirection. (ref ..) is just a pointer to something, but that can be treated as the value behind the pointer.

There is however another type of reference, that of a setf-function, well, i don't see the point of setf and setq- so i only have set, if you have both a (defun fun-name ( ..args..) ..) (defun (set fun-name) (to ..args..) ..), then you can: (let ((a (ref fun-name))) .. code where you can set and use 'a.)

Some other notation of nested lists, as i suggested here, what. The lang-lisp project has an implementation. I don't dig clojures vector things in {} at the moment, it seems a little arbitrary, and how do you decide when to do this.(I'd implement it as {&rest body} -> `({} ,@body), and let macros do the rest.) I guess both might cause some problems that people might start whine about conventions regarding when to use them.

MrSelfDefstruct wrote:Files are modules, implict namespacing (like python and otter)

Maybe it is better to do it optionally. Like (load :as-module "file.lisp"), and (load :as-prog "file.lisp").
The latter is another idea i have; being able to use programs as objects with a namespace, a structure for all the variables that are global, all the functions have as extra first argument this structure, and a main, that runs the program returns the final state.

Some other things is a better specification, relegating a lot of things that are in there to libraries. Better websites with documentation for basic libraries. With a good search function. (Hmm, maybe i should try see if i can make a search-keyword with a 'specialized' google search)

Cond with variables. I have made some common lisp macros like (when-with var cond &rest body), (if-with var cond if-t if-f), (if-use cond-out if-f), but not entirely sure how to do this properly with cond.

Slurping stuff directly from C files; (include<> "opengl.h"), producing output to javascript/flash and such would be very cool.

Forcing lisp to stay in different subsets of functions, as to make 'safe' scripting, where the code cannot damage your computer. For instance, as in an html-like 'language', force it to stick to functions determining how a page looks.

@danb: good one Edit: just realised, (if cond (ref a) (ref b)) would produce something settable. Not perfect though, still need to repeat ref. Maybe try make anything being setted to be a reference.. somehow.

Perhaps lisp should either have more explicit separation of the function namespace, or be lisp-1: I just realized this:

(defvar miauw 45)

(defpackage #:miauw
  (:use #:common-lisp)
  (:export miauw))

(in-package #:miauw)
(defun miauw (&optional (n 3))
  (dotimes (k n) (format t "Mew~%")))

(in-package #:cl-user)

(use-package '#:miauw)

(miauw 4)

As often, silly i didn't think of this before..

IMO, Lisp-1 is a needless headache. It's useful for newbies to avoid early complexity, but later it seems like a straightjacket with no big benefit. FUNCALL is not as ugly as Lisp-1 proponents make it out to be.

I agree, but i was talking about the namespacing. With choosing lisp-2 you do get a choice between really separating the function namespace for the packages or namespaces, etc, or only separating the symbols. CL has chosen the latter. One could argue that one should avoid having this choice.

One could separate the function namespace the other way by letting #' access the function namespace, rather then getting the function of that name; subtil difference being that then you would export functions with (:export #'fun) Still, i think it is rather minor.