CLOS vs closures

In the year or so that I've spent with Common Lisp, I've been trying to make a conscious effort to try to make effective use of lexical closures, rather than immediately run to the relative familiarity of objects.

I've found them really useful on occasion, but as I've tried to use them in more ambitious use cases, I've found myself wanting to use CLOS instead. I'm not sure if that's just a matter of familiarity.

There is some discussion on the topic here, but much of that seems a bit of a pissing contest, and I didn't find it all that enlightening.

My gut feel so far is that I'd tend to use closures in simpler cases, such as:

• 1. Where you might see a singleton - a number of top-level functions closed over the same lexical bindings

  (let ((things ())
        (stuff 0))
    (defun add-thing (thing)
       ...)
    (defun rm-thing (thing)
        ...))

  2. Where you might be generating an abstraction that really is a single function

  (function make-fsm (x)
    (let ((state 0)
          (other-stuff 42))
      (lambda ()
        (do-things-with state x)))

When I've tried to implement hierarchies of closures returning closures - or multiple closures - I found them harder to conceptualize and also debug.

Am I short-changing closures?

IMO, objects should be used for intentional program structure.

Closures should be used for ad-hoc cases, such as registering callbacks, spawning threads, passing behavior through higher-order functions such as map, etc.

So objects are ideal when you are defining structures, and closures are ideal when you are passing a "function object" to an API to be invoked.

Said differently, C++ has only had objects for many years. The C++11 standard added a "lambda" feature because the added cost of creating an object that implements an interface really is overkill for many applications. Almost nobody used std::for_each because it required moving the body to a separate class definition, passing the required variables to the constructor, and then passing the object to std::for_each. The remedy was worse than the disease. Now that C++ has lambdas, the balance is much different. This is true even though C++ closures are still restrictive compared to those in Lisp. For example, you must manually specify which variables to close over and guarantee that the closure will not outlast any referenced variables.

Formally closures and objects are equivalent, but so are all turing-complete programming languages, equivalent doesn't always mean convenient

Let Over Lambda by Doug Hoyte is all about doing stateful programming using closures (and macros) insted of objects. I found it a pretty interesting read, but I haven't really changed my programming style much since reading it. OTOH knowing about alternative techniques can help you even when you don't use them often (there's a lisp quote in this vein).

I love closures so my heart wants to implement it's own object system. With the right macros you might make your own object system that is more efficient and requires less code to use.

When that said, I would prefer other people implemented with CLOS or other standard features so it would be easier to read and modify.

It's pretty much the same as you need a lots of benefits to justify writing a macro. Paul Graham touched this in On Lisp and mentions it in this essay.

Great answers, all - thanks.

I've read LoL, but I probably did so too soon - I'd probably gain more from it now on re-reading it.

pjstirling wrote:OTOH knowing about alternative techniques can help you even when you don't use them often (there's a lisp quote in this vein).

I rather wish esr hadn't concluded his remarks with that, actually. It was why I put off learning Lisp for a dozen years, which I now sincerely regret.

It's been a year since I began using Common Lisp and to this point I've been happy using plain closures over CLOS (this could be because years of C++ and Java programming have taught me to abhor OO). After all, the traditional way to write 'generic' functions without using formal generic functions is to go higher-order, no? Then, if you want to pass your 'generic' function an array, say, you wrap it in a closure and pass that, as in (generic-fun (curry #'aref array-name)).

Shouldn't CLOS be used only if you have a real need for features you can't easily get with closures, like multiple inheritance and access to the internal state? It seems like so much unnecessary baggage otherwise.

mthom wrote: Shouldn't CLOS be used only if you have a real need for features you can't easily get with closures, like multiple inheritance and access to the internal state? It seems like so much unnecessary baggage otherwise.

I think it's mostly a matter of taste and preference, though I think CLOS can help with maintainability as systems grow larger and more complex.
Personally, I find it very useful for organising code and for passing bundles of state around with a minimum of programmer overhead; it also allows me to use pre-build generic machinery to quickly build up an API. It's possible that said machinery can become a performance bottleneck, but I don't operate at the scale of, say, ITA. If it _does_ become a performance limitation, remember that a call to a generic function looks like a call to any other kind of function, so you can seamlessly replace the OO stuff with closures (or whatever else) where your profiling shows that it's slowing you down. I'm also a devotee of the school of first making it work, then making it work correctly, then making it faster when and where speed actually emerges as a problem.

This article does a pretty good job of illustrating the mindshift that comes from thinking in terms of defgeneric first and worrying about actual objects later.

I'll probably get shot down for this, but I see an analogy with RDBMSes: sure, you can roll your own purpose-designed and perfectly performance-optimised system that's tailored for the specific problem at hand, but you also get the maintenance burden and learning-curve that goes with it, which is multiplied if/when other people get involved. Alternatively, you can use a pre-built generic system that can be molded to a very close approximation in much less time, and that many other people are already familiar with. It _could_ become a performance bottleneck, but a)it probably won't in practice, and b)if it does become a bottleneck, you can re-engineer the solution then. In the meantime, it lets you get on with the bits of your application that _don't_ already have a general solution - it speeds you up more than it slows you down.

Personally, I have never understood the Lispers that want to avoid CLOS at all costs and use closures for everything. While I do think CLOS can be overused, primarily by people with strong OOP backgrounds who haven't learned to think in terms of functions and high-order programming, objects are still useful and CLOS provides a very rich object system. When you see people trying to avoid CLOS, they typically end up building their own ad-hoc object system, which nobody understands because it isn't CLOS and which doesn't offer much of anything over CLOS. In fact, if it did offer anything over CLOS, it would end up being as sophisticated as CLOS, and therefore you'd have to wonder why you built and not have to maintain your own version of CLOS that isn't CLOS.

So, the upshot is:

• Don't use CLOS for everything.
• Learn high-order functional programming and use that when appropriate.
• But when objects are appropriate, use CLOS. Don't reinvent the wheel unless you are just experimenting and want to understand how it works. And in that case, just buy a copy of The Art of the Metaobject Protocol (which is a great book and you should own in any case).