CLOS usage

I was just reading the poll topic in the Books forum about which Lisp book is your favorite. At one point in the discussion, I mentioned that a big difference between ANSI Common Lisp and Practical Common Lisp is the use and coverage of CLOS. Paul Graham doesn't seem to like CLOS and spent only a chapter covering it in ANSI Common Lisp, whereas Peter Seibel uses it all the time in Practical Common Lisp. I thought I'd ask people how much they use CLOS when writing day-to-day programs. In the discussion, feel free to provide some "color commentary" on your answer.

Personally, I tend not to use CLOS very often. It's not that I have some sort of a l33t spirit -- I just don't fancy object orientation for some projects. In fact, unless I absolutely require the use of OOP to make a meaningful architecture (e.g. I really need inheritance), I tend not to use it at all. Structures, or for some simple cases even simple lists are often more than enough.

I will admit, however, that I do enjoy CLOS when I use it. It's a very complete and powerful feature, and I especialy liked it when I saw it, considering that I was coming from the world of C++.

CLOS is amazingly powerful and above all it's amazingly simple to use when you look at how powerful it is. The other system i have played with, that seemed as friendly in it's approach to Object Orientation was Smalltalk but it didn't feel as natural to me since it enforces a strict OOP style and i tend to find that limiting and harder than it should. CLOS is great because you can use a powerful OO model without having to abandon other programming styles, you can simply mix the best of two worlds when they make sense.

I tend to use methods with structs more than classes.

Hm, I like CLOS and fairly often use objects. However, I prefer using "regular" functions instead of methods wherever methods aren't explicitly neccessary.

I was originally going to vote for "When it makes sense" (or even a step down), until I realized that I frequently have a print-object method defined for my structs or conditions or whatever, and use conditions, and use defmethod on structs, etc. CL-SQL uses it, and I use that as well.

luskwater wrote:I was originally going to vote for "When it makes sense" (or even a step down), until I realized that I frequently have a print-object method defined for my structs or conditions or whatever, and use conditions, and use defmethod on structs, etc. CL-SQL uses it, and I use that as well.

"Resistance is futile..."

To my own surprise, because I come from Java, I'm not using clos, only lists, maps, etc, and mostly functions or methods that are functional (no setf). I may be better off moving towards the middle with structs for certain things. But I would like to stay mostly functional.

mike wrote:To my own surprise, because I come from Java, I'm not using clos, only lists, maps, etc, and mostly functions or methods that are functional (no setf). I may be better off moving towards the middle with structs for certain things. But I would like to stay mostly functional.

What speaks against using structs or objects and still do functional programming? A simple example might be the implementation of a binary tree using structs. In spite of using structs there's no reason to use setf or other functions with side-effects...

Alexander Lehmann wrote:

mike wrote:To my own surprise, because I come from Java, I'm not using clos, only lists, maps, etc, and mostly functions or methods that are functional (no setf). I may be better off moving towards the middle with structs for certain things. But I would like to stay mostly functional.

What speaks against using structs or objects and still do functional programming? A simple example might be the implementation of a binary tree using structs. In spite of using structs there's no reason to use setf or other functions with side-effects...

I agree. I only meant to contrast it more with java where side-effects are commonplace.

Yes, I thought so, but wasn't sure

I don't use or think about CLOS as much as I'd like. I think Kenny Tilton's Cells project looks pretty cool, and it uses CLOS through-and-through. I've done most of my programming over the last 10 years in shell and Perl, which has warped me so that I tend to think in terms of lists and hashtables (arrays and hashes, in Perl). I'd like to change my thinking to flow more along the lines of dataflow and dependencies instead of datastructures and functions.

It has always seemed to me that CLOS is a product of that age when it was in vogue to make everything object oriented. Now, it was written in Lisp, so I know that CLOS and Smalltalk are probably the highlights of OO and the C++ that I use in my day job has prematurely turned me off from OO, but I don't think that is the whole story. In Lisp, it has been so trivial to build up a data structure, store it, retreive it and destroy it just a few methods later, that I feel no need to ever make a heavyweight solid data structure of the type that OO champions. Another problem with OO is that it tends to limit exactly how code can get reused. The structureless nature of Lisp allows me to take any meaningful chunk of code and not just put code before or after it, but to take callbacks (which are so awkward in C++ that for the first ten years, I had to look up the syntax), and make them trivial, so that I can make a piece of code that is designed to be (X A X') with your custom chunk A in the middle. In an OO paradigm, the callback would be to call out to obj.A and to overload A. At least java made everything virtual. I cannot tell you how much pain that causes me, when the child class doesn't even get the final say over which versions of methods will be used 100% of the time. Sorry for the rant, I just hope that OO has the mark of shame all over it when stateless functional programming takes over. Seriously, a stack trace that is 100% meaningful, where I don't have to play detective to figure out why the values I am looking at are what they are... that would be nice to have all the time.

pTymN wrote:Seriously, a stack trace that is 100% meaningful, where I don't have to play detective to figure out why the values I am looking at are what they are... that would be nice to have all the time.

Well, I was really excited about (pure) functional stuff when I first got interested in Haskell, after coming from the pain of C++ and the verbose boulder-dragging of Java. Then I saw some stack traces which kind of stopped me in my tracks.
Maybe you get used to it, but... ouch.

That's also something that I find a little painful with Lisp; especially the lack of line numbers in the uncaught/REPL error handler stack traces, but the generally mystifying nature of error messages I've been getting as a newbie to the language. There's a lot to be said for messages like "Array index out of bounds (CarThingy.java, line 85)" and a backtrace which lists the classes (usually = files) and line numbers of the higher stack frames.

At ITA Software, we use CLOS very heavily. One reason is that we have built an object-relational database access subsystem, in which (to simplify a big) every database table corresponds to a CLOS class. However, even when we aren't using that, we still use CLOS heavily. Object-oriented programming is a very useful tool for achieving modularity in many common scenarios that arise in programming, particularly in large systems.

Alexander Lehmann wrote:Hm, I like CLOS and fairly often use objects. However, I prefer using "regular" functions instead of methods wherever methods aren't explicitly neccessary.

In CLOS, you are still using functions. The best way to think about CLOS is that what you're writing is generic functions. (This is clearer if you always write explicit defgenerics, rather than letting CLOS put them in automatically.) From the caller's point of view, it's just a function! The defmethod's are part of the internal implementation of the function. When is it good to implement a function using this (defmethod) mechanism. Well, if you're going to do different implementations of the same concept based on the type (subclass, in practice) of one (or more!) of your arguments, then it's perfect.

CLOS is just one more tool in the toolbox. It's not the answer to everything; it's the answer to some specific problems.

I always think of the "stream" concept is the best example of OOP. A stream is an abstraction with operations like "write this string to the stream" and such. That has one, unified generic meaning, once you understand what a stream is. But there are many specific implementations (depending on the underlying device). So it's very useful to have a "stream" class, and subclasses for the specific concrete implementations (send the string to the serial port, send the string to a file, etc).

I am planning to write more about this on my blog (dlweinreb.wordpress.com). Some issues of CLOS philosophy have been under heavy discussion at ITA lately (particularly the interaction of keyword arguments with generics functions), so I've been thinking about it a lot lately.

Exolon wrote:

pTymN wrote:That's also something that I find a little painful with Lisp; especially the lack of line numbers in the uncaught/REPL error handler stack traces, but the generally mystifying nature of error messages I've been getting as a newbie to the language. There's a lot to be said for messages like "Array index out of bounds (CarThingy.java, line 85)" and a backtrace which lists the classes (usually = files) and line numbers of the higher stack frames.

The quality of the error messages depends primarily on which implementation of Common Lisp you're using. There are eleven of them out there still being maintained actively (see http://common-lisp.net/~dlw/LispSurvey.html). I've been happy with the error messages in SBCL and CCL (OpenMCL), and the stack traces.

One thing that you may need to do is to make sure to compile with proper "safety" declared. This is like the -g argument to C compilers, telling the compiler to keep around extra information to allow the runtime to produce good debugging info. Check the reference material for your Lisp implementation.

pTymN wrote:It has always seemed to me that CLOS is a product of that age when it was in vogue to make everything object oriented. Now, it was written in Lisp, so I know that CLOS and Smalltalk are probably the highlights of OO and the C++ that I use in my day job has prematurely turned me off from OO, but I don't think that is the whole story. In Lisp, it has been so trivial to build up a data structure, store it, retreive it and destroy it just a few methods later, that I feel no need to ever make a heavyweight solid data structure of the type that OO champions. Another problem with OO is that it tends to limit exactly how code can get reused. The structureless nature of Lisp allows me to take any meaningful chunk of code and not just put code before or after it, but to take callbacks (which are so awkward in C++ that for the first ten years, I had to look up the syntax), and make them trivial, so that I can make a piece of code that is designed to be (X A X') with your custom chunk A in the middle. In an OO paradigm, the callback would be to call out to obj.A and to overload A. At least java made everything virtual. I cannot tell you how much pain that causes me, when the child class doesn't even get the final say over which versions of methods will be used 100% of the time. Sorry for the rant, I just hope that OO has the mark of shame all over it when stateless functional programming takes over. Seriously, a stack trace that is 100% meaningful, where I don't have to play detective to figure out why the values I am looking at are what they are... that would be nice to have all the time.

Sorry, I'm having a lot of trouble understanding what you mean by things like "(X A X')" and "obj.A".

What's the problem with doing callbacks with CLOS? There are two senses of this that you might mean. First, in OO programming, it's common to have abstract base classes that call some method that is not defined by the abstract base class itself. Java calls these abstract methods; CLOS unfortunately has no particular name for them. To make a concrete subclass that can be instantiated, you must provide an implementation of this abstract method. You can think of this as a callback, in some sense.

Second, there are just plain ordinary callbacks that have nothing special to do with CLOS. You just pass a callback function to a generic function, which stores it away somewhere, and calls it later at the appropriate time. It's all just Lisp, so it's easy to do that, CLOS or no CLOS.

dlweinreb wrote:

Alexander Lehmann wrote:Hm, I like CLOS and fairly often use objects. However, I prefer using "regular" functions instead of methods wherever methods aren't explicitly neccessary.

In CLOS, you are still using functions. The best way to think about CLOS is that what you're writing is generic functions. (This is clearer if you always write explicit defgenerics, rather than letting CLOS put them in automatically.) From the caller's point of view, it's just a function! The defmethod's are part of the internal implementation of the function. When is it good to implement a function using this (defmethod) mechanism. Well, if you're going to do different implementations of the same concept based on the type (subclass, in practice) of one (or more!) of your arguments, then it's perfect.

Please correct me if I'm wrong, but I thought that the major difference between "functions" and "methods" -- with respect to Lisp and/or CLOS -- was that for methods, CLOS first looks up the most suitable (specialized) version of a method in regard to the given parameters, whereas this need not be done for plain "functions"?

Alexander Lehmann wrote:Please correct me if I'm wrong, but I thought that the major difference between "functions" and "methods" -- with respect to Lisp and/or CLOS -- was that for methods, CLOS first looks up the most suitable (specialized) version of a method in regard to the given parameters, whereas this need not be done for plain "functions"?

What you are calling methods are actually generic functions. Methods are simply function-like elements associated with generic functions. dlweinreb's point was that plain functions and generic functions can be used in the same manner, although the internals are quite different.

Geoff Wozniak wrote:

Alexander Lehmann wrote:Please correct me if I'm wrong, but I thought that the major difference between "functions" and "methods" -- with respect to Lisp and/or CLOS -- was that for methods, CLOS first looks up the most suitable (specialized) version of a method in regard to the given parameters, whereas this need not be done for plain "functions"?

What you are calling methods are actually generic functions. Methods are simply function-like elements associated with generic functions. dlweinreb's point was that plain functions and generic functions can be used in the same manner, although the internals are quite different.

Thanks Geoff, then I've misunderstood dlweinreb's point. I know that "methods" actually aren't "methods" but "generic functions" in Lisp. Anyways -- what about the (possible) overhead that is needed for finding and jumping to the most suitable generic function in contrast to "ordinary" functions? I suppose there is some overhead, but I don't know for sure...

Alexander Lehmann wrote:

Geoff Wozniak wrote:

Alexander Lehmann wrote:Please correct me if I'm wrong, but I thought that the major difference between "functions" and "methods" -- with respect to Lisp and/or CLOS -- was that for methods, CLOS first looks up the most suitable (specialized) version of a method in regard to the given parameters, whereas this need not be done for plain "functions"?

What you are calling methods are actually generic functions. Methods are simply function-like elements associated with generic functions. dlweinreb's point was that plain functions and generic functions can be used in the same manner, although the internals are quite different.

Thanks Geoff, then I've misunderstood dlweinreb's point. I know that "methods" actually aren't "methods" but "generic functions" in Lisp. Anyways -- what about the (possible) overhead that is needed for finding and jumping to the most suitable generic function in contrast to "ordinary" functions? I suppose there is some overhead, but I don't know for sure...

A note on the terminology: a "generic function" means a Lisp function that's implemented by "methods". A method and a generic function are not the same thing. This is made somewhat confusing by the fact that CLOS does not require an explicit "defgeneric"; the presence of methods causes it to be as if an implicit "defgeneric" form had been seen. I am generally a big fan of using explicit "defgeneric" forms, since that gives you a good single place to document the contract of the generic function.

Yes, when you call a generic function, CLOS has to find the appropriate method(s) to call, and in general this costs something. (The "jump" part is just a Lisp function call, so that's very fast.) The amount of overhead depends on a lot of things. There are different implementations of CLOS in the different Lisp implementations. (Many of them are forks of the PCL implementation, with improvements; some of them are not.) CLOS implementations do a lot of caching and pre-computing. The book "The Art of the Metaobject Protocol" explains how this works. Some CLOS implementations optimize certain cases, such as a generic function that has only one defmethod. (Since Lisp is dynamic, a second defmethod could get defined later, so such optimizations have to adjust themselves for that at runtime!) The only way to know the overhead is to come up with the specific case you're interested in, and measure it.

If using OOP is the right modularity for your program, you rarely have to worry about this overhead. The main exception is if you're in a performance-critical area (known informally as "an inner loop"). Then it's worth taking a look at the speed and seeing whether the overhead is a problem or not. But don't optimize prematurely!

Thanks Geoff. As I totally agree with what you've written, IMO the "problem" was just due to my misusage of the term "method".

The origin of "my" part of the discussion was where I stated that I preferred "functions" over "methods" wherever possible. My sole intention with that was to mention the exact difference like you've explained nicely and in detail above. While re-reading this thread I also noticed that my writing is basically just a repetition of what had been said before