How to organize source code in multiple files?

[humpolec]

Most of the time I work in Emacs with interpreter running and all my code loaded, but I wanted to make an ASDF definition file.
I understand that when a multi-file project is compiled and loaded, it's done sequentially, file after file, so you have to be careful about the definitions order:

- Compilers issue warnings on compiling forward references to functions, but I suppose this is harmless - and unavoidable in case of a circular dependency. But what about a function declared inline?
- Macros should be compiled before usage, so they should be grouped somewhere near the beginning of build order - even if a macro actually belongs to a specific domain of code (i.e. specific class)?
- Classes have to be defined before a DEFMETHOD can specify them. Does that mean they should be separated from the relevant methods and put all in a different file?
- Is it important to specify the generic methods? And where should I put them?

This is probably again me not understanding "the Lisp way to do things", but I feel I have to group code in a way that pleases the compiler, not in one that makes most sense semantically...

[dlweinreb]

You ought not get a warning for a forward reference to a function. Lisp compilers generally defer these checks until the end of a "compilation unit", so that it will only tell you that a function is undefined if the function was not seen in this compilation unit, or earlier in the compilation.

Macros must precede usage. If you're going to use the macro throughout many files, e.g. if it is a general utility, it's often good to put it in a separate file of utility macros. If it's only going to be used within one file, just put it at the beginning of that file.

If a class's methods are going to be dispersed amongst many files, then you might want to put the class definition in a separate file. Otherwise it's fine to just put it into a file and then put the method later in the file.

I think it's very good practice to use explicit defgeneric forms, particularly so that you can use their doc strings to explain the generic meaning of the function. You can put them anywhere after the class definition. You might want to put them all together, after the class definition, so that someone reading the code can see the whole "protocol" all together.

A slightly more tricky issue is Lisp functions that are called by macros: not by the code that the macro expands into, but by the macro itself during expansion. Such functions must be available when the macro is expanded. You can either use eval-when to make
sure that they get defined at compile-time, or you can put them into another file (Y) and tell ASDF that in order to compile X, you must first load Y.

Yes, it's unfortunate that a Lisp programmer has to worry about all this. Many of us have talked about the possibility of doing a Lisp IDE that would take care of all this automatically. But as far as I know, nobody has that yet.

[findinglisp]

Yes, it's unfortunate that a Lisp programmer has to worry about all this. Many of us have talked about the possibility of doing a Lisp IDE that would take care of all this automatically. But as far as I know, nobody has that yet.

Not to go off-topic here too much, but this has always stuck me as being really strange. I would think that it would be relatively easy to create a great Lisp IDE that would provide all sorts of fancy refactoring and cross-checking tools. It doesn't seem like that has happened, at least in the open source world. I have never used Allegro or LispWorks, so maybe I just don't know what I'm missing. SLIME is great for what it does, but it just doesn't address some of these other issues that I would think would be relatively easy to do given that Lisp is trivially parseable in the editor.

Dan, Lisp machines did this stuff all the time back 20 years ago, right? So why have we lost it today?