User-defined functions in macros

Hi all,

I ran into a situation recently where I had a macro that was a getting a bit long, so I decided to factor out some of its functionality.

I looked at what it needed to do, and decided to implement it as a function that would be called from the macro.

It took me a while to figure out why I needed to compile the file twice to get it to work - on the first pass, the function was not yet defined, so the macro couldn't do its job. On the second pass, of course, it worked.

Is there a standard way to deal with this sort of pattern? Does the pattern itself raise code smell alarms?

Wrap your function definition into EVAL-WHEN like this:

(eval-when (:compile-toplevel)
  (defun ...

  ))

There is eval-when for this, but when I tried to reproduce the mentioned behaviour I couldn't find out the layout of functions and macros which are using them so maybe your problem is about something else.

What implementation are you using?

Thanks for the replies, everyone. Sorry for the delayed response.

For posterity, here's a pathological case I extracted:

(defpackage :ca.telperion.test
  (:nicknames :test)
  (:use :common-lisp))

(in-package :test)

(defvar *af* 0)

(defun expand-flag-ref (flag)
  (let ((idx (ecase flag
               ((c) 0)
               ((n) 1)
               ((p) 2)
               ((h) 4)
               ((z) 6)
               ((s) 7))))
    `(ldb (byte 1 ,idx) *af*)))

(defmacro get-flag (flag)
  (expand-flag-ref flag))
    
(defun add8 (a b)
  "8-bit addition with flag side-effects"
  (let ((res (+ a b)))
    (setf (get-flag s) (if (< res 0) 1 0)
          (get-flag z) (if (= res 0) 1 0)
          (get-flag h) (if (and (= (ldb (byte 1 3) a) 0)
                                (= (ldb (byte 1 3) res) 1)) 1 0)
          (get-flag p) (if (= res #x7F) 1 0)
          (get-flag n) 1
          (get-flag c) (if (= res #x7F) 1 0)) ;*FIXME*
    (logand res #xFF)))

Using SBCL 1.1.3/Emacs/SLIME, compile with C-c C-k. Fails, complaining that

error:
during macroexpansion of (SETF # #). Use *BREAK-ON-SIGNALS* to intercept.
The function CA.TELPERION.TEST::EXPAND-FLAG-REF is undefined.

C-c C-k again and it compiles and runs successfully.

Wrapping (defun expand-flasg-ref() ... ) in (eval-when (:compile-toplevel) does indeed address the problem, but I don't grok why. I've used that to invoke some functions and do a little processing at compile time, but I'm not sure what it means to wrap a defun in it. I'm not sure why it doesn't just... you know... compile it. Guess I need to cozy up with the standard a little more.

Oh, and feel free to constructively mock anything else you see in my code!

See this and I know now that I actually was loading the file and, therefore, the problem wasn't arising for me.

I've struck the same problem in the past; I fixed it by putting the prerequisite function into a different file that got compiled before the one containing the macro. This guaranteed that it was available when the call to defmacro occurred, and made me appreciate the convenience of ASDF even more.

Note that I said "fixed" rather than "resolved"; eval-when may well be the better solution. I was already clambering up a couple of learning-curves too many at the time, so when I found a way to make this problem go away, I didn't dig any further at the time.

Compilation is a 3-stage process, first a file is compiled, the the generated code is written to a .fasl (fast loader) file, and finally that .fasl file is loaded into your image.

During compilation each form in turn is READ from the file (invoking any relevant read macros), the form is expanded with MACROEXPAND to produce a form that is only comprised of special-forms[1], the form is then given to the compiler. Some forms have effects at compile time (e.g. IN-PACKAGE) but many don't.

When the compiler is given a top-level[2] form that is a DEFMACRO it is compiled and added to the current image in addition to being written to the .fasl. This means that later forms in the same file can use that macro and be properly expanded with a gotcha: Top-level DEFUNs only record the name and argument list at compile time, the generated code only goes into the .fasl to be LOADed later.

This means that, by default, you can't make a macro that relies on a helper in the same file, when that file also contains an invocation of the macro, because when MACROEXPAND expands the macro it will try to run the helper, but the helper isn't loaded in the image yet, hence your error.

Top-level EVAL-WHENs can allow you to force a form to be evaluated at compile time, or you can put the macro helpers in another file which is loaded earlier by ASDF, as has been said earlier in the thread.

[1] http://www.lispworks.com/documentation/ ... ecial_form
[2] http://www.lispworks.com/documentation/ ... level_form

Thanks for the excellent responses.