macro-friendly equivalent of APPLY

I've encountered this situation a couple of times now, and I'm not sure how to address it.

I will have some macro I've defined to use a &rest parameter. For example, it might be a macro that follows the same basic form as CL:+

(defmacro plus (&rest terms)
   ...)

It's a macro because it needs to do some funky manipulation of its args before evaluating them. It uses &rest because in most cases, this is convenient, and it seems familiar due to its symmetry with the standard function CL:+).

Then I come to use it, and I find myself most with a list of args I want to pass to it and wanting to use APPLY to split the arg list TERMS into individual args, but I can't because PLUS is not a function.

Is it reasonable to write a macro that will let me do this? The fact that there isn't such a thing in CL already makes me think there's something fundamentally wrong with that idea.

I think it's a bad idea, it's not needed. Anyway I come up with this snippet:

(defmacro get-env (&environment env) env)

(defmacro macro-apply (symbol &rest args)
	`((lambda (form) (funcall (macro-function ,symbol) form (get-env))) '(,symbol ,@(butlast args) ,@(first (last args)))))

(defmacro plus (&rest args) `(+ ,@args))

(macro-apply 'plus 1 2 3 (4 5)) ; (+ 1 2 3 4 5)

Still you must wrap macro-apply by eval and use quasiquotation to get a functional behaviour like apply.

There are many times when it is convenient to write a macro's body as an ordinary function. Then the macro is a simple wrapper that hooks in the compiler, and the function does all the real work.

This is one case where that's a useful pattern.

Could you give us an example where you would like this macro-apply to be used?
Eg. an example from the top of your head and what the desired result should be.

sylwester wrote:Could you give us an example where you would like this macro-apply to be used?
Eg. an example from the top of your head and what the desired result should be.

Indeed, that would probably be more useful than the vague description I gave.

I'm trying to extract a representative case from what I'm writing that illustrates the proper context without being too long. It's not obvious how to do that - and maybe that's part of the problem.

sylwester wrote:Could you give us an example where you would like this macro-apply to be used?
Eg. an example from the top of your head and what the desired result should be.

Hope it's not bad form to resurrect my own thread here. I fixed my previous problem with a different approach, but this came up again for me recently.

The simplest case I could think of is:

"Write a function that returns a lambda, with lambda list (&rest args), that simply does an 'and' of all the ARGS - using AND.

It's a bit contrived, as you'd end up with a non-short-circuiting and function. But for what it's worth, this theoretical MACRO-APPLY might let us do something like this:

(defun ander ()
  (lambda (&rest args)
    (macro-apply and args)))

I'm not quite sure how else I would do this.

garethw wrote:I'm not quite sure how else I would do this.

See the Lisp FAQ: Why can't I apply #'AND and #'OR? and CLHS: EVERY, SOME, NOTEVERY, NOTANY

(defun ANDer (&rest args)
  (every #'identity args))

(defun ORer (&rest args)
  (some #'identity args))

(defun NANDer (&rest args)
  (notevery #'identity args))

(defun NORer (&rest args)
  (notany #'identity args))

Unfortunately this only solves the special AND, OR, NAND, NOR problems, but not the general case of APPLYing special operators and macros.

- edgar

edgar-rft wrote: Unfortunately this only solves the special AND, OR, NAND, NOR problems, but not the general case of APPLYing special operators and macros.

Thanks for the reply Edgar; unfortunately, it's the general case I'm trying to address. I just gave the AND as an example.

I'm left with the impression that maybe I want to avoid defining macros to use &rest arguments. Or pair them all up with functions that do the work as I think nuntius is suggesting.

Coming back to your original PLUS question (because the examples explain better what I mean), here is a PLUS macro that can compute normal numbers as well as string numbers from "zero" to "nine":

(defmacro plus (&rest args)
  `(apply #'+ ',(mapcar #'(lambda (x)
                            (cond ((numberp x) x)
                                  ((stringp x)
                                   (cond ((string-equal x "zero")  0)
                                         ((string-equal x "one")   1)
                                         ((string-equal x "two")   2)
                                         ((string-equal x "three") 3)
                                         ((string-equal x "four")  4)
                                         ((string-equal x "five")  5)
                                         ((string-equal x "six")   6)
                                         ((string-equal x "seven") 7)
                                         ((string-equal x "eight") 8)
                                         ((string-equal x "nine")  9)
                                         (t (error "unknown string ~s" x))))
                                  (t (error "illegal argument ~s" x))))
                          args)))

Here is how it works:

CL-USER> (macroexpand-1 '(plus "one" "two" "three"))
(APPLY #'+ '(1 2 3))
T

CL-USER> (plus "one" "two" "three")
6

The argument dispatcher can be made a separate function:

(defun arg-dispatcher (x)
  (cond ((numberp x) x)
         ((stringp x)
          (cond ((string-equal x "zero")  0)
                ((string-equal x "one")   1)
                ((string-equal x "two")   2)
                ((string-equal x "three") 3)
                ((string-equal x "four")  4)
                ((string-equal x "five")  5)
                ((string-equal x "six")   6)
                ((string-equal x "seven") 7)
                ((string-equal x "eight") 8)
                ((string-equal x "nine")  9)
                (t (error "unknown string ~s" x))))
         (t (error "illegal argument ~s" x))))

The arg-dispatcher function is called from within the macro definition:

(defmacro plus (&rest args)
  `(apply #'+ ',(mapcar #'arg-dispatcher args)))

CL-USER> (macroexpand-1 '(plus "one" "two" "three"))
(APPLY #'+ (1 2 3))
T

This can be even simplified to:

(defmacro plus (&rest args)
  `(+ ,@(mapcar #'arg-dispatcher args)))

CL-USER> (macroexpand-1 '(plus "one" "two" "three"))
(+ 1 2 3)
T

CL-USER> (plus "one" "two" "three")
6

Using an argument dispatcher also works with special operators and macros.

Special note: In my examples above above it's important that the dispatcher works independent from any local state from the environment where the macro is expanded (usually lexical values from LET variables surrounding the macro call). If the local environment is needed for correct dipatch then things start to get more hairy like in Goheeca's GET-ENV example.

The general pattern is:

(defmacro <macro-name> (&rest args)
  (<function-special-or-macro> ,@(mapcar #'<lambda-or-dispatch-function> args)))

Is this what you are looking for?

- edgar

P.S.: Paul Graham's On Lisp contains lots of macro tricks.

edgar-rft wrote:This can be even simplified to:

(defmacro plus (&rest args)
  `(+ ,@(mapcar #'arg-dispatcher args)))

CL-USER> (macroexpand-1 '(plus "one" "two" "three"))
(+ 1 2 3)
T

CL-USER> (plus "one" "two" "three")
6

Using an argument dispatcher also works with special operators and macros.

Special note: If the dispatcher is a function outside the macro definition as shown above it's important that the dispatcher works independent from any local state from the environment where the macro is expanded (usually lexical values from surrounding LET variables). If the local environment is needed for correct dipatch then either the complete dispatcher must be defined inside the macro definition like the LAMBDA dispatcher in the first example above, or otherwise things start to get more hairy like in nuntius' GET-ENV example.

The general pattern is:

(defmacro <macro-name> (&rest args)
  (<function-special-or-macro> ,@(mapcar #'<lambda-or-dispatch-function> args)))

Is this what you are looking for?

Not quite - I think it's just pushing my problem one layer up. I'm actually doing something somewhat like this already, although much less elegantly. My arg-dispatchers are themselves macros, as they need may not want to evaluate their args directly.

The problem comes when in the general case of using PLUS itself. At many call sites for PLUS, I want to call it with some arguments that are already in a list. If it were a function, I'd apply it to the args.

Here's a silly example of maybe where I'd like to use your PLUS to add up all the numbers extracted from a string, using a function EXTRACT-NUMERIC-TOKENS that might return a list like ("one" 2 "five" 92):

(defun sum-numbers-in-string (string)
  (let ((number-list (extract-numeric-tokens string)))
    (apply #'PLUS number-list)))    ;; apply? 

Or have I missed something?

I think you have already discovered why Common Lisp doesn't want special operators and macros to be FUNCALLed or APPLYied, because there's a potential infinite number of special cases to observe. Complex syntax manipulations usually are implemented by recursive tree traversal, not by stacking macros on macros. Sorry, but I would need a specific code example to see why your macros cannot be implemented as functions, otherwise I have no idea how to solve this problem.

- edgar