Programming Style & (eval ...)

[taylor_venable]

Hello all. I've been using Lisp in various forms for the last few years, but I've only recently started to really get into it. I find that for me to really get to know a language it helps to write some common libraries in it, so I decided to start porting some of Haskell's list library functions into Lisp. I ran into a problem right away though when trying to write all which takes a predicate and a list, returning true when all the elements in the list satisfy the predicate. So, for example

Code: Select all

CL-USER> (all #'(lambda (x) (= (mod x 2) 0)) '(2 2 4 8 6 0))
T
CL-USER> (all #'(lambda (x) (= (mod x 2) 0)) '(2 2 4 8 6 1))
NIL

My first thought was that this is just a fold of the and operation into the result of mapping the predicate over the list. That means basically applying and to all the list elements, so in my ignorance I tried to do:

Code: Select all

(apply 'and (mapcar #'evenp '(2 4 6 8 9))) => nil

But of course that doesn't work because and isn't a function, it's a macro. So the solution I eventually discovered was:

Code: Select all

(defun all (f l)
  (let ((r (mapcar f l)))
    (eval `(and ,@r))))

Which leads me to my question: is it considered acceptable style to use eval to do something like this? I typically shun the use of eval in other languages, because it's usually just a cheap way of getting something done which could be handled in more elegant ways, but with the data-is-code orientation of Lisp, is this a common tactic to employ?

I'd also be interested to see any alternate implementations of and; I thought it might be cool to have a macro that expanded into [using the example above] (and t t t t nil) but I can't seem to make it happen. My best guess was

Code: Select all

(defmacro all (f l)
  `(let ((r (mapcar ,f ,l)))
     (and ,@r)))

But that doesn't work because r is unbound -- which I think is because it doesn't exist in the environment into which the macro is expanded. However, even if I define an r using let before using the macro, it still doesn't work, not that that would be what I wanted anyway.

Thanks for any advice!

[ramarren]

I ran into a problem right away though when trying to write all which takes a predicate and a list, returning true when all the elements in the list satisfy the predicate.

First, note that such function already exists: every.

My first thought was that this is just a fold of the and operation into the result of mapping the predicate over the list. That means basically applying and to all the list elements, so in my ignorance I tried to do:

Code: Select all

(apply 'and (mapcar #'evenp '(2 4 6 8 9))) => nil

This is not a fold. Apply calls a function with arguments given in a list. A fold is done by reduce function, and in this case, since and is a macro, would look:

Code: Select all

(reduce #'(lambda (lv rv) (and lv rv)) (mapcar #'evenp '(2 4 5 8 9)))

Note that this is not short-circuiting, as mapcar will obviously evaluate the predicate on all arguments. It can be done trivially by iterating on the list:

Code: Select all

(dolist (e '(2 4 5 8 9) t)
  (unless (evenp e) (return  nil)))

If you really want to use a macro as a function you can always wrap it in lambda. It is different from eval in that the function is constructed at compile-time (or maybe even read-time?), not run-time. But this is rarely needed in practice, because for short-circuiting operators functional versions already exist (every, some, notevery, notany).

Which leads me to my question: is it considered acceptable style to use eval to do something like this? I typically shun the use of eval in other languages, because it's usually just a cheap way of getting something done which could be handled in more elegant ways, but with the data-is-code orientation of Lisp, is this a common tactic to employ?

In my (not very extensive, mind you) experience using eval is very rare, and it's use can be almost always replaced with something better.

I'd also be interested to see any alternate implementations of and;

I'm just making this up, so this is not even remotely efficient and may even not work, but:

Code: Select all

(defmacro my-and (&body body)
  (let ((step-fun (gensym))
        (thunk-list (gensym))
        (thunks (gensym)))
    `(let ((,thunk-list (list ,@(loop
                                   for b in body
                                   collect `(lambda ()
                                              ,b)))))
       (labels ((,step-fun (,thunks)
                  (cond ((null ,thunks)
                         t)
                        ((not (funcall (car ,thunks)))
                         nil)
                        (t (,step-fun (cdr ,thunks))))))
         (,step-fun ,thunk-list)))))

I thought it might be cool to have a macro that expanded into [using the example above] (and t t t t nil) but I can't seem to make it happen. My best guess was (...)

Expanding to and is not a good idea for the same reason apply isn't when used on arbitrary sequences, there is an argument number limit. In any case, evaluating the predicate at compile time would require the arguments to be known at compile time, and then why would you expand to (and t t t nil) at all, if you could just compute nil right there?

I hope that helped in some way.

[reuben.cornel]

I'd also be interested to see any alternate implementations of and;

Here's my two cents...

Code: Select all

(defun my-and(&rest bool-list)
  (null (remove t bool-list)))

[reuben.cornel]

Nice reply Ramarren, liked your method of explanation.

[taylor_venable]

I hope that helped in some way.

It did, and your explanation was enlightening. I had forgotten about argument length limits as well; in my mind I was mistakenly mapping (and x y z ...) [Lisp] to something like (x AND y AND z AND ...) [non-Lisp] and I see that in point of the fact that there are length limits, this would not be entirely the case, when the argument list is sufficiently long as to cause an error. But one question for clarification around that: since (and x y ...) just expands to (if x (and y ...) nil) then the limit of the argument list length would be hit in the function which does the expansion, right?

Thanks very much!

[ramarren]

Actually, I am not sure how this works really, but I think so. Macros are normal functions after all, they are just called by the compiler on the source tree... of course, in my SBCL:

Code: Select all

CALL-ARGUMENTS-LIMIT is an external symbol in #<PACKAGE "COMMON-LISP">.
It is a constant; its value is 536870911.

And quite possibly on 64 bit systems this would be even higher, so this is unlikely to be a problem in practice unless operating on really big datasets. Still, apply on arbitrary lists seems like bad style to me.

[theclapp]

Still, apply on arbitrary lists seems like bad style to me.

Agree. Especially since on some Lisps it's much lower. Lispworks for Linux Professional v5.2: 2047. Big enough for any hand-written function call you care to write (probably -- how masochistic are you? ), but too small to not worry about your APPLY arguments.

(I run into the shell version of this problem occasionally at work (AIX 5.2). The command line limit is fairly low (~20 kb). I like Linux, where it's on the order of 2 mb. Fortunately I'm not an idiot and am fully aware of xargs. )

I'm not sure if it's surprising or not that the Open Source tools have much higher limits than the commercial tools.

[metageek]

And quite possibly on 64 bit systems this would be even higher

Yes: it's 1152921504606846975 (still in SBCL). That's (2^60)-1, a little over 1 quintillion. I suppose a 64-bit machine could have enough memory to have that many arguments...but not this year.

(Let's see, assume each argument takes one cons cell and one integer, or 12 bytes, for a total of 12 exabytes of RAM. RAM is currently around $25/GB, so 12 EB would cost 300 billion dollars. Anybody who can spend that much on RAM can afford to have the compiler extended to handle longer argument lists. Or I suppose it could be virtual memory; with terabyte hard drives selling for around $200, 12 EB of disk would only cost 2.4 billion dollars...but it'd take over 100 years to read it all in over a 16x PCIe card. Again, anybody who can wait 100 years for an answer is not living within the same limitations I am.)

I'm not sure if it's surprising or not that the Open Source tools have much higher limits than the commercial tools.

I'd say not. Smaller limits can generally be implemented more efficiently, and commercial Common Lisp implementations have to distinguish themselves on performance.

[taylor_venable]

Which leads me to my question: is it considered acceptable style to use eval to do something like this? I typically shun the use of eval in other languages, because it's usually just a cheap way of getting something done which could be handled in more elegant ways, but with the data-is-code orientation of Lisp, is this a common tactic to employ?

In my (not very extensive, mind you) experience using eval is very rare, and it's use can be almost always replaced with something better.

I have another related question. Since macros don't evaluate their arguments, is there a way to get an argument to a macro to evaluate without using eval? The specific instance which makes me ask this is that I've been working with cl-who, a package that takes lists and turns them into SGML. There's a macro with-html-output-to-string which takes an argument like (:p "hello") and turns it into <p>hello</p>. But what I need is to evaluate some code, then pass the result to with-html-output-to-string, like so:

Code: Select all

(cl-who:with-html-output-to-string ... (car '((:p "hello")))) => "<p>hello</p>"

But since the argument isn't evaluated, cl-who doesn't know what to do with (car ...) so it outputs nothing. What technique is commonly employed to do this? By using eval I can write a macro like:

Code: Select all

(defmacro test (form)
  `(cl-who:with-html-output-to-string (*standard-output*) ,(eval form)))

This works, but along the lines of avoiding eval I wondered if there was another way to escape from macro non-evaluation?

[findinglisp]

This works, but along the lines of avoiding eval I wondered if there was another way to escape from macro non-evaluation?

Typically, there is an "escape symbol" of some sort that looks like a function call that tells the macro to insert that part of the form such that it will be evaluated in the macro-expansion. The macro must then process its parameters such that it looks for that tag and does the right thing. I think Peter Siebel had an example of that in Practical Common Lisp. See Chapter 31: Practical: An HTML Generation Library, the Compiler:
http://www.gigamonkeys.com/book/practic ... piler.html