How to turn function into a list?

Hello,
My nick is Methusala and I am a recovering Javaholic.

I would like to be able to turn a function into a list at the repl, so I can experiment with meta programming in the REPL. I know how to use a list like a function:

CL-USER> (setf g '(+ 2 2))
(+ 2 2)
CL-USER> g
(+ 2 2)
CL-USER> (eval g)
4

But how do you use a function like a list?
Here is my current best guess, which doesn't output a list, but does at least show the functions code:

CL-USER> (defun h (x) (+ x x))
H
CL-USER> (h 2)
4
CL-USER> (symbol-function 'h)
#<CLOSURE H (X) (DECLARE (SYSTEM::IN-DEFUN H)) (BLOCK H (+ X X))>
CL-USER> (car (symbol-function 'h))
-causes backtrace.
What should I use instead of symbol-function to get the above to return the car of a list?

Thanx

methusala wrote:Hello, My nick is Methusala and I am a recovering Javaholic.

We're here for ya, buddy!

how do you use a function like a list?

You don't. You define a macro that returns a function call* as a list, and then you call the macro. (see below)

CL-USER> (defun h (x) (+ x x))
H
CL-USER> (h 2)
4
CL-USER> (symbol-function 'h)
#<CLOSURE H (X) (DECLARE (SYSTEM::IN-DEFUN H)) (BLOCK H (+ X X))>
CL-USER> (car (symbol-function 'h))

Why do you want to do that? The function may be compiled, in which case it won't be a list.

What should I use instead of symbol-function to get the above to return the car of a list?

What are you trying to do? If you just want to explore that whole code-is-data thing, then what you want is a macro that encapsulates a piece of code. The macro returns a list that represents a function call*, and the call is automatically evaluated.

So for example, suppose you want to define synonyms for binary functions. This can do it:

(defmacro defop (name op) `(defun ,name (x y) (,op x y)))

DEFOP will return a list that contains a call to DEFUN. The backquote notation is common in macro definitions. It's equivalent to this:

(list 'defun name '(x y) (list op 'x 'y))

The list returned by the backquote is the return value of the macro DEFOP. Use MACROEXPAND-1 to see the list:

CL-USER> (macroexpand-1 '(defop plus +))
(DEFUN PLUS (X Y) (+ X Y))

When you call DEFOP, the list is automatically evaluated:

CL-USER> (defop plus +)
PLUS
CL-USER> (plus 1 2)
3

*or a call to another macro or a special operation.

Thanks Dan,
That helps alot. I started studying lisp on and off after working on a Java project that I realized would have been easier and better if the program could have changed some of it's code while running. I wanted to do that to reuse and dynamicaly modify web framework code based on user permissions. There was a tree of objects and a complex potential mix of user permissions. I tried to do that with bean shell and reflexion features but quickely found that was pretty much an impossible task. The Java language was clearly not meant for that.

Having browsed through 'Painters and Hackers' by Paul Graham I decided to try Lisp and picked up 'Ansi Common Lisp' by PG. I was excited to read in the first chapter about Lisp programming being 'real time.' What I took that to probably mean, was something like being able to use the REPL to:
-list all the functions loaded in the lisp image.
-print the code of any function loaded into the lisp image.
-change the code of any function loaded into the lisp image, including programmaticaly, for projects such as the one mentioned, and also for speedy development and maintainance.
-'compose' programs from loaded functions and run programs and functions at the REPL for fast tests.

Another motivating idea being that by doing this in a REPL in emacs, development would go at lightspeed because functions could be changed so easily and dynamically. For example one could print a function, ctrl-p up, changes lines in it and reload it. This sounded faster than switching between an editor with various open files, compiler, and a browser to view and test the results. Another part of this ideal environment I imagined, was to have a REPL for a running web app available at any time in which one could presumably check the state of current running code and data.

I experimented a little with weblocks to do this, but found less data than I had wanted. Ideally a debugger type display would be available showing what's happening with the program and data. A hurdle I had to deal with for attempting this, was dealing with threads in SBCL, since the REPL was busy while weblocks was running, and I thus had to run weblocks as a thread in order to still have access to the REPL. So doing this type of thing with threaded Lisp programs is another part of it, since a web app will likely end up having multiple threads.

Since I'm somewhat rambling on about my idealized development environment, I may as well add that it seems to me that Lisp should have some kind of built in replacement for SQL also. Having this would get rid of another time waster in web app development, having to switch back and forth to the mysql command line. I guess for functions or programs that use forms with mutiple fields, one may have to go outside the REPL since slime doesn't appear to support ncurses. I was kind of also hoping that Lisp would have the magical ability to uncompile Lisp made binaries, but I guess that's not a very logical thing to be wanting.

Cheers,
Methusala

methusala wrote:I may as well add that it seems to me that Lisp should have some kind of built in replacement for SQL also. Having this would get rid of another time waster in web app development, having to switch back and forth to the mysql command line.

If such a thing doesn't exist, you could surely tack together something that does the job and wraps up database queries and perhaps generates SQL statements, if you want. You could use closures or, since you've spent a long time on the Java railroad (me too) use CLOS to keep track of state.

methusala wrote:Thanks Dan,
Having browsed through 'Painters and Hackers' by Paul Graham I decided to try Lisp and picked up 'Ansi Common Lisp' by PG. I was excited to read in the first chapter about Lisp programming being 'real time.' What I took that to probably mean, was something like being able to use the REPL to:
-list all the functions loaded in the lisp image.

Lists all "worthy" functions:

(loop for symbol in (apropos-list "")
                  when (fboundp symbol) collect symbol)

-print the code of any function loaded into the lisp image.

SBCL seems to save the macroexpanded(?) source of functions with a high enough DEBUG setting.

-'compose' programs from loaded functions and run programs and functions at the REPL for fast tests.

Which is something you can already do.

Another motivating idea being that by doing this in a REPL in emacs, development would go at lightspeed because functions could be changed so easily and dynamically. For example one could print a function, ctrl-p up, changes lines in it and reload it. This sounded faster than switching between an editor with various open files, compiler, and a browser to view and test the results. Another part of this ideal environment I imagined, was to have a REPL for a running web app available at any time in which one could presumably check the state of current running code and data.

Sounds like the Meta-. feature in SLIME (locates a function's definition).

Lets not also forget the ED function. In CLISP, you can define a function foo at the REPL, then do (ed 'foo) and it will open the function up in your default editor and let you make changes.

Another facility are the functions 'describe', 'describe-object' and 'inspect'. You could parse their output to get the code for a function in list form. Performance concerns aside, these functions are implementation dependent and therefore not portable; so you probably wouldn't want to use them for programmatic runtime code analysis. However, they're handy for interactive programming and IDE integration (providing intellisense, etc.).

For example, in SBCL given (defun foo (a b) nil) I can do,

> (describe-object 'foo *standard-output*)

FOO is an internal symbol in #<PACKAGE "COMMON-LISP-USER">.
Function: #<FUNCTION FOO>
Its associated name (as in FUNCTION-LAMBDA-EXPRESSION) is FOO.
The function's arguments are: (A B)
Its defined argument types are:
(T T)
Its result type is:
(VALUES NULL &OPTIONAL)
On Mon, Oct 6, 2008 11:17:07 AM PDT it was compiled from:
(LAMBDA () (PROGN (SB-INT:NAMED-LAMBDA FOO (A B) (BLOCK FOO NIL))))

Thanks everyone,
This definetly helped clear up an area I was puzzling over. I can see that knowing the slime and sbcl documentation is important to getting the fast lisp development speed talked about in the books. I was surprised to see over 3000 functions loaded by default in a lispbox image. CLOS as a db stand in during development sounds great.

I also found some more information abut my questions. In the macro chapter of Ansi common lisp it's explained that using string list manipulation and eval to create macro like behavior, such as I suggested, is almost never used, because 1) doing it that way doesn't allow the use of lexical content, and 2) making eval compile or interpret a raw list is much slower than compiling it before hand.

In the CL reference appendix, I found a function that can change a function into a list of strings format:

(defun h(x) (* 2 x))
H
CL-USER> (function-lambda-expression 'h)
(LAMBDA (X) (DECLARE (SYSTEM::IN-DEFUN H)) (BLOCK H (* 2 X)))
#(NIL NIL NIL NIL ((DECLARATION OPTIMIZE DECLARATION)))
H
CL-USER> (list '(function-lambda-expression 'h))
((LAMBDA (X) (DECLARE (SYSTEM::IN-DEFUN H)) (BLOCK H (* 2 X))))

The appendix says the 2 values returned by function-lambda-expression after the lambda list are information about the lexical environment and the name of the function. Hyperspec says the second value is of type closure-p, apparently closure-p is also a vector type? I'm also not sure why the list function didn't try to append the second and third values to the list. I would guess this is a relatively esoteric function.

M

ps:
I mis-pasted a couple lines of lisp above, it shuld have read:

CL-USER> (list (function-lambda-expression 'h))
((LAMBDA (X) (DECLARE (SYSTEM::IN-DEFUN H)) (BLOCK H (* 2 X))))

Not:

CL-USER> (list '(function-lambda-expression 'h))
((LAMBDA (X) (DECLARE (SYSTEM::IN-DEFUN H)) (BLOCK H (* 2 X))))

methusala wrote: In the CL reference appendix, I found a function that can change a function into a list of strings format:

(defun h(x) (* 2 x))
H
CL-USER> (function-lambda-expression 'h)
(LAMBDA (X) (DECLARE (SYSTEM::IN-DEFUN H)) (BLOCK H (* 2 X)))
#(NIL NIL NIL NIL ((DECLARATION OPTIMIZE DECLARATION)))
H
CL-USER> (list '(function-lambda-expression 'h))
((LAMBDA (X) (DECLARE (SYSTEM::IN-DEFUN H)) (BLOCK H (* 2 X))))

From the Hyper Spec on FUNCTION-LAMBDA-EXPRESSION,
"Implementations are free to return ``nil, true, nil'' in all cases".

SBCL for instance returns (for H):
NIL
T
H

I'm also not sure why the list function didn't try to append the second and third values to the list. I would guess this is a relatively esoteric function.

Common Lisp supports multiple return values.

Take the function GETHASH for example, often you would just want to use the first returned value even though it returns more. If you don't use the other return values, they are discarded. You couldn't do that (as easily) if you returned a list of the values. This is what your are seeing with FUNCTION-LAMBDA-EXPRESSION; LIST is expecting only one value, so it uses the first value.

methusala wrote:In the macro chapter of Ansi common lisp it's explained that using string list manipulation and eval to create macro like behavior, such as I suggested, is almost never used, because 1) doing it that way doesn't allow the use of lexical content, and 2) making eval compile or interpret a raw list is much slower than compiling it before hand.

Right.

In the CL reference appendix, I found a function that can change a function into a list of strings format:

CL-USER> (function-lambda-expression 'h)

That's not something people usually do IRL. If you need to change the program's behavior at runtime, you probably want to generate some closures with LAMBDA.

methusala wrote: Since I'm somewhat rambling on about my idealized development environment, I may as well add that it seems to me that Lisp should have some kind of built in replacement for SQL also. Having this would get rid of another time waster in web app development, having to switch back and forth to the mysql command line.

Hi,

I'd say that for a lot of projects SQL is a tad overkill (people commonly use what they know). Consider having a look at http://common-lisp.net/project/elephant/ and http://common-lisp.net/project/rucksack/; persistence libraries. They're both relatively easy to use (elephant has quite a few dependencies though; rucksack is written in mostly portable CL). And they're both relatively fast. They don't, however, come with any pre-built GUI's or fancy command-line tools (that I know of).

My two cents.

methusala wrote:Hello,
My nick is Methusala and I am a recovering Javaholic.

I would like to be able to turn a function into a list at the repl, so I can experiment with meta programming in the REPL. I know how to use a list like a function:

CL-USER> (setf g '(+ 2 2))
(+ 2 2)
CL-USER> g
(+ 2 2)
CL-USER> (eval g)
4

But how do you use a function like a list?

No one actually answered your question,as you are probably asking how to do the wrong thing.
However, if you happen to be right about what you want to do, it is possible in portable common lisp.

You just have to create an alternative to defun that stores code as well as defining it, and use that instead.

(let ((functions (make-hash-table)))
	(defmacro redef (fn args &rest body)
		(setf (gethash fn functions) (cons args body))
		`(defun ,fn ,args ,@body))        
	(defun def (fn)
	 `(defun ,fn ,@(gethash fn functions))))
DEF
* (redef plus (a b) (+ a b))

PLUS
* (plus 1 2)

3
* (def 'plus)

(DEFUN PLUS (A B) (+ A B))
 * (fourth (def 'plus))

(+ A B)

Remember to rerun redef with the edited code to compile it again (You'd probably need to use eval ).