Only in Lisp

A friend of mine strongly recommends learning Lisp. It could actually be useful for my algebra program.
Now it takes a long time to learn a new language. I've read many comparisons, but most of them don't get to the point apart from mentioning performance issues. To me Lisp seems like a normal language like Python, with the only difference that Lisp is very hard to read.

So can someone enlighten me with a simple data handling example (no special syntactic tricks) which one cannot translate 1-to-1 to Python?

I can vaguely guess the Lisp syntax and all little programs I've seen seem to be convertable to any language. I would really like to see Lisp's advantage as concrete code.

Once you learn lisp's prefix notation, you will love its simplicity. Find an HP calculator buff to see how RPN lets them write simple tools to automate tasks. [Anecdote: For prob & stats homework, I spent a half hour writing RPN macros while my friend bashed keys on his TI. I then finished my homework in ten minutes, while he was less than halfway through.]

Same thing applies to prefix. The regular syntax makes it easy to write tools that rewrite your code.

Rewriting code is a common thing to do (its the core function of every compiler or interpreter); but most languages make it hard for the "normal programmer" to rewrite code, hiding the ASTs in deep internals.

Why would you want to write your own code-writing tools? Same reasons the compiler guys recommend you use a "high-level" language in the first place. Avoid boilerplate, write custom optimizers, automate patterns into "language features" (e.g. build looping operations out of conditionals and jumps), cross-platform abstractions, etc.

I don't know enough Python to clearly distinguish between its actual limitations and my shallow knowledge.

Would the following simple macros be easy to write in Python?

(defmacro computed-jump (value &body body)
  (append
   `(case ,value)
   (loop for i from 0
      for clause in body
      collecting (list i clause))))

;; Example:
(computed-jump (random 3)
  "hi" "bye" "what")
; expands at compile-time to
; (CASE (RANDOM 3) (0 "hi") (1 "bye") (2 "what"))
; at runtime, randomly evaluates one of the three strings (simply returns the value)

(defmacro one-of (&body body)
  `(computed-jump (random ,(length body)) ,@body))

;; Example:
(one-of "hi" "bye" "what")
; expands at compile-time to
; (COMPUTED-JUMP (RANDOM 3) "hi" "bye" "what")
; which expands as above

;; Example:
(one-of (print "hi") (* 3 4)) ; might print "hi" or return 7

BTW, powerful tools often seem clumsy for simple tasks. Hence the difficulty in making a compelling "toy example".

Gerenuk wrote:To me Lisp seems like a normal language like Python, with the only difference that Lisp is very hard to read.

I, for one, find Lisp code much easier to read than any other programming language, since it can usually be actually read, with punctuation serving similar role as punctuation in prose, rather than syntax being the primary determiner of meaning. This is obviously extremely subjective. Editing Lisp code is also much easier thanks to Emacs capability to semi-structurally edit nested symbolic expressions.

Of course the syntax allows effective macros, but since they are compiler customizations they are not that important if you don't care about having a compiler in the first place. Although the ability to add new control constructs at language-user level is still useful, for example: if someone does not like the built in language iteration construct, they can add a new one, and in fact there are at least two major ones: iterate and series. In Python or most anything except languages derived from either Lisp of Forth this would only be possible by language developers or front-end preprocessing.

And there are features unrelated to syntax. For example, Common Lisp Object System, especially combined with a metaobject protocol. Optional static typing. There is a list of some of those with explanations here.

Sure, the prefix notation is simple and universal. A mathematician would love the consistency. I just find it unnatural, so for me it doesn't help if Maths like the notation. I rather want to it to match with natural reasoning. But anyway, this is not the topic here (but I appreciate links to webpages with comparisons!)

Here I would only like to discuss written out examples. Is there anything about data treatment that makes Lisp superior (to another scripting language like Python)? I'm aware of the huge performance gap, but now I'm also interested in the methods.
Maybe even someone knows an advantage for my specific case: I want to store mathematical expressions (with + * and variables) as trees and manipulate them according to arithmetics.

In order to understand the difference, I tried to replicate the Lisp example in Python. It conceptually slightly different, but doesn't it work just as good?
Maybe someone can point out the differences.

from random import randrange

def computedJump(value, *commands):
  return commands[value]
  
def oneOf(*commands):
  return computedJump(randrange(len(commands)),*commands)

print computedJump(randrange(3),"hi","bye","what")   # result needs to be printed

def p():
  print "hi from p"

print oneOf(p,3*4)  # of course here one needs to decide beforehand whether you want to execute the function or use the result as data; you could do both but then you need 5 more lines to define a new class thats flexible

Btw, in Python you also have iterators and generators if that is what you mean by "iterate and series". Basically it's a function which when called "yield"s a return value and the next time it is called, it *continues* after the last yield statement and possibly loops again to yield the next value.

PS: so I know scripting languages are good, but why is anyone of them better than the other?

Gerenuk wrote:Is there anything about data treatment that makes Lisp superior (to another scripting language like Python)?

Common Lisp is not really a scripting language. That term is rather nebulous anyway, but many strengths of Common Lisp only become apparent with large programs, where you can build up the language to the problem domain. For one-off scripts a good standard library is more important, and since CL was standarized before huge "batteries-included" standard libraries became popular (not that it would have made it through ANSI comitee anyway) by today standards it is a very small language. Its reputation of being a huge language came from early nineties, when it was compared to languages of that time, especially Scheme.

Gerenuk wrote:Btw, in Python you also have iterators and generators if that is what you mean by "iterate and series". Basically it's a function which when called "yield"s a return value and the next time it is called, it *continues* after the last yield statement and possibly loops again to yield the next value.

But my point was that those things in Python had to be added by language developers, who had to modify the intepreter and so on. In Lisp those are userspace libraries written in pure Lisp.

Gerenuk wrote:Maybe someone can point out the differences.

You noted the difference in the comment: in Lisp you can just expand to needed code. In Python you would need a boilerplate to create a function object, and then pass this object around. The point of one-of is that only one branch is executed, which is important if they are either side-effecting or expensive. The crippled lambda and statement/expression division in Python alone are enough for me to strongly prefer Lisp over it.

Of course for small examples the gains are small, which is a problem with small examples. And bigger examples are incomprehensible without context.

Never mind this, but imo 'natural reasoning' doesn't really have too much to do with how you denote it, and how mathematicians denote things isn't always particularly natural. Edit: It's convenient on paper and such, but it remains just notation.

One feature that doesn't seem mentioned enough is special variables is a pretty awesome feature, allowing you to 'function-ize' a program, for one. Seems like a comparatively easy one to implement, weird that other languages don't seem to be interested in doing so. CLOS is probably a lot better than whatever Python has, but i must admit i don't know what Python has

One could make macros to make it look more like math, or even just a new filetype. Since lispers very often seem to use macros that are in a subset, with functions, this could probably save a bunch of parenthesis. The problem that it is too easy to do so, many lispers don't like it and one would probably want consensus about it.

Maybe a superset-of-Python notation-wise. Or ML-like notation. A potential problem with polish notation and CL is that CL has functions and variables in different namespaces, the reader can't always see the difference. But CL users are nearly always in the subset where it won't cause problems; they use *special-var*, +constant+, and macros attaching to values variables allow you to set the variable name. Slot values could be a bit of an issue, but a macro can be made for that too.

I think that is a good idea, it would help get people to CL and as they'd 'meet' macros they'd be lured into macros anyway, and even otherwise, Lambda and Funcall be strong.

Btw, about iterating/collecting there are rather many ways we do stuff like that.. There is libraries Iterate, Series, Loop(attached to CL), there is using (tail-)recursion, there is using higher order functions, there is using callbacks with accumulation macros.. Not many lispers use iterators as in objects you can move forward with a 'next' function and backward with a 'previous' function or anything like that.

Lisp makes you think about the problems in a different way. In general, when a Lisper wants to solve a problem, he tries to abstract and solve the entire kind of problem instead. It gets an amount of time to understand this.
There is one thing I managed to do which is either annoying or not easy to do in other languages. I wanted to store dynamically created functions into files, including closures. Functions are first-class values in lisp, they can be created by other functions, passed around in variables and called the way you want. Closures are functions with a free variable, like this:

(defun make-adder (n)
  (lambda (m)
    (+ m n)))

cl-user> (defvar func (make-adder 30))
func
cl-user> (funcall func 10)
40
cl-user> (funcall func 24)
54

Which are, by the way, very useful programming tools.

I managed to this by creating macros that are intended to substitute lambda and let (among others). The macros then "remember" the free variables and the code from functions, which are taken and stored into files whenever you ask to store a function.

Now, there are a couple of ways you would do it in, e.g., Python:

• Instead of creating the function you want to create, you would create a string with the code, which then is passed to another function that "remembers" it and sends the code to the interpreter so the interpreter would create the function, which is then returned.
• Create a small interpreter that revolves around the functions you want to use.

The second solution is obviously painful. The first get trickier as you want to associate the function with the free variables, and also remember which free variables are shared among which functions. It would be so strange to deal with such a thing that you probably would give up and find another solution to do what you want with your program.

In any case, like everything in the world, you might like Lisp or not. The only way to know is learning.

There is no such thing as "Only in X language". By default all programming languages are the same in the sense that they exist to solve problems, they're only different when you actually use them.

I've picked up an interest in lisp because it offers more tools than other languages to get away from scripting languages. And ways to do the same amount of "stuff" with less code(at least during development periods).

Ramarren wrote: But my point was that those things in Python had to be added by language developers, who had to modify the intepreter and so on. In Lisp those are userspace libraries written in pure Lisp.

That's absolutely true - before iterators/generators were introduced there was a really useful part missing. But OK: can you think of a feature that isn't implemented in Python yet?
Iterators and generators are, but beyond that there isn't much special. So is anything missing in Python?

Ramarren wrote: You noted the difference in the comment: in Lisp you can just expand to needed code. In Python you would need a boilerplate to create a function object, and then pass this object around. The point of one-of is that only one branch is executed, which is important if they are either side-effecting or expensive.

Hmm, I need to think about that. But true, it wasn't so straightforward for me to translate the Lisp example.

Ramarren wrote: The crippled lambda and statement/expression division in Python alone are enough for me to strongly prefer Lisp over it.

What's crippled? You mean you cannot put commands in the lambda? So far it seemed fine, because as soon as your lambda is so big as to include statements, you rather write a full function definition. "def f():" isn't that much to write.

Jasper wrote: One feature that doesn't seem mentioned enough is special variables is a pretty awesome feature, allowing you to 'function-ize' a program, for one.

I'd highly appreciate real, short examples in this thread. I hardly know enough Lisp to make up my own, but I can guess what example code means. Can you write out an example, which cannot easily be replicated with Python?

gugamilare wrote: Now, there are a couple of ways you would do it in, e.g., Python:

I'd say there is only one straightforward way.

def makeAdder(n): return lambda x:n+x

For more complex task you can use

def makeFunc(a):
  def Func(x):
    return x+a
  return Func

I never understood why it works and where it stores the variable "a", but it does work!

gugamilare wrote: In any case, like everything in the world, you might like Lisp or not. The only way to know is learning.

I have my own opinion about style and beauty of expressions.
In this thread I merely want to examine one particular facet, namely "what is example code, that shows functionality differences to python?".
As with the first Lisp example, I cannot fully translate the way to intermix code and data, but it seems I can write a program just as short and clear which has the same functionality.

Gerenuk wrote:

gugamilare wrote: Now, there are a couple of ways you would do it in, e.g., Python:

I'd say there is only one straightforward way.

Actually, you got the wrong problem. Creating the closure, ok, that is easy and has nothing to do with macros. The macros which are created are macros that make it possible to store a function into a file. In my particular case, I would have a program that generated code (using lists) that would be evaluated (compiled) and produce various functions. Those functions would be lost around objects (class objects), many would be substituted by other functions, ans so on. Then, at some point, I would like to be able to save everything into a file and close the application in such a way that I could continue computation later when opening the program again.

There my macros come into place. There is already many libraries that are able to store any kind of objects into files. I chose one that was extensible, cl-store, and extended it to allow storage of functions. How, you ask? Creating macros that would remember the code of the functions and variables in their creation, associating them with their code (in a hash-table) and free variables.

Gerenuk wrote: That's absolutely true - before iterators/generators were introduced there was a really useful part missing. But OK: can you think of a feature that isn't implemented in Python yet?
Iterators and generators are, but beyond that there isn't much special. So is anything missing in Python?

A MetaObject Protocol and... macros

I think you are viewing this the wrong way. Of course Python has every general-purpose tools available today. But, in Lisp, you can create not-so-general tools, specific for your problems. It is a common concept that libraries in Lisp are actually DSLs created for their specific purpose. So it is hard to say which tools Python does not have, except the meta-tools that allow you to create your own tools as you need them. Whether each specific problems need their own tool or not, that varies with the problems themselves.

Try reading ANSI Common Lisp from Paul Graham, it has a neat explanation about what is Lisp all about, The Art of MetaObject Protocol, that explains what is MOP and why it is so useful, or Practical Common Lisp, which is a gentle and yet deep introduction into Common Lisp.

This might suit your needs a bit: http://rosettacode.org/wiki/Playing_cards (note: lisp is lableled as "common lisp).

Pretty darn honest comparisons between languages. If I still had access to an iSeries server* I'd be tempted to add a COBOL example

*type was sever, very Freudian.

Gerenuk wrote:Iterators and generators are, but beyond that there isn't much special. So is anything missing in Python?

As gugamilare said, while most common patterns might have been already implemented, in Lisp you can use the same method to implement uncommon patterns strongly coupled to your problem domain. This doesn't appear that often if your problem domain is not very complex and very similar to what is already common.

Gerenuk wrote:What's crippled? You mean you cannot put commands in the lambda? So far it seemed fine, because as soon as your lambda is so big as to include statements, you rather write a full function definition. "def f():" isn't that much to write.

I rather wouldn't, because this, in my opinion, breaks the flow, especially since Python forces indentation, and introduces irrelevant names. There are also some issues with scope handling, which also affects more complicated closures. Not that, as I think about it, I know what those issues exactly are... but I find conflating setting and creating variables confusing anyway.

Gerenuk wrote:I'd highly appreciate real, short examples in this thread.

As has been said multiple times, for powerful features there aren't any examples which would be short and real at the same time. Special variables are, essentially, properly composable global variables. It is not that easy to use that in non-contrived way in anything short. I suggest that you spend some time learning basics of Lisp and then read some real code. Librariers by Edi Weitz are usually held up as examples of quality code.

In general, Lisp features which hadn't been already reimplemented in other languages (remember that Lisp is about fifty years old, and Common Lisp over fifteen) are most useful for longer programs in complex problem domains. Unless you consider the regular syntax as a major feature, which I do, but apparently most people don't. Also, there is no magic. Remember that whatever the program is written in, they are all executed on the same hardware, so there is not anything which some language can do which other cannot.

Maybe one important question which comes to my mind:
What is an example for a software project which is based on Lisp programming code *exclusively*?

gugamilare wrote: Actually, you got the wrong problem. Creating the closure, ok, that is easy and has nothing to do with macros. The macros which are created are macros that make it possible to store a function into a file.

Well OK. In that case I really would need to define a new class.

class makeAdder:
  def __init__(self,n):
    self.n=n
  def __call__(self,x):
     return self.n+x
a=makeAdder(10)
print a(20)

Any object can be stored with "pickle" even though there might be some caveats necessary.
Just my impression is that all these alternative ways to write code all at least as good as macros for all practical reasons. Of course I haven't programmed any Lisp, but that's why I'm asking for code examples here to convince myself.

gugamilare wrote: I think you are viewing this the wrong way. Of course Python has every general-purpose tools available today. But, in Lisp, you can create not-so-general tools, specific for your problems. It is a common concept that libraries in Lisp are actually DSLs created for their specific purpose. So it is hard to say which tools Python does not have, except the meta-tools that allow you to create your own tools as you need them.

The "wrong way" argument doesn't convince me if for all practical reasons clean alternatives can be found just work just as well.
Of course I don't want to talk about "tools" related to libraries, because in any language you can add the missing libraries yourself. And actually that's what's the thread for: an example from your experience as programmers for the surprising "not-so-general tools" for a specific problem.

gugamilare wrote: Try reading ANSI Common Lisp from Paul Graham, it has a neat explanation about what is Lisp all about

Actually this page my friend was refering me to. I read a couple of paragraphs and got the impression he is a fanatic who lost the ability to judge freely. His essays are long, with many repeats, mostly vague and he rarely refers to real facts which could convince me. He makes many manipulative reinterpretations of concepts as to suit his own needs.
So I tried googling other comparisons, because I cannot trust Paul a word since he seems so much polarized.

gugamilare wrote: The Art of MetaObject Protocol, that explains what is MOP and why it is so useful, or Practical Common Lisp, which is a gentle and yet deep introduction into Common Lisp.

I'll read that. It could be a good answer to my question... Maybe...

lithos wrote:This might suit your needs a bit: http://rosettacode.org/wiki/Playing_cards (note: lisp is lableled as "common lisp).

That's a great page and I had already looked at some example. However Lisp does not beat Python for length in the examples.

Ramarren wrote:As gugamilare said, while most common patterns might have been already implemented, in Lisp you can use the same method to implement uncommon patterns strongly coupled to your problem domain. This doesn't appear that often if your problem domain is not very complex and very similar to what is already common.

So can you describe an example with a more complex problem?

Ramarren wrote: Remember that whatever the program is written in, they are all executed on the same hardware, so there is not anything which some language can do which other cannot.

Oh sure. But I'm thinking about speed of software development. Lines of code are probably similar to python. So the last factor is code readability and natural problem solutions. I think most people agree that purely readability is better in python.
And here I was searching for code examples which solve problem in a way which is more natural to human thinking than equivalent solutions in python.

Gerenuk wrote:What is an example for a software project which is based on Lisp programming code *exclusively*?

Gerenuk wrote:So can you describe an example with a more complex problem?

I am not sure what exactly are you asking, but some examples of large software open-source projects written in Lisp are: ACL2 (which, among other things, had been used for formal verification of processors by AMD) and Maxima. There are also quite many proprietary application using commercial implementations, this page lists some using Allegro Common Lisp. Generally applications from the domain formerly known as Artificial Intelligence are usually where Lisp is useful.

Gerenuk wrote:And here I was searching for code examples which solve problem in a way which is more natural to human thinking than equivalent solutions in python.

The problem is since you obviously already know Python, and you internalized it, solutions in Python appear "natural" to you. There is nothing natural about programming, since the necessity of this mode of thinking did not exist in natural environment. Therefore any differences between languages which are relatively similar in expressiveness are swamped by preexisting knowledge. So what you want to be shown is impossible for you to see. You have to learn Lisp at least as well as you know Python, and then decide which do you want to use.

Or if you are just looking for an excuse for your friend to not learn Lisp, then: no, there is no magic and no silver bullet and Lisp will not write your program for you. If you don't care about performance, syntax and those features listed at the page I linked some time ago, then Lisp is just another rather nice language, but not obviously superior to Python.

Gerenuk wrote: As with the first Lisp example, I cannot fully translate the way to intermix code and data, but it seems I can write a program just as short and clear which has the same functionality.

That's true when comparing any reasonably high-level turing- complete language with another thus classifiable language.
Different idioms that result in the same result.

Ramarren wrote: The problem is since you obviously already know Python, and you internalized it, solutions in Python appear "natural" to you. There is nothing natural about programming, since the necessity of this mode of thinking did not exist in natural environment.

Oh there is. You should try http://www.muppetlabs.com/~breadbox/bf/
And end, to me it only matters in which language I have the most success. So I won't start something completely new if the success per learning time ratio is just the same as for other languages.

Ramarren wrote: You have to learn Lisp at least as well as you know Python, and then decide which do you want to use.

Not necessarily. If my hypothesis that almost all languages are equivalently comfortable seems plausible, then I can judge by something superficial like the looks of the code.

Ramarren wrote: Or if you are just looking for an excuse for your friend to not learn Lisp, then: no, there is no magic and no silver bullet and Lisp will not write your program for you.

It's not about excuses. He doesn't know Lisp either. It's rather that on the Internet I often read "Lisp actually has the silver bullet and it's a shame that no-one sees it. But I cannot tell you what it is"
This reminded me of religions which try to save their view by shouting out claims, but disallowing investigations.

Ramarren wrote: If you don't care about performance, syntax and those features listed at the page I linked some time ago, then Lisp is just another rather nice language, but not obviously superior to Python.

That's a useful statement. So I will look into the mentioned features. I hoped to get a first impression from some example here.
Macros don't convince me so far. Maybe the Metaobject scheme will. In my view the syntax is rather a disadvantage. Storing bits 0 or 1 only is even a more "simple" scheme, yet it doesn't mean it's more convenient to think this way. So simplicity or universality is not an arguement.

TheGZeus wrote: That's true when comparing any reasonably high-level turing- complete language with another thus classifiable language.
Different idioms that result in the same result.

I mean not theoretically, but considering that it should be easy to program. It's not hard to show why Python is much more convenient than C for programming.

Gerenuk wrote:Oh there is. You should try http://www.muppetlabs.com/~breadbox/bf/

Just because two things are both not natural doesn't mean they are equally hard or equally easy. I am not saying that two languages cannot handle complexity better or worse, but it still doesn't have that much to do with natural human thinking. Although I guess there are different capacities for unnatural thinking... Also, human brains are quite variable. In fact I think affinity for Lisp syntax must be at least partially neurologically determined... something about syntactic and semantic reasoning.

Gerenuk wrote:It's rather that on the Internet I often read "Lisp actually has the silver bullet and it's a shame that no-one sees it. But I cannot tell you what it is"

Well, most of that is inertia. Lisp is somewhat of a silver bullet when compared to Fortran, C and other Algol-derived languages, like Java. At the time Common Lisp was standardized by ANSI (1994) it was way ahead of most widely used languages. Most of that "silver-bulletness" has already been acquired since then by languages like Python or Ruby. The primary objective advantage of Lisp remaining is that compilation has been mostly figured out, unlike for Python where it is still at best experimental, or Ruby which I believe cannot be usefully compiled at all due to its semantics.

Most of the rest is either personal preference, or features that are hard to explain. Macros especially, since any simple macro is easy to translate by hand, and then of course it appears as it would be easier just to write out the translation. And it is not as writing macros happens that often, I think the most complex I have written was this. Or maybe the formula unit verification thing.

Although through all this it might be worth noting, I am not a professional programmer, so most of my programming is more-or-less recreational, and so my opinions, not to mention code, might not be representative for serious Lisp programmers.

Gerenuk wrote:He doesn't know Lisp either.

Why would someone advise doing something which they themselves have not done, or learning something they do not know?

Ramarren wrote: Most of the rest is either personal preference, or features that are hard to explain. Macros especially, since any simple macro is easy to translate by hand, and then of course it appears as it would be easier just to write out the translation. And it is not as writing macros happens that often, I think the most complex I have written was this. Or maybe the formula unit verification thing.

Thanks for the references. I noticed I can best examine the difference by trying to replicate code in Python. Can you recommend a specific other high-quality macro I could attempt? (as I don't know much Lisp it's hard for me to judge suitable macros myself)

Ramarren wrote: Why would someone advise doing something which they themselves have not done, or learning something they do not know?

He has read Paul Grahams essays, and believes every word of it. He says if he wanted to start learning a comfortable language, he'd chose Lisp.

Gerenuk wrote:

gugamilare wrote: Try reading ANSI Common Lisp from Paul Graham, it has a neat explanation about what is Lisp all about

Actually this page my friend was refering me to. I read a couple of paragraphs and got the impression he is a fanatic who lost the ability to judge freely. His essays are long, with many repeats, mostly vague and he rarely refers to real facts which could convince me. He makes many manipulative reinterpretations of concepts as to suit his own needs..
So I tried googling other comparisons, because I cannot trust Paul a word since he seems so much polarized.

Actually, Paul Graham is one of the most successful Lispers around. He is quite a good programmer, mind you, much better than anyone here in this forum. If you decide to read more than two paragraphs of the book, you will also see that he has a great ability to abstract and simplify problems. He also made a couple of articles explaining simple yet clever ideas in algorithms. So, don't think he is just a random "mindless fanatic". But of course he is fanatic. Everyone who is very good at something is fanatic.

Gerenuk wrote:

gugamilare wrote: Actually, you got the wrong problem. Creating the closure, ok, that is easy and has nothing to do with macros. The macros which are created are macros that make it possible to store a function into a file.

Well OK. In that case I really would need to define a new class.[...]

Ok, that would work, but it is a workaround. Substituting functions with classes is a very known method to avoid "needing" functions or closures, but they are not as flexible. You would be forcing users of your library create classes and methods instead of functions if they want to store them in a file. You might be satisfied with this, but I wouldn't. Forcing you to do workarounds is very common in other languages, but AFAIK they are quite rare in CL.

Just to summarize, I wouldn't be able to find a small example where Lisp supersedes Python greatly. Python developers make sure that doesn't happen. If I show a small Lisp program, you add some bits of code here, other there and transport that program to Python in a satisfactory way. However, both Lisp and Python have their strong and weak points which tend to be more and more apparent as your programs grows in size and complexity. Lisp is very good in general when your problem or idea to solve it is complex, abstract or goes far from common approaches. It is defective in modern tools like multiple threading, sockets and so on.

Gerenuk wrote:Can you recommend a specific other high-quality macro I could attempt? (as I don't know much Lisp it's hard for me to judge suitable macros myself)

To be clear, I don't claim my code to be especially high quality. I would suggest cl-cont, but I guess that would be cheating. Or parenscript. Also cl-unification is an interesting application of macros.

Although do note that as I have written before, macros are compiler extensions. If you do not have a compiler, then by definition you cannot have macros and are not doing what a macro is doing. This might not matter for pure expressiveness, as usually you can just write an interpreter for some data language with fairly minimal additional markers, but it does affect both time and space performance.

A complete unit test framework capable of accepting any type of code and testing it compared to any type of result and with well formatted output. Which isn't bad for 16 lines of code(without comments), if you printed it out a piece of paper would feel almost naked. You even get the full walk-through for how the code was developed here: http://www.gigamonkeys.com/book/practic ... ework.html, Practical Common Lisp of course.

(defvar *test-name* nil)

(defmacro deftest (name parameters &body body)
  "Define a test function. Within a test function we can call
   other test functions or use 'check' to run individual test
   cases."
  `(defun ,name ,parameters
    (let ((*test-name* (append *test-name* (list ',name))))
      ,@body)))

(defmacro check (&body forms)
  "Run each expression in 'forms' as a test case."
  `(combine-results
    ,@(loop for f in forms collect `(report-result ,f ',f))))

(defmacro combine-results (&body forms)
  "Combine the results (as booleans) of evaluating 'forms' in order."
  (with-gensyms (result)
    `(let ((,result t))
      ,@(loop for f in forms collect `(unless ,f (setf ,result nil)))
      ,result)))

(defun report-result (result form)
  "Report the results of a single test case. Called by 'check'."
  (format t "~:[FAIL~;pass~] ... ~a: ~a~%" result *test-name* form)
  result)

Special vars:

(let ((*special-variable* something else))
  (any-program-using-that)) ;Feeding arguments to a program by non-destructive changing of its variables.

You can look at it as adding then as arguments everywhere. Also useful if you have a recursive function and values are passed unchanged a lot.

The more i head into functions using and returning functions, the less i need macros.(So the more suitable macroless languages seem) However macros like with-slots will remain a requisite for me, and will likely still want to make my own macros like that. Anyway, how do you access elements of a structure in python? In C++ the member functions approach is silly,(and can only do one object at a time) and object.slot all the time is cumbersome.(and the '.' notation not particularly natural, but whatever)

Does python have &keyword arguments, &rest arguments? Know a good link summarizing pythons features thoroughly? For people learning CL i usually link to PCL. (Though that is a learning book, not really a feature summary.)

Well, i think we have forgotten one thing in our goal to 'convert' Gerenuk.
In LISP it is very simple to debug your code. You can visit your code line by line, make changes on the values (if you want) or inspect symbols and go on. You have no overhead with debugging-tools or breakpoints and such things.

Ben wrote:Well, i think we have forgotten one thing in our goal to 'convert' Gerenuk.
In LISP it is very simple to debug your code. You can visit your code line by line, make changes on the values (if you want) or inspect symbols and go on. You have no overhead with debugging-tools or breakpoints and such things.

If we go down on that route, there are also many SLIME's feature that could be mentioned. Like the ability to inspect values, which is very useful to visualize the object you are dealing with. Or the ability to copy-by-reference: just point to a printed representation of a value in the REPL and you can use the value (unless already garbage collected, of course). Consistency: in Python, AFAIK, you can't copy files' contents to the REPL directly, you have to copy function by function or something. In Lisp you don't have to worry about it. Other features Python might also have, like finding the source of a function, checking documentation of a function or variable...

I wonder about the feasability of creating a python to lisp compiler. Reuse all of that wonder python code. especially since all features of python are available in lisp.

Suroy wrote:I wonder about the feasability of creating a python to lisp compiler. Reuse all of that wonder python code. especially since all features of python are available in lisp.

You mean cl-python? There is also python-on-lisp, which communicates with Python rather than recompiling Python.

Oh, didn't see that one. So it compiles python to lisp? Nice

Edit: Any catch? I would think with all those python libraries out there, compiling them to lisp and using them would be really advantageous.

Suroy wrote:Any catch? I would think with all those python libraries out there, compiling them to lisp and using them would be really advantageous.

I talked to the author at ILC09. The two catches were (1) he wasn't tracking the latest additions to python and (2) it didn't work with python libraries which were wrappers over a C/C++ library. Other than that, it sounded rather full featured and fast.

This page has some languages in lisp. Would be awesome if as asdf gets fixed, then, with an extension, we probably could add Python code and other languages just by adding the files to the defsystem, if this C/C++ library thing gets fixed.

Gerenuk wrote: I'd highly appreciate real, short examples in this thread. I hardly know enough Lisp to make up my own, but I can guess what example code means. Can you write out an example, which cannot easily be replicated with Python?

I'm learning Lisp at the moment and use Python fairly regularly but am certainly not an expert. This question is for my own edification as much as anything. Before Python introduced the "with" statement in 2.5, could you have implemented "with" yourself with the same level of integration into the language?

with open("hello.txt") as f:
    for line in f:
        ---arbitrary code in here----
        print line

In Lisp, this kind of thing is easy.

Heres an example of something in lisp you cant do in python.
Lets say you want to abstract this pattern away (ok, so this is more in common with java language, dont know python):
if (item.equals("HI")) {

} else if (item.equals("HHH")) {

} else if (item.equals("BB")) {

}

etc. Same function, 'equals'

In lisp, it would be
(cond
((equal item "HHH") ...)
((equal item "BB") ...)
)
etc.


New way to do it:
(condp equal item
("HHH" ...)
("BB" ...))

Boom! Bet u cant do that in python

(defmacro condp (func item &rest args)
  (let ((gen-item (gensym)))
    `(let ((,gen-item ,item))
       (cond
	 ,@(loop
	      for arg in args
	      collecting `((,func ,gen-item ,(car arg))
			   ,@(cdr arg)))))))

Think of it this way. Lets say we stripped do,while,and for loops from your language and then i just told you you could rewrite it using gotos. But look, you say, i can just write do loops using this idiom of goto statement, for loops using this idiom, .... But if you had lisp without those loops and just gotos, you could just write it in a macro which expands into that 'repetitive idiom.' and thus create your own loops! After all, we should abstract away all common code. The key thing is that macros do not evaluate their arguments, like functions, so anything which shouldn't evaluate their arguments are not possible using functions (thus not possible in python).

Also to note is the syntax. How much harder would it be to create macros which produce code if we were using anything but parantheses? i think it would be much harder which explains why lisp is unique in this macro aspect.

For some reason Python doesn't allow the conditionals to be returned either.. if ... : ... else ... must be toplevel because the indentation requirement at least.

Seems to me like a completely arbitrary restriction. Even C has ... ? ... : ...

Jasper wrote:For some reason Python doesn't allow the conditionals to be returned either.. if ... : ... else ... must be toplevel because the indentation requirement at least.

Seems to me like a completely arbitrary restriction. Even C has ... ? ... : ...

Python has a new ternary operator (if this is what you mean):
true-clause if testclaus else false-clause

I learned Lisp for 4 weeks and then switched to Python.
This is a fairly long comparison.

Syntax:

Lisp syntax is very regular. It is basically a repetition of
open_parens_( function_name/macro_name arg0_name arg1_name ... )_closing_parens
Each argument is either a direct variable or another subexpression which starts with a opening parens.

Python has a more complicated "construct-tree". First it devides in compound statements that execute a body,
like
if (cond):
body
and in simple statements like
a = 2
myfunc (2,4)
raise ValueError

The statements are not regular. While the function calls use a prefix style like lisp, mathematical operations and assignments use a infix style.

So which is actually better?
Lisp is regular and systematic, therefore requiring very little time to learn. Macros are only possible (/useful) if you have direct access to the syntax tree and are therefore hard to do in python.

Python is often more intuitive, because real life math is using infix notation. Python code makes program structure easily visible, while in Lisp you always have to parse the program in your head.
Consider this example:
for x in range (10):
print ("\n")
for y in range (10):
print (my_array[x][y])

(dotimes (x 10) (format t "\n") (dotimes (y 10) (format t (aref my_array x y))))

The Python code makes the structure of the program clear: The print array command is inside the double loop and a new line is printed in the outer loop. While experienced Lisp users may read their code easily I needed to count the parenthesis and mentally model the parse tree which I found daunting. YMMW

Macros:

Macros are to this day still unique to lisp (there are languages which employ macros, but there are either simple textreplacing macros like in C
or in languages that are even more underground than Lisp).

In Paul Grahams book "On Lisp" he desribes 3 things macros do that are impossible with functions:
1) Conditional Evaluation
2) Lexical Context
3) Setf Accessor
I might add
4) Compiler macros

Compiler macros are a good way to shift runtme.-computations to compile-time computations and therefore increase speed.
Setf-accessors are important for Lisp but Python provides the variable = expression assignment and good control over objects with the magic __setattr__ methods.

That leaves the main point about macros: Conditional evaluation and lexical context. The main thing to remember about macros is that they can execute a (code) body. In Python you have to use the builtin compound statements like if, for and with. In Lisp you can create new control flow instruments that execute a given body n-times, depending on condition y and other magic stuff. Take the "conditional jump" example earlier this thread. It is easy for python to keep up when only short expression are involved.
But lets construct a more detailed example: You are given a list of points which represent geometrical figures. If the list include only 2 points, it is a line, if it includes 3 point it represents a triangle and four point represent a rectangle. You want to write a function that calculates the area of these figures. In Python you have to use the "native" control structures

def area (*args): # &rest arg in Lisp
    if (len(args) == 2):
        return (0)
    if (len (args) == 3):
        return (0.5 * dist (args[0], args[1]) * calc_height (args[0], args[1], args[2]))
 ......

In lisp you can make a specific control structure that takes a list argument and 3 different code bodies and completly abstracts the python if... if.. clauses.

(my_macro input 0 (* 0.5 (dist (first input) (second input)) (height_calc (first input) (second input) (third input))) rectangle_calc)

Another thing are the lexical enviroment macros, which bind certain variables while the code body is evaluated. I think this is easily replicated in Python by using a function that takes a object and returns a dictionary with certain properties of the object. It is useful if combined with the conditional evaluations macros and can create for example an anaphoric-if.

The big question is "Is it worth it?". My answer is no, I choose Python over Lisp. Macros arent compelling enough to make a difference for me, because they are difficult to write and I dont need additional control structures beyond a well designed core. You often hear the following arguments from lisp coders: "What if they take away your if and with statement tomorrow and you only have AND and OR left? What would you do then without macros?". That is a pointless argument. Because python has no possibilities to create basic structures from scratch it ships with the (well rounded) builtins.

I never use dotimes. I like to use iterate

for x in range (10):
print ("\n")
for y in range (10):
print (my_array[x][y])

equal in lisp

(iter (for x to 10) (print "\\n")
(iter (for y to 10) (print (aref my-array x y))))

. And i disagree with your usage of macros. The point of macros is to substitute patterns. Why would you rewrite that 'if' code as a macro? Is it common to test if the length of something is equal to 2 or if it is equal to 3?

At least have the decency to list the lisp code as it would be indented by any decent editor.

(dotimes (x 10)
  (format t "\n")
  (dotimes (y 10)
    (format t (aref my_array x y))))

You showed the python code on four lines, with the lisp on 1. When lisp is on 4, I would argue it is as least as clear as the python. No paren counting is required -- editors do that easily, and its not particularly important in hand-written pseudocode.

I also believe a properly-formatted polynomials are clearer in lisp than in "algebraic" languages (*). Here's a crude example; it carries more weight when the coefficients are large expressions rather than single terms.

(+ (* (expt x 3) a)
   (* (expt x 2) b)
   (* x c)
   d))

(*) These languages don't even allow for proper 2D math layout, with superscripts, subscripts, different division notations, etc. They require learning how to convert a graphical language into something less than ASCII art.

This thread is getting old, but I figured I'd chip in. You seem like a knowledgable fellow, so I'll get right to the good stuff.

• Dynamic Variables: This is like a global variable, but bound with let. All subsequent functions see the most-recently-bound value. After the let the variable holds its old value. I use them to record the current HTTP request, current DB socket, etc.
• Few Reserved Characters: I have function names with !, @, /, etc. My names are smaller because I don't have to write '-set', '-access', or '-varient'.
• Macros.
  • With-open-file: it looks like python borrowed this from Lisp, which is good because this is so useful.
  • Continuatons: Paul Graham has some simple macros to simulate these in his book 'On Lisp', which is all about things you can ONLY do in Lisp.
  • Generalized setf: setf is a macro that looks at the expression its setting and compiles to the correct setter. Thats why (setf (gethash :n *ht*) new-value) works.
  • Compile time computation: Macros can run lisp code at compile time. I've used macros to embed CSS and images directly into my program. This way I avoid the OS+file roundtrip for commonly requested files.
• unwind-protect: Specify a cleanup form that will be run when the block exits. It'll run if you leave, or error out. Used to ensure connections and files get closed, macros use it a lot.
• Signal Handling: Lisp error handling is a low level stack unwinder/message passing system, with the high level error trapping built from macros. You can do cool things with it. And the debugger is top-notch.
• Lisp tries to be functional. Most Lisp primitives are functional which makes the whole thing a lot easier to use.

Infix math feels natural because its been taught over 12 years of school. BEDMAS (Brackets, Exponents, Division, Multiplication, Addition, Subtraction) is not natural.

Although this thread is pretty old as mentioned, I want to post a pointer to an excellent blog post describing various features of Common Lisp quite well, http://abhishek.geek.nz/docs/features-of-common-lisp. It is a really good resource to refer beginners to, I wish more people would use it instead of coming up with weak arguments over and over again. It's left to the reader to determine whether this features are unique or not, but be advised that the beauty lies in combining them. You can not tell how good a language is just by hearing about it, only by learning and using it can you be a good judge, as Peter Norvig kind-of points out in his famous article.

Ramarren wrote:The crippled lambda and statement/expression division in Python alone are enough for me to strongly prefer Lisp over it.

This is one reason that I prefer Ruby over Python. Every time I start writing Python, I get bitten by the fact that statements don't return values and can't be used in expressions. I LOVE the fact that in Lisp and Ruby everything is an expression and returns a value. Sometimes the value is not very useful, but it makes it easy to write code anyplace and have it make sense. With Python, I find myself reworking code to convert expressions to statements or vice-versa.

Can this be done in Python

findinglisp wrote:

Ramarren wrote:The crippled lambda and statement/expression division in Python alone are enough for me to strongly prefer Lisp over it.

This is one reason that I prefer Ruby over Python. Every time I start writing Python, I get bitten by the fact that statements don't return values and can't be used in expressions. I LOVE the fact that in Lisp and Ruby everything is an expression and returns a value. Sometimes the value is not very useful, but it makes it easy to write code anyplace and have it make sense. With Python, I find myself reworking code to convert expressions to statements or vice-versa.

I also love that in Lisp. I find so odd when you have to explicitly create a variable and assign different values to it. Incredibly people in general find that awkward and end up creating a lot of variables in Lisp (not to mention they use SETF instead of LET). That is actually awkward.

I think it might be the greatest reason for people avoiding to learn Lisp. They make it look like imperative language, like they are used to, which doesn't work well and they just give up.

August wrote:

Gerenuk wrote: I'd highly appreciate real, short examples in this thread. I hardly know enough Lisp to make up my own, but I can guess what example code means. Can you write out an example, which cannot easily be replicated with Python?

I'm learning Lisp at the moment and use Python fairly regularly but am certainly not an expert. This question is for my own edification as much as anything. Before Python introduced the "with" statement in 2.5, could you have implemented "with" yourself with the same level of integration into the language?

with open("hello.txt") as f:
    for line in f:
        ---arbitrary code in here----
        print line

In Lisp, this kind of thing is easy.

This is a really important point, and it's one that is easily lost. There is something bigger at work here than just macros (said in an awesome voice).

Lisp, you see, is self-contained and very, very, very evolvable in ways that other languages aren't. The "core" of Lisp, it's special forms, is insanely small. The rules for interpreting Lisp code fit in a few lines (I actually have a coffee cup with the original McCarthy interpreter printed on its side). Everything else is essentially a library routine or macro on top of that core. A full-featured Lisp, like Common Lisp, has slightly more infrastructure, but even that is built on that insanely small core.

Essentially, a Common Lisp implementation works by translating ASCII program text into data structures (sexprs). That step is customizable and a programmer can, using reader macros, change the way the Lisp READ function creates the data structures. This potentially allows you great freedom in customizing the surface syntax that Lisp offers. In practice, most people don't go that far and find that the basic sexpr syntax works just fine. Then, just before the interpreter or compiler gets a crack at the sexpr, it is subject to macro expansion (standard macros, symbol macros, etc.) that again allows potentially radical transformation of the code. Finally, the code gets interpreted/compiled/executed.

It's the small, almost trivially simple core, couple with a flexible set of processing steps that allow radical transformation of the data structures that represent the program code that makes Lisp so flexible. And, of course, since these data structures can be constructed on the fly (code is data is code) it's very easy to write code that writes code.

To me, those things are what Lisp has that no other programming language can really match. The only other language in the same league is Forth. If you look at Lisp and barf because of the parenthesis, then you simply don't get it (yet). I know that I didn't get it for a long time, either. But once you understand it, you never look at another programming language the same. I should also point out that if you look at Forth and just see RPN, you also don't get it.