Literal Lists

I've been reading a little bit in LOL. Please look at this:

;;; From LOL, chapter 4, page 80.

(defvar to-splice '(B C D))

`(a ,.to-splice e)

to-splice ; => (B C D E)

Here apparently a literal list is being modfied and we all know that the results are undefined. Either this is a bad example or I'm completely missing something here.

I would think a literal list being modified would be defined as 'the literal list gets modified' =).

Must be something to do with the ,. operator... how does the ,@ operator fare?

Well sure the ,@ is non-destructive and conses a new list.

I was just wondering that the book shows this example without saying anything about it's undefined effects. (Well it does so in the next example which is similar.) As somebody who's new to LISP thinks then he might have misunderstood something somewhere...

Mostly "LOL" is related to Land of Lisp

edit: the following refers to Let over Lambda
And yes, there is a warning in the book with the example calling this function twice:
(defun dangerous-use-of-bq ()
`(a ,.'(b c d) e))

I gather you're talking about Let over Lambda? You're right, the consequences of modifying literals are undefined (this is explicitly noted in the CLHS), and I think it's wrong not to note this somewhere in the text.

Overall, I have mixed feelings about the book. On the one hand, it's interesting and unique, but on the other hand, it contains a couple of questionable design decisions and tends to ignore subtle issues that you ought to be aware of as a Lisp programmer. Modification of literals is one. Another is this talk about “duality of syntax” being a reason not to lexically distinguish special variables from lexical ones, which is problematic from an engineering point of view, as noted in numerous texts on Lisp style. (And in any case, duality of syntax is preserved by using earmuffs, so I don't really get the reasoning anyway.)

That said, I haven't yet worked through the book. Perhaps the issues are addressed somewhere rather than dismissed. Also, its virtues might well be much greater than its flaws. It is an intriguing piece of work, after all.

Yes it is Let Over Lambda.

FAU wrote:I've been reading a little bit in LOL. Please look at this:

;;; From LOL, chapter 4, page 80.

(defvar to-splice '(B C D))

`(a ,.to-splice e)

to-splice ; => (B C D E)

Here apparently a literal list is being modfied and we all know that the results are undefined. Either this is a bad example or I'm completely missing something here.

[I know this is old, but...]

Presumably you're supposed to be typing this at the REPL, so there is no problem (the results are perfectly well defined as long as you don't file-compile it). In some sense it's a "bad example", in that you don't want to teach bad habits...but on the other hand, I think it's better to teach people to understand how things really work, rather than just "cargo cult" programming (and get people saying things like "we all know the results are undefined" when you do this kind of thing...which is not actually true), anyway, so "bad examples" can be useful/good.

Paul wrote:Presumably you're supposed to be typing this at the REPL, so there is no problem (the results are perfectly well defined as long as you don't file-compile it).

Well...

“The consequences are undefined if literal objects (including quoted objects) are destructively modified.” (CLHS, Special Operator QUOTE)

What you're probably thinking of is the fact that literals may be coalesced only when doing file compilation. However, literals may never be destructively modified, regardless of whether or not they have been coalesced.

In some sense it's a "bad example", in that you don't want to teach bad habits...but on the other hand, I think it's better to teach people to understand how things really work [...]

I agree with the general point you're making, but in this particular case, understanding how things really work includes understanding that there are important differences between a quoted list and a list constructed by user code, the former, after all, being a part of the program's code. (This is actually a pretty significant point, and I'm quite puzzled about the fact that I've never actually seen introductory material that explains it well. In fact, I suspect that figuring out why the CLHS page on the QUOTE operator is as short as it is might get one more than half-way towards grokking Lisp's evaluation model.)

Kompottkin wrote:

Paul wrote:Presumably you're supposed to be typing this at the REPL, so there is no problem (the results are perfectly well defined as long as you don't file-compile it).

Well...

“The consequences are undefined if literal objects (including quoted objects) are destructively modified.” (CLHS, Special Operator QUOTE)

What you're probably thinking of is the fact that literals may be coalesced only when doing file compilation. However, literals may never be destructively modified, regardless of whether or not they have been coalesced.

It's not just a matter of coalescing it with other similar lists, the file compiler can do other things, like putting it in read-only memory...but the REPL can never do such things, so in fact you can modify "literals"; it's perfectly well defined (the only thing that matters is what QUOTE does, and that's fully specified (i.e., it just returns the exact same object that appears as its argument), so there's no way quoted literals can behave in any way differently from unquoted objects...including those constructed at run-time. If the former were "undefined", so would the latter be, and then you'd have to stick to purely functional code )

Paul wrote:It's not just a matter of coalescing it with other similar lists, the file compiler can do other things, like putting it in read-only memory...but the REPL can never do such things

(Emphasis mine.)

I don't believe the spec says that. See below.

so in fact you can modify "literals"; it's perfectly well defined

Again, the HyperSpec is very explicit in this regard:

“The consequences are undefined if literal objects (including quoted objects) are destructively modified.” (CLHS, Special Operator QUOTE)

“The consequences are undefined if literal objects are destructively modified.” (CLHS, 3.7.1 Modification of Literal Objects)

Finally, if still in doubt, see CLHS, Issue CONSTANT-MODIFICATION.

the only thing that matters is what QUOTE does, and that's fully specified

No, it's not the only thing that matters. In fact, QUOTE seems to me to be irrelevant. It's true that modifying the return value of (eval `(quote ,(list 1 2 3))) is (I believe) perfectly well-defined, but that's because in this case, the argument to quote is not a literal (while in (quote (1 2 3)), it is).

“The consequences are undefined if literal objects are destructively modified.” As far as I can see, that's all there is to it. (Of course, it wouldn't be the first time I misread the spec. )

Kompottkin wrote:“The consequences are undefined if literal objects are destructively modified.” As far as I can see, that's all there is to it. (Of course, it wouldn't be the first time I misread the spec. )

But you're reading the spec as if it has some mystical powers. The spec says that because it's true, but it's not true in every instance; the fact that the spec says it doesn't make otherwise well-defined (by that same spec) operations magically behave differently.

No, it's not the only thing that matters. In fact, QUOTE seems to me to be irrelevant. It's true that modifying the return value of (eval `(quote ,(list 1 2 3))) is (I believe) perfectly well-defined, but that's because in this case, the argument to quote is not a literal (while in (quote (1 2 3)), it is).

You're right, quote isn't relevant because most data types don't need it: "123" is the same as (quote "123"), etc. The identity of the object inside the (putative) quote form is what matters. There is no way for Lisp to distinguish (list 1 2 3) from (quote (1 2 3)) after the object exists (the only difference is when it comes into existence: read time vs. eval time). The reason it's "undefined" is that the file-compiler can (=necessarily must) change object identities: if it puts that list in the file, it's not going to be the same, identical, list when you load the file. But that can't happen "inside" Lisp (to stuff you type at the REPL). In other words, even if a Lisp implementer actively wanted to make destructive modification of a literal do something unexpected, just for laughs or whatever, there's no way he could do it (in general; you can catch special cases, like where it's lexically apparent...so SBCL can issue warnings, etc.)

I.e., the consequences of what is defined in the spec takes precedence over what isn't. (If I define ⊕ such that a⊕b=a+b when 6<=a<=b<=14, my spec would be correct to say "the consequences are undefined if ⊕ is applied to two integers", nevertheless 8⊕11 is 19!)

It is common for compilers to put constants in read-only memory... It is also common to identify multiple instances of a constant value and coalesce them. A compiler may precompute the length of a literal list. etc. etc.

Modifying literals is not recommended in most any language.

nuntius wrote:It is common for compilers to put constants in read-only memory... It is also common to identify multiple instances of a constant value and coalesce them.

But the in-core compile (COMPILE, as opposed to COMPILE-FILE) can't do those things; it would require changing the identity of the objects.

Paul wrote:But you're reading the spec as if it has some mystical powers. The spec says that because it's true, but it's not true in every instance; the fact that the spec says it doesn't make otherwise well-defined (by that same spec) operations magically behave differently.

Good point.

even if a Lisp implementer actively wanted to make destructive modification of a literal do something unexpected, just for laughs or whatever, there's no way he could do it

That's the thing I'm not so sure about. eval could be evil and magically tag the code objects given to it by putting them into a hash table or something so as to be able to detect modification of literals. In this case, the following would be ill-defined even though *stuff* is perfectly mutable from the point of view of the outside code:

(declaim (special *stuff*))
(let ((*stuff* (copy-list '(setf (car *stuff*) 'cons))))
  (eval *stuff*))

Moreover, it's not clear to me that the following is a non-conforming REPL implementation (given that the spec doesn't really say much about the behavior of a REPL):

(loop
  (let ((input (move-into-read-only-memory (read))))
    (print (eval input))))

Of course, all of the above is pretty pathological. After all, I don't know of any implementation that disagrees with your interpretation here, so it's mostly a moot point. If some behavior is quasi-standard, it may as well be considered standard behavior.

Kompottkin wrote:Moreover, it's not clear to me that the following is a non-conforming REPL implementation (given that the spec doesn't really say much about the behavior of a REPL):

(loop
  (let ((input (move-into-read-only-memory (read))))
    (print (eval input))))

(defvar *x* (list 1 2 3 4))

So *x* is safely mutable...

(quote #.*x*)  ; anything involving #.*x* here 

Oops...REPL just put *x* in read-only memory...

Paul wrote:

(quote #.*x*)  ; anything involving #.*x* here 

Oops...REPL just put *x* in read-only memory...

Ah, sharp-dot! I completely forgot about that one. It may well prevent REPLs from doing anything disruptive here.

You win. I guess.

To me, the sharp-dot appears to be a red herring. How does evaluating a variable make its value into a literal list? It counters Kompottkin's example, but not the general statement that modifying literals invokes explicitly undefined behavior.

Relevant passages (from a quick search):
http://www.lispworks.com/documentation/ ... /03_bd.htm
"eval and compile do not copy or coalesce constants"

http://www.lispworks.com/documentation/ ... s083_w.htm

http://www.lispworks.com/documentation/ ... y/03_g.htm

http://www.lispworks.com/documentation/ ... ql.htm#eql
"(eql '(a . b) '(a . b))
=> true
OR=> false
...
(eql "Foo" "Foo")
=> true
OR=> false"

You don't need coalescing to cause a violation. Constant data can be put into read-only memory or induce optimizations all by itself. Here's a simple example that IMO rightly gives rather contradictory information in SBCL's REPL (ecl and clisp both give a "consistent" final answer).

(let ((a '(1 2)))
  (setf (cddr a) (cons 3 nil))
  (list a (length a)))

nuntius wrote:To me, the sharp-dot appears to be a red herring. How does evaluating a variable make its value into a literal list?

It doesn't. The point is that there's no place in the implementation where it could possibly tell the difference between a literal and a non-literal value, therefore it can't do anything that would make modifying literals behave badly (as long as you don't go through the file compiler; note that the #.*x* "literal" in would be split off from the in-core *x* variable if you put that through the file-compiler, too ... so that value would indeed have the same restriction on mutation as any (other) literal; it's only because the in-core identity can't be changed that it works there)

http://www.lispworks.com/documentation/ ... ql.htm#eql
"(eql '(a . b) '(a . b))
=> true
OR=> false
...
(eql "Foo" "Foo")
=> true
OR=> false"

But these, again, are making allowance for the file-compiler, etc.; the reader can't know, when it reads the first '(', that what follows is similar to something the implementation already knows about, which it would have to know in order to return the same object. I.e., there's no way for (eql '(a . b) '(a . b)) or (eql "Foo" "Foo"), typed directly at the REPL, to return anything but NIL; that would either require READing (a . b) or "Foo" to return the identical object as a previous invocation of READ, which this one has no way to know about, or, lower down the call chain, for one object to be switched for another similar one—which could again be defeated by the #. trick! It can't change object identity after it's constructed (except for fixnums and characters, theoretically...but I think not in practice)

Here's a simple example that IMO rightly gives rather contradictory information in SBCL's REPL (ecl and clisp both give a "consistent" final answer).

(let ((a '(1 2)))
  (setf (cddr a) (cons 3 nil))
  (list a (length a)))

[/quote]

Yes, you can do optimizations when the "literalness" is visible to the compiler. Hence my comment above, "(you can catch special cases, like where it's lexically apparent...so SBCL can issue warnings, etc.)". But that's an optimization on LENGTH, nothing to do with the list itself. You can't write

(defun flub (x)
  (setf (cddr x) (cons 3 nil)
  (list x (length x)))

(flub '(1 2))

and get the same result.

Paul wrote:

http://www.lispworks.com/documentation/ ... ql.htm#eql
"(eql '(a . b) '(a . b))
=> true
OR=> false
...
(eql "Foo" "Foo")
=> true
OR=> false"

But these, again, are making allowance for the file-compiler, etc.; the reader can't know, when it reads the first '(', that what follows is similar to something the implementation already knows about,

The compiler might decide to fold common literal sub-expressions. In that case (eq '(a . b) '(a . b)) might return true.

Your mistake is that you exclude ways the compiler might work just because you cannot imagine it. That doesn't mean that a smart compiler builder couldn't imagine either. In fact, the only thing you really know is that an ANSI compiant lisp compiler obeyes the rules of the ANSI standard. If you cannot derive a compiler behaviour form those rules then you cannot assume it. You cannot make assumptions just because they are sound.

Konfusius wrote:

Paul wrote:

http://www.lispworks.com/documentation/ ... ql.htm#eql
"(eql '(a . b) '(a . b))
=> true
OR=> false
...
(eql "Foo" "Foo")
=> true
OR=> false"

But these, again, are making allowance for the file-compiler, etc.; the reader can't know, when it reads the first '(', that what follows is similar to something the implementation already knows about,

The compiler might decide to fold common literal sub-expressions. In that case (eq '(a . b) '(a . b)) might return true.

The compiler only sees the tree of objects that comes out of READ, not the text that goes in (in real Lisp; not necessarily true in Scheme), so it can't distinguish between (eq '(a . b) '(a . b)) and something like (eq '#.*x* '#.*y*) where *x* and *y* are guaranteed to be distinct conses which require it to return NIL. The file-compiler would break the connection to *x* and *y*, so it can make them EQ. Since it can see them both, and has reason to think they're similar constants, the compiler could replace the whole form with T (never even looking at *x* and *y*), but then you'll have different behaviour between the compiler and interpreter (which is only a problem if you have an interpreter, I suppose, but still...)