Changing syntax within a form

Hi all,

I was playing around with an idea for a code generator for a language we'll call "platypus" by defining an s-expression syntax for it, something like c-amplify

I defined (in the "platypus" package) some expansion macros with names like "class" and "function" that happen to be symbols in CL. For convenience, I'd like the user to be able to use these tokens "unadorned" by introducing the new syntax by wrapping it in a form like so:

(platypus
  ;; Introduces new syntax & symbol table
  (class blah ...
    ... etc... ))

I thought I could make this happen by lexically binding *package* like this:

(defmacro platypus (&body body)
  `(let ((*package* (find-package :platypus)))
     ,@body))

But this doesn't seem to work with the binding either inside or outside the quasi-quote. I presume this is because the reader has already snarfed all the tokens for the whole form by the time we macroexpand?

Then next best I could do was:

(defmacro platypus (&body body)
          `(macrolet ((class (&body body)
                        `(platypus:class ,@body)))
             ,@body))

This seems to work, at least for trivial uses, which is probably good enough for now, but it seems a little clunky.

Is there a better approach to this? Am I looking at the problem in the wrong way?

Why can't a user use use-package?

Goheeca wrote:Why can't a user use use-package?

That could probably be workable, but it's not really what I want - this syntax is embeddable, so it seems like it should be introduced within an enclosing form that delimits its scope. I don't really want all these extra symbols spewing all over the user's own package space.

If you've ever used parenscript, that's sort of what I was picturing. Or picture having an s-expression syntax on top of Java so that you could write Java code in terms of domain concepts.

And what about these macros?

(defmacro platypus (&body body)
  `(unwind-protect
       (progn (use-package 'platypus)
              ,@body)
     (unuse-package 'platypus)))

and

(defmacro platypus (&body body)
  `(let ((*package* (make-package (gensym) :use (list *package*))))
      (use-package 'platypus)
      ,@body))

They seem pretty promising - will play with them a bit. Thanks, Goheeca!

*package* is used by the reader; once you hit macros, symbols have already been INTERNed and its too late.

That said, its perfectly fine to traverse through the source forms, use symbol-name, and replace one symbol with another.
CL:LOOP uses symbol-name to identify loop "keywords".

Here's a simple project that uses this trick to achieve lexical scoping for package aliases.
http://git.tentpost.com/?p=lisp/package-aliases.git

Hm man is still learning resp. I'm sometimes unaware, but nuntius my first macro is still working, isn't it? Since it gives a sense/binds symbols in the package in which the symbols have been read.

No it's not working either. So I've played with that problem and I've created this:

(defun save-symbol (sym)
  (copy-symbol (find-symbol (symbol-name sym)) t))

(defun restore-symbol (sym)
  (unintern (find-symbol (symbol-name sym)))
  (import sym))

(defun replace-symbol (in-sym)
  (let ((sym (intern (symbol-name in-sym))))
    (setf (symbol-plist sym) (symbol-plist in-sym))
    (if (boundp in-sym)
      (setf (symbol-value sym) (symbol-value in-sym))
      (makunbound sym))
    (if (fboundp in-sym)
      (setf (symbol-function sym) (symbol-function in-sym))
      (fmakunbound sym))
    sym))

(defmacro rebind-from (pckg &body body)
  (let ((backup (gensym))
        (symbols (gensym)))
    `(let ((,backup)
           (,symbols))
        (unwind-protect
           (progn (do-external-symbols (sym (find-package ,pckg)) (push sym ,symbols))
                  (setf ,backup (loop for sym in ,symbols collect (save-symbol sym) do (replace-symbol sym)))
                  ,@body)
          (loop for sym in ,backup do (restore-symbol sym))))))

It has disadvantages according to CLHS:

• The consequences are undefined if an attempt is made to change the value of a symbol that names a constant variable, or to make such a symbol be unbound. -- This is a problem, I don't know how to find out if the variable is a constant or not. The unbinding I don't actually have to do with that bindigs would cumulate though.
• The consequences are undefined if an attempt is made to change the functional value of a symbol that names a special form. -- This can be handled by special-operator-p, but probably you can only rise an error.

Moreover SBCL is gibbering style warnings so I would entitle the code as a hack. A better/nicer solution will be code walking or wrapping into labels, let, macrolet, etc.

Again, see the package-aliases project above for a simple, proven approach to this problem.
In essence, the macro walks the source form and replaces occurrences of symbol X with symbol Y.

Thanks for that pointer, nuntius. That's a really nice illustrative example - short enough to be manageable for a newb, but not over-simplified from real-world problems.

Much appreciated as always. I'm really grateful to all those who take the time to share their knowledge and experience.

Since I've got to know the predicate constantp, I've made a decision to update this thread. To improve the aforementioned toy for hackers:

(defun save-symbol (sym)
  (copy-symbol (find-symbol (symbol-name sym)) t))

(defun restore-symbol (sym)
  (unintern (find-symbol (symbol-name sym)))
  (import sym))

(defun replace-symbol (in-sym mode)
  (let ((sym (intern (symbol-name in-sym))))
    (setf (symbol-plist sym) (symbol-plist in-sym))
    (cond ((constantp sym) (error "~S is a constant." sym))
          ((boundp in-sym) (setf (symbol-value sym)
                                 (symbol-value in-sym)))
          ((boundp sym) (when (eq mode :substitute) (makunbound sym))))
    (cond ((special-operator-p sym) (error "~S is a special operator." sym))
          ((fboundp in-sym) (if (macro-function in-sym)
                                (setf (macro-function sym)
                                      (macro-function in-sym))
                                (progn (fmakunbound sym)
                                       (setf (symbol-function sym)
                                             (symbol-function in-sym)))))
          ((fmakunbound sym) (when (eq mode :substitute) (fmakunbound sym))))
    sym))

(defmacro rebind-from ((pckg &key (mode :substitute)) &body body)
  (unless (member mode #1='(:substitute :add))
    (error "A bad mode passed: ~S~%Available modes: ~{~S~^, ~}" mode #1#))
  (let ((backup (gensym))
        (symbols (gensym)))
    `(let ((,backup)
           (,symbols))
       (unwind-protect
	    (progn (do-external-symbols (sym (find-package ,pckg))
                     (push sym ,symbols))
                   (setf ,backup (loop for sym in ,symbols
                                    collect (save-symbol sym)
                                    do (replace-symbol sym ,mode)))
                   ,@body)
	 (loop for sym in ,backup do (restore-symbol sym))))))

It has original constraints, but now they are handled by rising an error. Further it affects symbols interned during runtime against that codewalker. However, it can be fixed with the *package* variable and then the codewalker is certainly better to use. As I say it's a toy.