Local variables with indefinite extent?

Hi forum,

What would be the reasonable way to create a function-local variable with indefinite extent; i.e. that keeps its value between invocations of the function? The "function-local" part isn't really critical, of course, as long as it doesn't dirty the namespace otherwise. My naive implementation, I guess, would look something like this:

(defmacro let-indef (var init-form &body body)
  (let ((real-var (gensym)))
    `(locally
         (declare (special ,real-var))
       (unless (boundp ,real-var) (setf ,real-var ,init-form))
       (symbol-macrolet ((,var ,real-var))
	 ,@body))))

However, it doesn't seem to work. When I try to use a variable declared as such in SBCL, I get the error "The variable #:G0 is unbound.". I guess "#:G0" is my gensymed special symbol, but I don't understand why it is unbound, seeing how I assign it explicitly when unbound. Also, I'm unsure how this technique would interact with compilation (do gensymed specials work?). Does anyone know?

Dolda wrote:What would be the reasonable way to create a function-local variable with indefinite extent; i.e. that keeps its value between invocations of the function?

I don't really think this is a reasonable thing to do in the first place. If you want a global, then make a global, and let packages worry about namespaces. Creating a variable which is only accessible in some lexical scope but which maintains a value without being associated with some first class object like a closure is somehow not elegant for me. But it might be just due to functional point of view. Also, there is no good way to debug code like that.

You could perhaps just create a global closure by wrapping an entire function definition in a LET. But it would be best to thread the state through arguments, so that it is explicit.

Dolda wrote:I guess "#:G0" is my gensymed special symbol, but I don't understand why it is unbound, seeing how I assign it explicitly when unbound.

BOUNDP is a function. It evaluates its argument.

(let ((persistent-variable initial-value))
  (defun function-with-persistent-variable (arguments)
    body))

Don't use it too often though. I never use it on toplevel; using it this way, you can't reset the function(locally) neatly. If persistent-variable was a defvar/defparameter, you could use it insulatedly:

(defvar *persistent-variable* initial-value)
(defun function-with-persistent-variable (arguments)
  body))

(let ((*persistent-variable* initial-value)) ;insulate it.
  (function-with persistent-variable ...))

Anyway, try avoid code with side-effects, unless insolated inside a function, or it is some sort of definition/storage.

Btw, if you want to understand how variables are created, you can do everything with defvar(/defparameter), funcall and lambda. (See for yourself how to make a let with funcall and lambda)

Edit: Or do you mean a 'locally special' variable? Don't know if that is possible.

I believe that local variables with indefinite extent are a very nice feature. They are useful to create pools, caches, buffers and memoized functions.

The problem with your code though is that you don't bound the variable, i.e., you don't give a value to it. But don't use this hack, use let instead. You can use let and share the same variable through many functions:

(let (pool)
  (defun create-object (...)
    (prepare-object
      (or (pop pool)
          (make-object ...))))

  (defun destroy-object (object ...)
    (cleanup object ...)
    (push object pool)
    (values)))

Jasper wrote:Edit: Or do you mean a 'locally special' variable? Don't know if that is possible.

I can't even think how that would work. I mean, is there any (useful) way that a variable be special while being used only by one function?

gugamilare wrote:I can't even think how that would work. I mean, is there any (useful) way that a variable be special while being used only by one function?

Original poster did gave an example implementation and it does work if you insert a missing quote. Of course there is no actual reason for the variable to be special. You just need a load time object. A special symbol will simplify the symbol-macro though.

I actually failed to think of that in my original reply. There is LOAD-TIME-VALUE special operator, which can be used to obtain exactly that:

(defmacro let-indef2 (var init-form &body body)
  (let ((real-var (gensym))
        (unbound-mark (gensym)))
    `(let ((,real-var (load-time-value (make-array nil :initial-element ',unbound-mark) nil)))
       (when (eq (aref ,real-var) ',unbound-mark)
         (setf (aref ,real-var) ,init-form))
       (symbol-macrolet ((,var (aref ,real-var)))
         ,@body))))

I still don't think this is a good idea, it is best to leave a way to inspect the value.

Ramarren wrote:Of course there is no actual reason for the variable to be special.

That is what I meant.

Ramarren wrote:I still don't think this is a good idea, it is best to leave a way to inspect the value.

Leaving a special global variable will sometimes pollute your namespace, slow things down (thus rendering a pool or cache useless) and they are obviously bad when your are implementing a memoized function.

Your macro is a dirty hack that should not be used, though, just use let instead.

I can imagine a use for a local special variable(being exactly the same and applicable only on the functions within.), pretty marginal though(too marginal actually), as you can also just use regular special variables and just not export them out of the package.

Leaving a special global variable will sometimes pollute your namespace, slow things down (thus rendering a pool or cache useless) and they are obviously bad when your are implementing a memoized function.

Really? I always see special variables as arguments that are automatically added to functions for you.(And use them pretty much any time i get tired of entering unmodified arguments, or if there is a bunch of 'potential arguments') In cases where a function does not take a function as argument, at least the compiler should be able to see exactly which special vars a function is dependent on. In cases where it does take a function as argument, it has to look if both functions share special vars it (locally) sets/depends on. Edit: of course if the argument function is unsure, you always need to send the special vars that are changed in the other function.

Jasper wrote:Really? I always see special variables as arguments that are automatically added to functions for you.

That won't work if the function invokes another function that changes the value of that variable.

It is ok to use special vars as much as use keyword and optional arguments. In general you should not worry about them. But I've already seen code in which the bottleneck is the use of special variables. Think of the extreme case where the variable holds a number and the function is called many times to do simple computations with that value, possibly changing it.

I believe that the enemy here is multiple threading, it makes the implementation to need to look for the variables in different bindings.

I can't believe I didn't even think of just using LET.

But, well, that brain fart aside, there are reasons to avoid it. In particular, the LET has to be outside the function being declared, which makes it difficult to use in macros. What I have in mind, in the end, is the ability to do something like this:

(defmacro pre-fetched (value) (with-gensyms (cache) `(let-indef ,cache (slow-function ,value) ,cache))
(defmethod foo () (pre-fetched *a-constant*))

Maybe there's some better way to do that that hasn't crossed my mind, though?

Ramarren wrote:

Dolda wrote:I guess "#:G0" is my gensymed special symbol, but I don't understand why it is unbound, seeing how I assign it explicitly when unbound.

BOUNDP is a function. It evaluates its argument.

Em. Right. This is getting embarrassing.

Are you just looking for a memoization package? The wikipedia page has links to a couple CL implementations.

There is the library Memoize. It maintains a register of memoized functions so you can unmemoize it later. It was the only library I could find.