A "declare" form returned by macro

After xmas, I started to write my first code in Common Lisp. I started with the following function:

(defun fixvect (f_len f_initEl)
  (declare (type fixnum f_len)
           (type fixnum f_initEl))
  (make-array (list f_len) :element-type 'fixnum :initial-element f_initEl))

Of course, the function above is supposed to make a new 1-d array of fixnum values, filled with the given initial element.

Soon it becomes obvious to me, that I'll use the declare forms a lot in my code. So, I decided to write the following macro:

(defmacro decltype (&rest lst)
  (defun type-collect (lst)
    ;;; (some-type a b c) -> ( (type some-type a) (type some-type b)
    ;;;                        (type some-type c) )
    (let ( (typ (car lst))
           (vars (cdr lst)) )
      (loop for var in vars collect `(type ,typ ,var))))
  (let ( (type-list nil) )
    (loop for lstInt in lst do
      (setf type-list (append type-list (type-collect lstInt))))
    `(declare ,@type-list)))

Now, I could replace the form:

(declare (type fixnum f_len)
         (type fixnum f_initEl))

with the following line:

(decltype (fixnum f_len f_initEl))

At least, I thought so... Common Lisp compiler told me I was doing something wrong:

; in: LAMBDA NIL
;     (DECLTYPE (FIXNUM F_LEN F_INITEL))
; 
; caught STYLE-WARNING:
;   (in macroexpansion of (DECLTYPE (FIXNUM F_LEN F_INITEL)))
;   (hint: For more precise location, try *BREAK-ON-SIGNALS*.)
;   redefining TYPE-COLLECT in DEFUN

; in: LAMBDA NIL
;     (DECLTYPE (FIXNUM F_LEN F_INITEL))
; ==>
;   (DECLARE (TYPE FIXNUM F_LEN) (TYPE FIXNUM F_INITEL))
; 
; caught WARNING:
;   There is no function named DECLARE. References to DECLARE in some contexts
;   (like starts of blocks) have special meaning, but here it would have to be a
;   function, and that shouldn't be right.

; --> DECLARE 
; ==>
;   (TYPE FIXNUM F_LEN)
; 
; caught WARNING:
;   undefined variable: FIXNUM
; 
; caught WARNING:
;   The function TYPE is undefined, and its name is reserved by ANSI CL so that
;   even if it were defined later, the code doing so would not be portable.

; 
; caught WARNING:
;   This variable is undefined:
;     FIXNUM

; 
; caught STYLE-WARNING:
;   These functions are undefined:
;     DECLARE TYPE
; 
; compilation unit finished
;   caught 4 WARNING conditions
;   caught 2 STYLE-WARNING conditions

I have to admit, that I don't get it. AFAIK (or, to be more precise, as far as I get the concept of macros in CL), the macro is expanded to some form at the compile time, and that form is then evaluated. The macro I wrote expands to the right form (I checked it with macroexpand-1), so what's the problem, mr CL compiler?

Perhaps the problem is that the "declare" form shouldn't be evaluated (it's an information for CL compiler), and CL compiler tries to evaluate it, so both "declare" and "type" are treated as functions. If this is the case, how can I define syntax for (decltype (fixnum var*))?

TPJ wrote:how can I define syntax for (decltype (fixnum var*))?

The thing is that you do not need define this syntax. You can just use (declare (type fixnum var*)) (details at http://www.lispworks.com/documentation/ ... d_type.htm).

Yeap. I'm studying the CLtL, but I obviously missed that information. Now I feel like a silly...

Thanks for your help.

The specifications specifically prohibits macros expanding into declarations:
http://www.lispworks.com/documentation/ ... declar.htm

Macro forms cannot expand into declarations; declare expressions must appear as actual subexpressions of the form to which they refer. 

I think this is because declaration can affect how the rest of the form will be compiled, and so they must be there before arguments are macroexpanded.

Altering your macro to something like the following is the best you can do, I think. But since the declarations will apply to the LET form in the macro rather than to the function itself, I don't know how much good it will do if you're adding those for speed.

(defmacro with-declare (declarations &body body)
  `(let ,(loop for a in declarations
            append (loop for b in (cdr a)
                        collect (list b b)))
     (declare ,@(loop for a in declarations
                   append (loop for b in (cdr a)
                              with type = (car a)
                              collect (list 'type type b))))
     ,@body))

(macroexpand-1 '(with-declare ((fixnum a b) (string c )) (+ a b) (print c)))
==>
(LET ((A A) (B B) (C C))
  (DECLARE (TYPE FIXNUM A) (TYPE FIXNUM B) (TYPE STRING C))
  (+ A B)
  (PRINT C))

Two approaches I know of are

read macros (including #.)

(defun f (x) #.(decltype (fixnum x)) (+ x 2))

custom wrappers around def*

(defmacro add-decl (decl form)
  "Insert DECL into FORM"
  (destructuring-bind (type name args &body body) form
    (list type name args
      (eval decl) ; or macroexpand
      body)))

(add-decl 
  (decltype (fixnum x))
  (defun f (x) (+ x 2)))

This code is untested, but the idea is sound.

Both approaches can be designed for macros, functions, or even *special-variables*.
See also tools like http://common-lisp.net/project/parse-declarations/.

Of course you can also make a wrapper for DEFUN that includes syntax for type annotations.

In addition to all the other fine suggestions about how you can ease your pain in declaring types, might I suggest that you simply don't declare types at all. If you're really trying to write your first code in Common Lisp, the last thing you'll want to spend any time on is typing. Lisp doesn't require typing, so why use it? Yes, yes, I know performance, blah, blah. But come on. This is your first code. Figure out the language. Get it to run. Then worry about whether it's running fast enough. Trust me, if this really is your first program in CL, you'll be consing so much as to make all your type declarations all but irrelevant anyway.

findinglisp wrote:In addition to all the other fine suggestions about how you can ease your pain in declaring types, might I suggest that you simply don't declare types at all.

Amen to that. Most of the time, type declarations are pointless.

Although it does not work to have a macro that expands directly into a (declare ...),
you can do this kind of thing:

(defmacro with-dcl (var &body body)
`(let ((,var 0))
(declare (fixnum ,var))
,@body))

Paul Donnelly wrote:

findinglisp wrote:In addition to all the other fine suggestions about how you can ease your pain in declaring types, might I suggest that you simply don't declare types at all.

Amen to that. Most of the time, type declarations are pointless.

Yes. Type declarations are a silver bullet that you should pull out once you have profiled your code and determined where your inner loops are and you're going off to optimize things. Until then, they're just needless baggage. Further, you can actually shoot yourself in the foot if you start changing types during development and need to go back and get all your type declarations to match. If you miss something and your SAFETY settings are low, you can actually have a program that doesn't work.

So, even if you're a Lisp guru, I'd still recommend that you (1) write your code, (2) test your code, (3) profile your code, (4) THEN optimize your code, possibly with type declarations if necessary (better yet choose better algorithms), and finally (5) retest your code to make sure you didn't bork anything when you were optimizing.

If you're a Lisp newbie, simply ignore typing altogether. Lisp is not C.

Thank you all for your tips.

findinglisp wrote:In addition to all the other fine suggestions about how you can ease your pain in declaring types, might I suggest that you simply don't declare types at all. If you're really trying to write your first code in Common Lisp, the last thing you'll want to spend any time on is typing. Lisp doesn't require typing, so why use it? Yes, yes, I know performance, blah, blah. But come on. This is your first code. Figure out the language. Get it to run. Then worry about whether it's running fast enough. Trust me, if this really is your first program in CL, you'll be consing so much as to make all your type declarations all but irrelevant anyway.

I really don't want to start (yet another) rant on the "dynamic vs. static typing" here, so I'll just point my point of view.

I'm coming from static typing world (mainly Java). Although type hints *might* (not: *will*) improve performance, in my opinion they make code more readable. When I say:

(defun my-fun (a)
  (declare (type fixnum a))
	...)

I'm making it clear that "a" is a fixnum number (and not a string, a list, or something else).

My experience from using both static (mainly Java) and dynamic (mainly Python) languages shows, that using such a type hints (let's treat them as hints) allows to write less error-prone and more readable code (at least for me).

That's the main reason why I'm trying to use type hints in CL.

In my opinion, if it is not clear on a glance what arguments the function accepts, then it is either too long or the arguments are badly named. At least for simple types, and more complex things are best put into CLOS objects and operated upon with generic functions, which have something quite close to type signatures. Anyway, if the type hints are primarily for human consumption, I think they should be in a docstring, where SLIME or (describe ...) can find them, rather than in declarations which might be thrown out by some compilers (CLISP I believe ignores all type declarations, or at least it did few years ago).

TPJ wrote:

(defun my-fun (a)
  (declare (type fixnum a))
	...)

This just makes me wonder why it wants a fixnum specifically.

Ramarren wrote:In my opinion, if it is not clear on a glance what arguments the function accepts, then it is either too long or the arguments are badly named. At least for simple types, and more complex things are best put into CLOS objects and operated upon with generic functions, which have something quite close to type signatures. Anyway, if the type hints are primarily for human consumption, I think they should be in a docstring, where SLIME or (describe ...) can find them, rather than in declarations which might be thrown out by some compilers (CLISP I believe ignores all type declarations, or at least it did few years ago).

When you use packages and suggestive names for functions and variables, the whole Java world begins to look like this: http://paulgoscicki.com/pictures/static-typing.gif
Of course, some languages have very good type inference and look smarter than the above example but I still believe in documentation (describing functionality, not just classes and functions) so maybe I'm loosing modern Java IDE features because of my ignorance ...

TPJ wrote:I really don't want to start (yet another) rant on the "dynamic vs. static typing" here, so I'll just point my point of view.

I'm coming from static typing world (mainly Java). Although type hints *might* (not: *will*) improve performance, in my opinion they make code more readable.

I respect that you find it more readable. If you do, that's fine. Many others won't, however. Idiomatic Lisp code doesn't use lots of typing. Writing Lisp as it it were Java does not result in readable code for native Lisp speakers. It's like trying to speak a foreign language and deliberately using your native accent or grammar constructions when you pronounce the foreign words. It just pisses off the native speakers of that language. If you're going to speak a foreign language, at least try to use the correct accent and grammar. Yes, you won't get it down immediately. That's understandable. But it will come over time and people will have more respect for you if they see you trying. Put simply, if you want to sit in your own room and talk French to yourself with a Russian accent, that's fine with everybody because nobody has to hear you, but don't expect the French to think you're doing it right.

Regarding performance, in many cases, the compiler can infer types in any case (see what SBCL does, for instance, and you'll be amazed. It's not perfect, but unless you're in an inner loop, I'm sure you'll find the compiler type inferencing more than enough for your needs. Again, particularly as a newbie.

When I say:

(defun my-fun (a)
  (declare (type fixnum a))
	...)

I'm making it clear that "a" is a fixnum number (and not a string, a list, or something else).

Except now my-fun doesn't work (assuming you have a high enough SPEED optimization setting for the compiler to listen to you) with bignums. Or floats. Or any other numeric type. Instead, I would suggest:

(defun my-fun (num)
     ...)

Use "lst" or "l" for lists, "arr" for arrays, and "seq" for general sequences, for instance. There is no standard for this sort of thing, but there are several conventions and people will immediately understand it.

My experience from using both static (mainly Java) and dynamic (mainly Python) languages shows, that using such a type hints (let's treat them as hints) allows to write less error-prone and more readable code (at least for me).

That's the main reason why I'm trying to use type hints in CL.

Okay, that's fine. You can do whatever you want. I won't argue with you further. Just realize that anybody else who knows Lisp will look at your code and say "Gak!"

findinglisp wrote:Use "lst" or "l" for lists, "arr" for arrays, and "seq" for general sequences, for instance. There is no standard for this sort of thing, but there are several conventions and people will immediately understand it.

If you're feeling really explicit, you could even type the whole word out.

blandest wrote:When you use packages and suggestive names for functions and variables, the whole Java world begins to look like this: http://paulgoscicki.com/pictures/static-typing.gif

I have to say that Common Lisp's package management feels quite cumbersome for a newbie (or maybe the tutorial I picked back then just introduced it in an overly-verbose way, creating at least two superfluous files just to define the package and asdf systems it relied on).

Exolon wrote:

blandest wrote:When you use packages and suggestive names for functions and variables, the whole Java world begins to look like this: http://paulgoscicki.com/pictures/static-typing.gif

I have to say that Common Lisp's package management feels quite cumbersome for a newbie (or maybe the tutorial I picked back then just introduced it in an overly-verbose way, creating at least two superfluous files just to define the package and asdf systems it relied on).

ASDF systems aren't related to packages. ASDF is a way to load Lisp programs and libraries, while packages are namespaces for symbols. And you can put your DEFPACKAGE form in a file with some other stuff if you like. It's just helpful to put it in a separate file so that you can tell ASDF to load it first. If you're not using ASDF or multiple files, there's no reason to make a package file. ASDF is pretty clunky though.

i really like sbcl "declarations are assertions" principle, it does helps sometimes, specially for interface (it's ugly writing assert everywhere)