Persistent Objects
Persistent Objects
Has anyone had any experience with persistent object libraries? I've tried cl-prevalence, elephant, and rucksack, but none work on clisp 2.45, running on windows xp . Is there any out there that actually work?
Oh, and i do realize that elephant uses uffi, which doesn't support clisp. I used the cffi-uffi-compat package that came with cffi. I eventually ran up against an error i couldn't understand, namely 'can't redefine standardized method. use call-next-method instead.' hmmm.
Also, rucksack doesn't support clisp either, but i modified the file package.lisp so as to not signal an error when clisp is used. Then i told it to use the clos package for clisp (the file package.lisp required each implementation to specify location of mop protocol). Then, i modified rucksack.lisp in order to make dummy with-lock, process-lock, and make-lock functions/macros, since clisp doesn't support multithreads. This time, the error that i ran up against was that file-position was attempting to position a stream past then end of the file. Argg...
And yes, it works on sbcl, but i want to use clisp!
Oh, and i do realize that elephant uses uffi, which doesn't support clisp. I used the cffi-uffi-compat package that came with cffi. I eventually ran up against an error i couldn't understand, namely 'can't redefine standardized method. use call-next-method instead.' hmmm.
Also, rucksack doesn't support clisp either, but i modified the file package.lisp so as to not signal an error when clisp is used. Then i told it to use the clos package for clisp (the file package.lisp required each implementation to specify location of mop protocol). Then, i modified rucksack.lisp in order to make dummy with-lock, process-lock, and make-lock functions/macros, since clisp doesn't support multithreads. This time, the error that i ran up against was that file-position was attempting to position a stream past then end of the file. Argg...
And yes, it works on sbcl, but i want to use clisp!
Re: Persistent Objects
A hah! I found an annoying implementation difference! On clisp, if file-position is given a number beyond it's eof, it signals an error, but if sbcl receives a number beyond it's eof, it just fills the non-used positions with a #\null character or 0 or whatever I redefined file-position in the rucksack package. Wow, now the macro with-rucksack works , now i have to see what's else is wrong, still getting that pesky end of file error!
Re: Persistent Objects
Ok, so i found the problem, but am a little clueless. Here's the new file-position function, which assumes the stream is of type '(unsigned-byte 8), since it always is in rucksack.
It appears that for one call tof ile-position, the file-length isn't increasing even though write-byte is called. The output for this particular case is (minus the "writing" 0 part, which indicates that the write has succeded, according to write-byte) :
As you can see, i need an extra length of (24-10) +1 = 15 added, so the end file-length should be 10+15 = 25.
But, since for some reason it stays at 10 for a while, it doesn't work.
Anyone knows what's happening? Thx!
EDIT:I even tried restarting clisp and re-loading, i got the same error, which is something of a relief
Code: Select all
(defun file-position (stream &optional position)
#-clisp (cl:file-positions stream position)
#+clisp (if (not position)
(cl:file-position stream)
(let ((eof-position (file-length stream)))
(if (>= position eof-position)
(progn
(format t "Before: stream-element-type is ~A~%" (stream-element-type stream))
(format t "EOF-Position is ~A current-file-position is ~A wanted-position is ~A~%" eof-position
(file-position stream) position)
(loop repeat (1+ (- position eof-position))
for index = 1 then (1+ index)
do(format t "current-rep is ~A cur-file-length ~A~%" index
(file-length stream))
do(progn (print "writing") (print (write-byte 0 stream))))
(format t "right-before-positioning: file-length is ~A~%" (file-length stream))
(cl:file-position stream position) (format t "After~2%"))
(cl:file-position stream position)))))
Code: Select all
Before: stream-element-type is (unsigned-byte 8)
EOF-Position is 10 current-file-position is 3 wanted-position is 24
current-rep is 1 cur-file-length 10
current-rep is 2 cur-file-length 10
current-rep is 3 cur-file-length 10
current-rep is 4 cur-file-length 10
current-rep is 5 cur-file-length 10
current-rep is 6 cur-file-length 10
current-rep is 7 cur-file-length 10
current-rep is 8 cur-file-length 10
current-rep is 9 cur-file-length 11
current-rep is 10 cur-file-length 12
current-rep is 11 cur-file-length 13
current-rep is 12 cur-file-length 14
current-rep is 13 cur-file-length 15
current-rep is 14 cur-file-length 16
current-rep is 15 cur-file-length 17
right-before-position: file-length is 18
But, since for some reason it stays at 10 for a while, it doesn't work.
Anyone knows what's happening? Thx!
EDIT:I even tried restarting clisp and re-loading, i got the same error, which is something of a relief
Re: Persistent Objects
Woops. I forgot to set the file-position to the end of the stream for this particular case
Modified Code:
Hmm... It seems sbcl is way faster than the same test program in clisp. Wonder why?
Modified Code:
Code: Select all
(defun file-position (stream &optional position)
#-clisp (cl:file-position stream position)
#+clisp (if (not position)
(cl:file-position stream)
(let ((eof-position (file-length stream)))
(if (>= position eof-position)
(progn
(cl:file-position stream eof-position)
(loop repeat (1+ (- position eof-position))
do(write-byte 0 stream))
(cl:file-position stream position))
(cl:file-position stream position)))))
Re: Persistent Objects
OK, so i think i finally got rucksack working with clisp. Just a quick question, though. Is the above file-position function the most efficient implementation or can i do better. In addition, just wondering, is the file-length property figured out by counting through the file or is this property stored somehow in the stream?
Re: Persistent Objects
I don't know the answer to your question, but I'm impressed by your diligence in making this work with CLisp
Re: Persistent Objects
Thx! The changes were actually very very minor, its just finding them thats really annoying. Which reminds me, i should probably relate what i did to get it to work, for others.
Locate the package.lisp file.
At the top should be something like #-(...bunch of lisp implementations) (error "...")
Include clisp in the bunch of lisp implementations.
Then, go to the :use statement. Add #+clisp :clos to the list
Now, we need to add a new file-position function. In sbcl, if we try to position
the file beyond the eof, it will add #\Null characters or 0s for padding depending on
stream type.
Add following statement to defpackage statement
(:shadow :file-position)
Then, after the (in-package :rucksack) at the end of the file, add the following.
Locate the rucksack.lisp file
Rucksack uses locks. Since clisp doesn't contain these, you need to add dummy
lock macros. Should end up looking like this
Locate the serialize.lisp file. Change the max-character-code function to the following.
The problem here is that in sbcl, it will return 0 if the string is empty, but clisp will return nil.
(defun max-character-code (string)
"Returns the highest character code in string."
#+clisp (if (string-equal string "") (return-from max-character-code 0))
(loop for char across string
maximize (char-code char)))
Finally, and this is a hack, we need to suppress clisp warnings. The thing is, when we have nested (with-rucksacks) for the same database, it tries to have more than one open stream to a file. If we don't suppress it, it will crash our program. I guess you could also try try catch blocks, but whatever.
In windows:create a shortcut to clisp by right clicking on clisp.exe and clicking create shortcut. Rename it as "click-appease" or something. Right click shortcut and click on properties. Under shortcut tab, rename target to something like following, replacing thing between parantheses by the pathname of your shortcut.
"C:\Program Files\clisp-2.45\clisp.exe" -on-error appease
Click OK, and your done. Use this shortcut when you use clisp and rucksack together.
No clue how to do it on other systems.
A much better way would probably be to create a global hashtable which has
the pathname as a key and an open stream or nil if there is none as the item. Something like that, anyways. Actually, you shouldn't have nested with-rucksacks for the same database in the first place, but you can figure out why the example signals a warning.
Locate the package.lisp file.
At the top should be something like #-(...bunch of lisp implementations) (error "...")
Include clisp in the bunch of lisp implementations.
Then, go to the :use statement. Add #+clisp :clos to the list
Now, we need to add a new file-position function. In sbcl, if we try to position
the file beyond the eof, it will add #\Null characters or 0s for padding depending on
stream type.
Add following statement to defpackage statement
(:shadow :file-position)
Then, after the (in-package :rucksack) at the end of the file, add the following.
Code: Select all
(defun file-position (stream &optional position)
#-clisp (cl:file-position stream position)
#+clisp (if (not position)
(cl:file-position stream)
(let ((eof-position (file-length stream)))
(if (>= position eof-position)
(progn
(cl:file-position stream eof-position)
(loop repeat (1+ (- position eof-position))
do(write-byte 0 stream))
(cl:file-position stream position))
(cl:file-position stream position)))))
Locate the rucksack.lisp file
Rucksack uses locks. Since clisp doesn't contain these, you need to add dummy
lock macros. Should end up looking like this
Code: Select all
(defun make-lock (&key (name "lock"))
#+allegro
(mp:make-process-lock :name name)
#+lispworks
(mp:make-lock :name name)
#+sbcl
(sb-thread:make-mutex :name name)
#+openmcl
(ccl:make-lock name)
#-(or allegro lispworks sbcl openmcl)
nil)
(defmacro with-lock ((lock) &body body)
#+allegro
`(mp:with-process-lock (,lock) ,@body)
#+lispworks
`(mp:with-lock (,lock) ,@body)
#+sbcl
`(sb-thread:with-mutex (,lock) ,@body)
#+openmcl
`(ccl:with-lock-grabbed (,lock) ,@body)
#-(or allegro lispworks sbcl openmcl) `(progn ,@body))
(defun process-lock (lock)
#+lispworks
(mp:process-lock lock)
#+sbcl
(sb-thread:get-mutex lock)
#-(or sbcl lispworks)
nil)
(defun process-unlock (lock)
#+lispworks
(mp:process-unlock lock)
#+sbcl
(sb-thread:release-mutex lock)
#-(or sbcl lispworks)
nil)
The problem here is that in sbcl, it will return 0 if the string is empty, but clisp will return nil.
(defun max-character-code (string)
"Returns the highest character code in string."
#+clisp (if (string-equal string "") (return-from max-character-code 0))
(loop for char across string
maximize (char-code char)))
Finally, and this is a hack, we need to suppress clisp warnings. The thing is, when we have nested (with-rucksacks) for the same database, it tries to have more than one open stream to a file. If we don't suppress it, it will crash our program. I guess you could also try try catch blocks, but whatever.
In windows:create a shortcut to clisp by right clicking on clisp.exe and clicking create shortcut. Rename it as "click-appease" or something. Right click shortcut and click on properties. Under shortcut tab, rename target to something like following, replacing thing between parantheses by the pathname of your shortcut.
"C:\Program Files\clisp-2.45\clisp.exe" -on-error appease
Click OK, and your done. Use this shortcut when you use clisp and rucksack together.
No clue how to do it on other systems.
A much better way would probably be to create a global hashtable which has
the pathname as a key and an open stream or nil if there is none as the item. Something like that, anyways. Actually, you shouldn't have nested with-rucksacks for the same database in the first place, but you can figure out why the example signals a warning.