remove-duplicates

Hello all,

I have a question about the remove-duplicates function.

I understand how it works, just want to ask a few things about it:

remove-duplicates sequence &key :from-end :test :test-not
:start :end :key


1. What exactly is the :test-not argument used for?

2. I wanted to use the remove-duplicates function to get rid of duplicate characters in a string. So I typed up something like this:

 (remove-duplicates "heyyyy cooooool aaaaaa" :TEST #'CHAR-EQUAL :from-end t)

Now my intention was to get hey cool a, but of course this did not happen. I instead got hey coola. #\space is also being seen as a duplicate. My question is, is there anyway to say "remove-duplicates EXCEPT ... "? I assumed maybe this is what :test-not does, but I tried it out and got this error: Error: Invalid :TEST-NOT argument: #\Space



Thanks!

I X Code X 1 wrote:1. What exactly is the :test-not argument used for?

The specification explains it here. In general the :test argument provides a two-argument predicate function which can be used to compare things. The :test-not argument has the same meaning, but the result of the predicate is reversed. The effect of providing both is undefined and likely an error.

I X Code X 1 wrote:My question is, is there anyway to say "remove-duplicates EXCEPT ... "

You can provide a predicate which always returns false when encountering a space:

(remove-duplicates "heyyyy cooooool aaaaaa"
                   :test #'(lambda (x y)
                             (unless (eql x #\space)
                               (char-equal x y)))
                   :from-end t)

Which results in "hey col a", since obviously all #\o are duplicates.

The thing about REMOVE-DUPLICATES is that it works on fully general objects, which places limitations on its performance. If your objects are sortable or hashable it is generally better to use a custom function.

[Ramarren was faster than me, he answered while I was still typing ... ]

The :test and :test-not keyword parameters are intended to be used as either :test or :test-not but not both at the same time. Altough it sometimes works [it's unspecified and therefore implementation dependent if an error will be signalled or not you give both] the result will usually be very confusing.

I think the main reason why there are both :test and :test-not is that Lisp originally was designed by mathematicians and testing numerical results often requires heavy complicated :test functions and often it's easier to write a :test-not math-test to exclude the opposite of the desired numbers.

The typical Lisp solution for your problem is to use an anonymous lambda-function as :test or :test-not argument, so both solutions shown below will work:

(remove-duplicates "heyyyy cooooool aaaaaa"
                   :test #'(lambda (x y)
                             (and (char-equal x y)
                                  (char/= x #\space)
                                  (char/= y #\space))))

(remove-duplicates "heyyyy cooooool aaaaaa"
                   :test-not #'(lambda (x y)
                                 (or (char-not-equal x y)
                                     (and (char= x #\space)
                                          (char= y #\space)))))

The :from-end keyword parameter is not necessarily needed here and can be omitted.

More code examples to manipulate strings [subtype of sequence] can be found in:

• Cliki: SPLIT-SEQUENCE - download link for the code at the bottom of the page
• common-lisp.net: Alexandria, i.e. Section Sequences

- edgar

Okay, that all makes sense now. Thanks!

edgar-rft wrote: I think the main reason why there are both :test and :test-not is that Lisp originally was designed by mathematicians and testing numerical results often requires heavy complicated :test functions and often it's easier to write a :test-not math-test to exclude the opposite of the desired numbers.

If you had a function FOO, but not NOT-FOO, then if all you had was :TEST and wanted NOT-FOO, you'd have to define it specifically for this purpose or use a LAMBDA form to create it. Both of these work, but are a bit clumsy. :TEST-NOT makes this simple and clean.

Note that if you use :TEST-NOT a lot, then you might be better off defining NOT-FOO and using it with :TEST, for readability.