DEFSTRUCT POINT

The following will work in CL and I believe elisp.

(psetf (point-x p) (point-y p)
       (point-y p) (point-x p))

If not, you may need to use an explicit temporary variable. Something like

(let ((x (point-x p)))
  (setf (point-x p) (point-y p)
        (point-y p) x))

A very old trick to swap values is:

(defmacro swap (place1 place2)
  `(setf ,place1 (prog1 ,place2 (setf ,place2 ,place1))))

Or with points:

(setf (point-x p) (prog1 (point-y p) (setf (point-y p) (point-x p))))

This has the same effect as PSETF, and needs no explicit temporary variable.