Common Lisp Tips

Formatting integers in different radixes

format has four directives for formatting integers with different radixes:

• ~B formats as binary: (format nil "~B" 42) => "101010"
• ~O formats as octal: (format nil "~O" 42) => "52"
• ~X formats as hexadecimal: (format nil "~X" 666) => "29A"
• ~R formats with an arbitrary radix between 2 and 36: (format nil "~36R" 18321) => "E4X"

Each directive takes padding and pad-character options (as well as other options). I define these functions in my init files to quickly dump out hex and bit views of integers:

(defun :hex (value &optional (size 4))
  (format t "~v,'0X" size value))

(defun :bits (value &optional (size 8))
  (format t "~v,'0B" size value))
  
* (:bits 42)
00101010

* (:hex 666)
029A

The lower-level write function accepts a :base argument that specifies a radix to use when printing integers. The default value is taken from *print-base*. For example, (write-to-string 65535 :base 16) => "FFFF". Since write underlies how other standard functions produce output, binding *print-base* will affect pretty much everything.

Literal syntax for integers

There are several ways to write literal integers with different radixes in Common Lisp. #b… is for binary, #o… is for octal, #x… is for hexadecimal, and #r is for an arbitrary radix from 2 to 36. Section 2.4.8.10 has this example:

#2r11010101  ;Another way of writing 213 decimal  
#b11010101   ;Ditto                               
#b+11010101  ;Ditto                               
#o325        ;Ditto, in octal radix               
#xD5         ;Ditto, in hexadecimal radix         
#16r+D5      ;Ditto                               
#o-300       ;Decimal -192, written in base 8     
#3r-21010    ;Same thing in base 3                
#25R-7H      ;Same thing in base 25               
#xACCEDED    ;181202413, in hexadecimal radix

How do I convert an integer to a list of bits?

Novices sometimes ask how to get a convert an integer to a list of bits, often for the purpose of iterating over the bits somehow. Common Lisp has a rich set of functions for directly accessing the bits of an integer, so constructing an intermediate list of bits is rarely necessary.

Here are a few examples of accessing the bits of an integer.

With integer-length and logbitp to test the bit at a particular index:

(defun list-of-bits (integer)
  (loop for index below (integer-length integer)
        collect (if (logbitp index integer) 1 0)))

With ash to shift the integer to the right and logand to test the least-significant bit:

(defun list-of-bits (integer)
  (loop repeat (integer-length integer)
        for i = integer then (ash i -1)
        collect (logand i 1)))

With ash to shift a one-bit mask to the left and logtest to test the mask bit against the integer:

(defun list-of-bits (integer)
  (loop repeat (integer-length integer)
        for mask = 1 then (ash mask 1)
        collect (if (logtest mask integer) 1 0)))

With byte to construct a byte specifier and ldb to extract a field from the integer:

(defun list-of-bits (integer)
  (loop for position below (integer-length integer)
        collect (ldb (byte 1 position) integer)))

update Unfortunately, all these examples are buggy - they collect bits in the wrong order. I used loop/collect to avoid having to reverse the results, but didn’t think it through properly. Here are some updated definitions with dotimes that give the results in the intended order:

(defun list-of-bits (integer)
  (let ((bits '()))
    (dotimes (index (integer-length integer) bits)
      (push (if (logbitp index integer) 1 0) bits))))

(defun list-of-bits (integer)
  (let ((i integer)
        (bits '()))
    (dotimes (j (integer-length integer) bits)
      (push (logand i 1) bits)
      (setf i (ash i -1)))))

(defun list-of-bits (integer)
  (let ((mask 1)
        (bits '()))
    (dotimes (i (integer-length integer) bits)
      (push (if (logtest mask integer) 1 0) bits)
      (setf mask (ash mask 1)))))

(defun list-of-bits (integer)
  (let ((bits '()))
    (dotimes (position (integer-length integer) bits)
      (push (ldb (byte 1 position) integer) bits))))

The optional arguments of deftype

deftype can be used to create user-defined types that expand into built-in types. For example:

(deftype octet-vector (length)
  `(simple-array (unsigned-byte 8) (,length)))

Given this deftype, in a type context, (octet-vector 32) expands into (simple-array (unsigned-byte 8) (32)), or a one-dimensional octet array of length 32.

Lambda lists for deftype are very similar to lambda lists for defmacro, with an important difference: where optional arguments to defmacro with no initialization form default to nil, the default value in deftype is the symbol *. The deftype above could be rewritten to take advantage of this:

(deftype octet-vector (&optional length)
  `(simple-array (unsigned-byte 8) (,length)))

Now, in a type context, a plain octet-vector expands into (simple-array (unsigned-byte 8) (*)), or a one-dimensional octet array of indeterminate length, and (octet-vector 32) expands into (simple-array (unsigned-byte 8) (32)).

For more information, see section 3.4.8, Deftype Lambda Lists.

Trying again with with-simple-restart

I have some code that reads data from a config file. If there’s a problem loading, I’d like the opportunity to fix it, outside of Lisp, and retry loading. It’s easy to do that with with-simple-restart and loop:

(loop 
  (with-simple-restart (try-again "Try again")
    (return 
     (progn 
       (setf *config* (load-config-file))))

If any error occurs during load-config-file, I can either fix up the file and choose the try-again restart, or give up and use an abort restart.

(Thanks to Frode Vatvedt Fjeld for the concise idiom.)

Semicolon style

The standard explains common semicolon comment style in section 2.4.4.2. Section 2.4.4.2.5 includes this short example showing typical use of one through four semicolons:

;;;; Math Utilities

;;; FIB computes the the Fibonacci function in the traditional
;;; recursive way.

(defun fib (n)
  (check-type n integer)
  ;; At this point we're sure we have an integer argument.
  ;; Now we can get down to some serious computation.
  (cond ((< n 0)
         ;; Hey, this is just supposed to be a simple example.
         ;; Did you really expect me to handle the general case?
         (error "FIB got ~D as an argument." n))
        ((< n 2) n)             ;fib[0]=0 and fib[1]=1
        ;; The cheap cases didn't work.
        ;; Nothing more to do but recurse.
        (t (+ (fib (- n 1))     ;The traditional formula
              (fib (- n 2)))))) ; is fib[n-1]+fib[n-2].
              

The tree-walkers of CL

If you need to visit a tree in some way, there are two built-in functions that, while not explicitly designed for the purpose, can walk a tree effectively.

First, subst-if takes a predicate function that is called for each cons and atom in a tree. As long as the predicate returns nil, no substitution will actually take place.

Here’s a simple way to turn it into a function:

(defun walk-tree (fun tree)
  (subst-if t
            (constantly nil)
            tree
            :key fun))

* (walk-tree 'print '(a b (1 2 . 10) c))

(A B (1 2 . 10) C) 
A 
(B (1 2 . 10) C) 
B 
((1 2 . 10) C) 
(1 2 . 10) 
1 
(2 . 10) 
2 
10 
(C) 
C 
NIL 
(A B (1 2 . 10) C)

Second, tree-equal is normally considered for comparing two trees, but it can also be used to call a function for each atom in a single tree; just pass the same tree to tree-equal, and use the test function to do something with each atom. As long as the test function returns true, the walking continues.

Here’s a function that wraps it up:

(defun walk-tree-atoms (fun tree)
  (tree-equal tree tree
              :test (lambda (element-1 element-2)
                      (declare (ignore element-2))
                      (funcall fun element-1)
                      t)))

* (walk-tree-atoms 'print '(a b (1 2 . 10) c))

A 
B 
1 
2 
10 
C 
NIL 
T

(This tip inspired by a few articles from Rob Warnock.)

Duplicated keyword arguments

If a function call has two pairs of keyword arguments with the same keyword, the leftmost argument pair is used. This can be helpful if a list of keyword arguments is received and you want to selectively override some value in that argument list in application. There’s no need to use mutation or produce a fresh variation of the list.

For example:

(defun write-escaped (object &rest args &key &allow-other-keys)
  (apply 'write object :escape t args))

* (write "hello" :escape nil)
hello

* (write-escaped "hello" :escape nil)
"hello"

See 3.4.1.4 for details.

The four causes of symbol conflicts

A symbol conflict arises when a package operation would result in two distinct symbols with the same name being accessible in a single package. There are exactly four ways this can happen.

Inheriting: When you use (via use-package or the defpackage :use clause) a package that exports a distinct symbol sharing the same name with a symbol already accessible in the using package. The conflict can arise between two inherited symbols, or between an inherited symbol and a present symbol.

Importing: When you import a symbol and a distinct symbol with the same name is already accessible in the target package.

Exporting: When you export a symbol from a package and a distinct symbol with the same name is already accessible in any package that uses that package.

Uninterning: When you unintern a shadowing symbol and the absence of its shadowing results in two distinct symbols with the same name being accessible via inheritance.

Here are short examples of each case. They all assume an environment in which this package definition is in effect:

(defpackage #:auto
  (:documentation "An automotive package.")
  (:use)
  (:intern #:tan)
  (:export #:car))

Inheritance conflict:

(defpackage #:inheritance-conflict
  (:use #:cl #:auto))

=> USE-PACKAGE #[PACKAGE "COMMON-LISP"] causes name-conflicts in
#[PACKAGE "INHERITANCE-CONFLICT"] between the following symbols:
  AUTO:CAR, COMMON-LISP:CAR
   [Condition of type NAME-CONFLICT]
See also:
  Common Lisp Hyperspec, 11.1.1.2.5 [:section]

Import conflict:

(defpackage #:import-conflict
  (:use #:cl))

(import '(auto:car) '#:import-conflict)

=> IMPORT AUTO:CAR causes name-conflicts in
#[PACKAGE "IMPORT-CONFLICT"] between the following symbols:
  AUTO:CAR, COMMON-LISP:CAR
   [Condition of type NAME-CONFLICT]
See also:
  Common Lisp Hyperspec, 11.1.1.2.5 [:section]

Export conflict:

(defpackage #:export-conflict
  (:use #:cl #:auto)
  ;; Avoid inheritance conflict with shadowing
  (:shadow #:car))

(export 'auto::tan '#:auto)

=> EXPORT AUTO::TAN causes name-conflicts in
#[PACKAGE "EXPORT-CONFLICT"] between the following symbols:
  AUTO::TAN, COMMON-LISP:TAN
   [Condition of type NAME-CONFLICT]
See also:
  Common Lisp Hyperspec, 11.1.1.2.5 [:section]

Unintern conflict:

(defpackage #:unintern-conflict
  (:use #:cl #:auto)
  ;; Avoid inheritance conflict with shadowing
  (:shadow #:car))

(unintern 'unintern-conflict::car '#:unintern-conflict)

=> UNINTERN UNINTERN-CONFLICT::CAR causes name-conflicts in
#<PACKAGE "UNINTERN-CONFLICT"> between the following symbols:
  AUTO:CAR, COMMON-LISP:CAR
   [Condition of type NAME-CONFLICT]
See also:
  Common Lisp Hyperspec, 11.1.1.2.5 [:section]

If you use Common Lisp, you have to interact with the package system. I think the best way to get along with the package system is not to keep it at arm’s length while holding your nose, but to learn how it works, inside and out.

A little bit of file-position

When used with one argument, file-position returns an integer representing the stream’s file position:

* (defvar *s* (open "data.bin" :element-type '(unsigned-byte 8))
=> *S*

* (read-byte *s*)
=> 42

* (read-byte *s*)
=> 107

* (file-position *s*)
=> 2

When used with two arguments, file-position sets the stream position:

* (file-position *s* 1)
=> T

* (read-byte *s*)
=> 107

When used this way, the position is often given as an integer, but the function actually accepts a designator. An integer position value represents itself, the keyword :start represents 0 (the first position in the stream), and the keyword :end represents the last position in the stream.

What would life be like without :start and :end? For :start, not too difficult; you could just use 0. But without :end you would need to use some other function, such as file-length, to correctly position the stream at the end.