Toward a curriculum for ANSI Common Lisp

1Beginner

1.1Community

• Too many cooks spoil the broth.
• <2020-01-16> There are several "community" sites common-lisp.net and lisp-lang.net. The former seems to have more content, but the latter seems to be more recent? lisp-lang.net is on HackerNews. common-lisp.net is a static website that can be community-edited by sending Git patches/pull-requests? We have duplication because, it seems to me, every Common Lisp community has at least one toxic authority.
• It seems that comp.lang.lisp is toxic, uncivilized, and newcomer-unfriendly. It seems that Lisp-related Reddits are more civilized.
• https://github.com/exercism/common-lisp/issues/206
• On Reddit: r/Common_Lisp (~4k members), r/lisp (~26k members).
• Some ideas:
  • SWI-Prolog has a Discourse group. Common Lisp should follow suit.
• annoying glibc version change since sbcl 1.5.6 https://sourceforge.net/p/sbcl/mailman/message/36780098/

1.2SBCL

• Enable debugging to get stack traces?

1.3References

• Know that the reference is CLHS (Common Lisp HyperSpec). It can also be installed with apt install hyperspec in Debian 9 or 10.

1.4Dynamic and lexical variables

• Temporarily binding a dynamic (special) variable: (let ((*print-escape* t)) ...)
• Temporarily binding a lexical variable: (let ((foo t)) ...)
• (declare (special Name)): see example in CLHS

2Problems

• read can execute arbitrary code. Theoretically, Lisp is unparseable. Practically, people keep their sanity by not touching readtables.
• Cannot shadow built-in symbols.

(flet ((replace ...)) ...)

3Types

• (declare (type Type Var*)).
• See the list of type specifiers.
• But the interpreter is not required to check the types?

4Guides

• Practical Common Lisp book
• Steve Losh's A road to Common Lisp

5Setup

• rlwrap sbcl https://lispcookbook.github.io/cl-cookbook/getting-started.html
• SBCL + Emacs + SLIME (but is it secure?)
• ASDF (Another System Definition Facility), like npm's package.json
  • ASDF + Quicklisp ~ npm
• Quicklisp and Quickdocs
• Run rlwrap sbcl.
  • Suppress license banner with --noinform.
  • Load file with --load Path.
• Use ASDF to describe, but use Quicklisp to load. Create an asd file, but use (ql:quickload :asdf-system-name) instead of (asdf:load-system "asdf-system-name"). https://www.darkchestnut.com/2016/quicklisp-load-personal-projects-from-arbitrary-locations/
• Reader case-sensitivity is adjustable.

Some people have written getting-started guides: common-lisp.net.

<2020-01-16> common-lisp.net's library recommendation list seems to be taken from CodyReichert/awesome-cl. There is also Borretti 2015 consolidation. lisp-lang.org's list seems shorter.

quickdocs.org: centralized Common Lisp libraries documentation and popularity.

There are no GitHub trends¹ for Common Lisp!? Is CL that unpopular!?

https://stackoverflow.com/questions/33848241/most-recent-standard-of-common-lisp

https://common-lisp.net/project/cdr/final.html

https://www.cliki.net/Scribble

"Screamer provides a nondeterministic choice-point operator, a backtracking mechanism, and a forward propagation facility. "

No legalese? Why do all those important projects like ASDF and Quicklisp don't have any LICENSE information in their source code repository!?

6SLIME

• Open an inferior-lisp buffer with M-x slime.
• List keybindings with C-c C-d ?.
• Load file with C-c C-l.
• Load current buffer's file with C-c C-k.
• History with M-p and M-n.
• Restart inferior Lisp process with M-x slime-restart-inferior-lisp.
• Open symbol documentation with C-c C-d d.
• https://stackoverflow.com/questions/33003993/how-to-get-emacs-slime-sbcl-to-recognize-quicklisp-packages

7Strings

• Concatenate several strings: (concatenate 'string "foo" "bar" "baz").

8Compilation

• The compiler must at least do the minimal compilation.
• Programs must conform to some constraints in order to make compilation semantics = interpretation semantics.
• Understand the compilation of top-level forms and the compilation of defining macros.
• Compile time is before run time.
• Run time spans from load time to execution time.
• "At compile time, only the compilation environment and the evaluation environment are available."
• "At run time, only the run-time environment is available."
• "The compilation environment inherits from the evaluation environment, […]."
• "The evaluation environment inherits from the startup environment, […]."
• The not-compile-time mode means that the form is evaluated at load time.
• The compile-time-too mode means that the form is evaluated at both compile time and load time.
• "eval-when forms cause compile-time evaluation only at top level."

9Timing, coarse profiling

• TIME: (time FORM)

10Naming conventions

• (defconstant +Name+ Value Docstring)
• (defvar *Name* Value Docstring)

11Gensyms

Gensyms are theoretically disgusting but practically amazing.

Gensyms require that programmers don't name their variables like gensyms (G<number>).

Do not call gensym too much. Assume that they are not garbage collected.

It is unknown whether gensym is thread-safe. ANSI CL does not specify concurrency.

12Conditions, exceptions, restarts

• http://www.gigamonkeys.com/book/beyond-exception-handling-conditions-and-restarts.html
• cerror signals a correctable/continuable/restartable error?
• catch and throw
• Should we use throw or signal?
• unwind-protect, like Java's try-finally.

13Projects

13.1Writing ASDF systems

• Write an asd file.
  • Write a defsystem form.
  • See also the defsystem grammar.
• Quickproject? http://articulate-lisp.com/project/new-project.html
• http://notes.eatonphil.com/starting-a-minimal-common-lisp-project.html

13.2Packages

• defpackage
• Use uninterned symbols for names in defpackage.
• in-package

Tricky stuff:

(defpackage my-package) ;; wrong
(defpackage :my-package) ;; less OK
(defpackage #:my-package) ;; OK

The first causes an unintended interning in the *package* (usually common-lisp-user) when the defpackage form is being read.

The second adds noise to SLIME autocompletion? SLIME does not auto-complete uninterned symbols?²

https://stackoverflow.com/questions/7894548/why-are-uninterned-symbols-used-for-package-names-and-exports-in-common-lisp

:export + use-package + uninterned symbols = conflict.

(defpackage temp (:export a))
(use-package 'temp) ;; conflicts because A is already interned in *PACKAGE* when the above DEFPACKAGE was read.

This uses keywords http://www.gigamonkeys.com/book/programming-in-the-large-packages-and-symbols.html

Should we use :keywords or #:uninterned-symbols?

https://stackoverflow.com/questions/46981477/variations-in-invocation-of-defpackage-and-in-package

Is an interned symbol never garbage-collected?

Tricky stuff: If read interns symbols, and symbols are never garbage-collected, then we can make the interpreter run out of memory by making it read too many distinct symbols?

13.3Show dependency graph as tree

• asdf-viz? But we want text output.
• (ql-dist:dependency-tree SYSTEM) does not work for non-quicklisp libraries:

Quicklisp has an idea about the dependencies of Quicklisp-provided systems, but not of any other systems available through ASDF.

14DEFSTRUCT vs DEFCLASS

• CLHS: DEFSTRUCT
• defstruct vs defclass is record vs class.
• defstruct defines a product type.
• https://stackoverflow.com/questions/34244855/difference-between-struct-and-class-in-common-lisp

15CLOS (Common Lisp Object System)

• defclass
• defgeneric
• defmethod
• make-instance

16FORMAT language

• Format to standard output with (format t Format Arg...).
• Format to string with (format nil Format Arg...).
• ~a
• ~s
• ~% newline.

http://www.lispworks.com/documentation/lw50/CLHS/Body/22_c.htm

17LOOP language

http://www.lispworks.com/documentation/HyperSpec/Body/06_aac.htm

• Simple loop form
  • Loop with (loop Form ...) where each Form is a compound form (list form).
    • Break with (return) or (return Value), similar to C break.
  • Loop forever like (loop (princ "A")).
• (loop repeat Count do ...)
• (loop for Var from Lower to Upper Action*)
• Summary of Value Accumulation Clauses
• Summary of Termination Test Clauses
• Order of Execution
• Destructuring
• Build a list.
• Loop with several variables.
• Know the pitfall: finally seems to be executed only if the loop body is terminated by for and not by return, always, never, etc.

(loop for i from 0 to 9
      if Cond
        if Cond
          do Form ...
        end
      end)

From http://www.lispworks.com/documentation/HyperSpec/Body/06_aha.htm

(loop for i from 0 to 9
      if (evenp i)
        collect i into evens
      else
        collect i into odds
      end
      finally (return (list evens odds)))
=> ((0 2 4 6 8) (1 3 5 7 9))

(loop for i from 0
      for j in '(a b c)
      collect (list i j))
=> ((0 A) (1 B) (2 C))

Syntax.

repeat X translates to for Gensym in 0 to (- X 1)

18MATCH languages

• Know some trivia MATCH patterns.
• Use EMATCH instead of MATCH, because, if no patterns match, the former raises an error and the latter returns NIL.

19Web application development

HTTP is too low-level for web application development.

"UnCommon Web provides developers with the illusion that web pages are nothing more than function calls."

20Parsing

Meta, parsing on Lisp https://www.cliki.net/Meta

21Tricky stuff

21.1eval vs load

• "SBCL processes initialization files with read and eval, not load" http://www.sbcl.org/manual/#Initialization-Files

21.2Keyword vs symbol

https://stackoverflow.com/questions/23969203/what-is-the-difference-between-a-keyword-symbol-and-a-quoted-symbol

In the toplevel:

• 'a evaluates to a symbol in the package common-lisp-user.
• :a evaluates to a symbol in the package keyword; this package is special.

See also: symbol-package.

21.3Equality comparison

In mathematics, X and Y are equal iff every X can be replaced with Y without changing the truth value of the containing statement.

Two things are identical iff …

In Common Lisp:

• eq is identity comparison.
• eql is eq or something? eql is the default for make-hash-table.
• equal is about structural similarity / isomorphism?
• equalp is equal or something?
• =
• string=
• char=

In Scheme:

• eq? is identity comparison.
• eqv? is ???
• equal? is deep/recursive comparison?
• = is numeric comparison?

21.4Nil vs false

https://www.google.com/amp/s/lispchronicles.wordpress.com/2017/03/16/the-truth-about-nothing/amp/

21.5Printing

• prin1 for machines (read).
• princ for humans.
• pprint for pretty-printing.
• Avoid write and print?

21.6macro-function, symbol-function, fdefinition, apply, funcall

• (setf (macro-function 'foo) ...)
• Understand FUNCTION vs CLOSURE.
  • Closure = Function + Lexical environment.
• #' (function) can access lexical environment; symbol-function can only access global environment (what about dynamic environment?)
• Expand macro form with MACROEXPAND.
• Evaluate macro form with EVAL.

21.7map

• Common Lisp map takes 3 arguments and works with lists and vectors.
• Scheme map takes 2 arguments and works with lists only.

22Writing libraries

• To avoid name clash, do not define package nicknames. https://www.reddit.com/r/lisp/comments/lcb09/package_nickname_conflicts/

23Some project ideas

23.1Clash-free file-local packages

Key idea: a file-local package generates its name from the file's truename (canonical path).

23.2Common Lisp Plus

• Create a language Lisp1 on top of Common Lisp (CL), that translates to CL, and fully interoperates with CL.
  • Reuse CL read and with-standard-io-syntax.
  • (my-read Stream) => My-File-Content
  • (translate My-File-Content) => CL-File-Content
  • Output with pprint.
  • 1:1 mapping between CL+ file and CL file.
  • Convenience method: (translate-my-file :from Src :to Dst).
  • Generate ASDF defsystem.
  • Be comfortable, allow programmers to underspecify, but enable programmers to inform the implementation to translate the program to a fast implementation.
  • Problems:
    • One name may have multiple context-dependent meanings.
    • A formal language requires that the programmer be explicit with context: (with-context Ctx Form...)
  • Compare and consider:
    • SRFI 83 (R6RS library syntax). Separate compilation. Parseability.
    • Chez Scheme module system.
    • Haskell module system.
    • Standard ML module system.
    • TypeScript module system.
    • C++ namespaces.
    • https://www.sacrideo.us/a-philosophy-on-scheme-modules/
  • Minimal additions; I just want a one-file-one-package package system. But doesn't ASDF already have that? https://common-lisp.net/project/asdf/asdf/The-package_002dinferred_002dsystem-extension.html
  • But I want (require File) like TypeScript. I don't want to repeat any package prefix.
  • For ASDF interoperation, see also https://common-lisp.net/project/asdf/asdf/How-do-I-work-with-readtables_003f.html

23.3Scheme on Common Lisp

23.4Emacs Lisp on Common Lisp

• Emacs Lisp (EL) to Common Lisp (CL) embedding: translator or perhaps interpreter?
  • Reuse CL read, manipulate readtable.
  • (set-macro-character C #'emacs-lisp-on-common-lisp:read t) for each character C
  • See also pitfalls and porting Common Lisp to Emacs Lisp

23.5Lisp "format" but more user-friendly?

format should use proper forms instead of format strings.

(format Fragment ...)

A string fragment is printed literally.

(display X) is Racket ~a.

(write X) is Racket ~s.

(print X) is Racket ~v.

newline is Racket ~n.

(max-width L F) is F but its width is truncated to L characters

(fixed-width L F).

See also: Racket fprintf

(format "Foo" (write 1) (print 2) newline)

23.6Racket syntax objects in Common Lisp and Scheme?

Racket's syntax object represents abstract syntax and not concrete syntax. If it had concrete syntax object, it would enable refactoring tools.

A Lisp should come with its own parser (about concrete syntax, not only about abstract syntax).

24Advanced?

• AP5? Datalog? https://www.reddit.com/r/programming/comments/9b9mq/lisp_prolog_model_driven_dev_aop_from_1989_please/
• Clojure contender? https://chriskohlhepp.wordpress.com/advanced-c-lisp/embedding-lisp-in-cplusplus-a-recipe/
• PowerLoom; model-driven? https://www.isi.edu/isd/LOOM/PowerLoom/documentation/documentation.html

25Other Lisps

• Scheme vs Common Lisp
• Hyperpolyglot's inter-Lisp comparison table may be incomplete but may help porting.
• Dybvig 2006 tells the history of Chez Scheme.
• Pitman 1980 argues that fexprs preclude compilation.
• Gerbil/Gambit Scheme process migration?³

26Namespaces?

https://www.emacswiki.org/emacs/Namespaces

"Debuggable" https://endlessparentheses.com/introducing-names-practical-namespaces-for-emacs-lisp.html

---

1. <2020-01-16> https://github.com/trending/common-lisp↩

2. https://www.reddit.com/r/learnlisp/comments/7a48i7/foo/dp7gr7i?utm_source=share&utm_medium=web2x↩

3. <2020-01-15> "Why I haven't jumped ship from Common Lisp to Racket (just yet)?" https://fare.livejournal.com/188429.html↩