enhanced-do

NAME
SYNOPSIS
DESCRIPTION
EXAMPLES
CONTACT

NAME

enhanced-do - Similar to standard DO and DO* but with a lot of additional functionality

SYNOPSIS

(use-package "enhanced-do")

Macro edo:
(edo ([name [SOED]]) [(:user-info data*)] (clause*) (term-cond result-form*) declaration* body)

name is a symbol. SOED stands for "sequence of events designator" and is one of :BUC, :BCU, :CBU, :CUB, :UBC, :UCB {i.e. all the possible permutations of B C U} or :D. data are arbitrary Common Lisp {CL} objects.
The syntax for a clause is
{
var | ([:inh] form [init-form [update-form]])}

var is a symbol; form is a SETF-able form i.e. what the CLHS calls "place" {see paragraph 5.1.1}; init-form and update-form are arbitrary CL forms.
term-cond
and result-form are arbitrary CL forms. declaration is a DECLARE expression.

The syntax for body is
{
tag | statement}*
where tag is a GO tag; statement is any CL form.

Macro edo* has the same syntax as edo .

Macro edo-for:
(edo-for ([name [SOED]]) [(:user-info data*)] [let-form] (init-form*) ([term-cond result-form*]) (update-form*) body)

name, SOED, data, body have the same syntax as with edo .init-form, term-cond, result-form, update-form are arbitrary CL forms. Note that term-cond is optional {the parentheses around {it and result-form} are not} whereas the corresponding form of edo must exist.
The syntax for let-form is
({:let | :let*} {var | (var [init-form])}* declaration*)
where var is a symbol, init-form any CL form and declaration a DECLARE expression. Unlike the syntax of standard LET ,here the list of variables does not appear inside a pair of parentheses; instead the macro assumes that the list of variables has ended upon either reaching the end of the let-form or a cons whose car is the symbol DECLARE of package COMMON-LISP .Within a let-form every form which follows a declaration must also be a declaration.

Macro forever:
(forever ([name [SOED]]) [(:user-info data*)] body)

name, SOED, data, body have the same syntax as with edo .

Macros edo-cont, edo-break, edo-exit:
(edo-cont [event [loop-designator]])
(edo-break [loop-designator])
(edo-exit [retval [loop-designator]])
where event is one of 1, 2, 3, :B, :B+, :B-, :C, :C+, :C-, :U, :U+, :U-, loop-designator is a symbol or an integer >= 0 and retval is a form; retval is evaluated, the rest not.

Function edo-user-info:
(edo-user-info &key env extended-info? loop-designator)

env is an environment object {see 3.1.1.4 in CLHS}, extended-info? a generalised boolean, loop-designator a symbol or an integer >= 0.

Symbol macro edo-version:
It expands into a list of 2 elements the first of which is the major version number and the second the minor version number of this package. The list is a quoted object hence it must not be modified.

DESCRIPTION

Everything which needed to be said about edo-version has been covered above so what follows will not refer to it. From now on LEM {loop establishing macro} will refer to edo , edo* , edo-for , forever and LCM {loop choosing macro} will refer to edo-cont , edo-break , edo-exit .

DEFAULTS

For name NIL , for SOED :CBU , for event :B+ , for loop-designator 0 , for env NIL , for retval NIL , for extended-info? NIL .

SCOPES

Each LEM may create several scopes. It creates an implicit block with name name; everything which follows the form which contains name is within this block. A RETURN-FROM referring name may be used to exit the loop.

An edo creates new lexical bindings as if with LET ; everything which follows the list of clauses, as well as the update-form's within each clause, are in the scope of those bindings; the scope of the declaration's {if such exist} is the same as the scope of the lexical bindings much like it works with a LET immediately followed by declarations. An edo* creates lexical bindings as if with LET* .

An edo-for which contains a let-form creates lexical bindings as if with LET or LET* ,depending on whether the let-form begins with :LET or :LET* respectively. Everything which follows the let-form is within the scope of those bindings.

Each LEM creates an implicit tagbody which contains body, term-cond, result-form's and update-form's. A GO tag will only be recognised if it is top level within body so for example a GO cannot be used to transfer control from body to a result-form but can be used to transfer control from a result-form to body. The latter is explicitly not permitted with the standard DO .Consider the following code:

    (edo () ((i 1 (+ i 1)))
            ((or (eql i 3) (eql i 5))
                (when (eql i 3) (format t "Not done yet~%") (go again)))
        (format t "i is ~a~%" i)
        again)
prints
    i is 1
    i is 2
    Not done yet
    i is 4

Each LEM has an associated edoloop scope {ES} which is a subset of the LEM's lexical scope. A point in the code is said to be edoloop-enclosed by a LEM if it appears in the ES of the LEM. For edo and edo* the ES contains everything which follows the list of clauses as well as update-form within each clause. For edo-for the ES contains everything from init-form until the end. For forever the ES contains body.

SEQUENCE OF EVENTS

Every LEM apart from forever starts with creating bindings and doing initialisations {if its invocation has the corresponding expressions}. For edo and edo* this is covered in more detail in the CLAUSES subsection below; edo-for with a let-form creates bindings as if by LET or LET* and then init-form's get evaluated as an implicit progn; forever does not create bindings.

Then the looping sequence of events starts. The "events" referred to are B for evaluating the body forms, C for evaluating term-cond and U for evaluating the update-form's. For edo and edo* the update-form's are evaluated left to right and the new assignments are done as if by PSETF for edo and as if by SETF for edo* .For edo-for update-form is an implicit progn. forever does not have a terminating condition or update forms so the body simply gets repeated again and again.

The SOED determines in which order the events will happen at every repetition of the loop; they will continue to happen in the same cyclic fashion until normal termination happens {see below} or an explicit transfer of control. The SOED of a forever invocation does not affect behaviour but will still be checked for validity. A SOED of :D causes the same sequence of events as a SOED of :CBU .

Here's an example:

    (edo-for (nil :CBU) (:let i)
             ((format t "Init~%") (setq i 0))
             ((progn (format t "Term cond~%") (< i 3))
                  (format t "Result form~%"))
             ((format t "Updating~%") (incf i))
        (format t "Body : i is ~s~%" i))
prints
    Init
    Term cond
    Body : i is 0
    Updating
    Term cond
    Body : i is 1
    Updating
    Term cond
    Body : i is 2
    Updating
    Term cond
    Result form

.An omitted SOED or one replaced by :D would give the same output. But if we change :CBU to :UCB we get

   Init
   Updating
   Term cond
   Body : i is 1
   Updating
   Term cond
   Body : i is 2
   Updating
   Term cond
   Result form

NORMAL TERMINATION

Normal termination for edo and edo* happens when term-cond becomes true; for edo-for when it becomes false; with all 3 the result-form's will then be evaluated as an implicit progn and their return value will be the return value of the LEM. If no result-form's exist then the return value is NIL .

An edo-for without a term-cond does not have normal termination i.e. it will loop indefinitely until an explicit transfer of control happens; same with forever .

edo AND edo* CLAUSES

A clause will be called inherited {IC} if it's a list and the first element of the list is either :inh or a cons; otherwise it will be called regular clause {RC}. A RC establishes a new binding whereas an IC refers to a binding existing in the environment. For example:

    (let ((k 16) (j nil) (i 123) (c (cons nil nil)))
        (format t "Before the loop : k is ~a    j is ~a    ~
                   i is ~a    c is ~a~%" k j i c)
        (edo () ((i 19 (- i 1))              ; Regular clause
                 (:inh j 64 (/ j 2))         ; Inherited clause
                 k                           ; Regular clause
                 ((car c) 5 (+ 5 (car c))))  ; Inherited clause
                ((equalp c '(15 . nil))
                    (format t "In a result form ~
                               : k is ~a    j is ~a    ~
                               i is ~a    c is ~a~%" k j i c))
            (format t "Inside the loop : k is ~a    j is ~a    i is ~a    ~
                       c is ~a~%" k j i c))
        (format t "After the loop : k is ~a    j is ~a    ~
                   i is ~a    c is ~a~%" k j i c))
prints
    Before the loop : k is 16    j is NIL    i is 123    c is (NIL)
    Inside the loop : k is NIL    j is 64    i is 19    c is (5)
    Inside the loop : k is NIL    j is 32    i is 18    c is (10)
    In a result form : k is NIL    j is 16    i is 17    c is (15)
    After the loop : k is 16    j is 16    i is 123    c is (15)

In an IC if init-form has the value :no-init then the form will not be assigned to when the initialisations and bindings happen; if you do actually want to assign the value use ':no-init .For example:

    (let ((i 1) (j 2))
        (edo () ((:inh i :no-init (+ i 1))   ; i will not be assigned to
                 (:inh j ':no-init (+ j 2))  ; j gets the value :no-init
                 (k :no-init (+ k 3)))       ; RC so no special meaning for :no-init
                ((> i 3))
            (format t "i is ~s    j is ~s    k is ~s~%" i j k)
            (when (symbolp j) (setq j 0))
            (when (symbolp k) (setq k 8))))
prints
    i is 1    j is :NO-INIT    k is :NO-INIT
    i is 2    j is 2    k is 11
    i is 3    j is 4    k is 14

The init-form's in the clauses get evaluated in a left to right order regardless of whether the clauses are ICs or RCs. For example:

    (let ((j nil) (c (cons nil nil)))
        (edo () ((i (progn (format t "Binding i~%") 9) (- i 1))
                 (:inh j (progn (format t "Setting j~%") 1) (+ j 1))
                 (k (progn (format t "Binding k~%") 7) (- k 2))
                 ((car c) (progn (format t "Setting car of c~%") 27)
                          (+ 5 (car c))))
                ((eql i 5))
            (format t "i is ~a    j is ~a    k is ~a    c is ~a~%" i j k c))
        (format t "j is ~a    c is ~a~%" j c))
prints
    Binding i
    Setting j
    Binding k
    Setting car of c
    i is 9    j is 1    k is 7    c is (27)
    i is 8    j is 2    k is 5    c is (32)
    i is 7    j is 3    k is 3    c is (37)
    i is 6    j is 4    k is 1    c is (42)
    j is 5    c is (47)

In each clause, every appearance at any depth of :@ in the init-form or update-form will be replaced by the form of the same clause. For example

    (let ((c (cons 1 2)))
        (edo () ((long-variable-name 1 (+ 1 (* 3 long-variable-name)))
                 ((car c) (* 2 (car c)) (* 3 (car c))))  ; Clauses end here
                 ....... ))
can be written more succinctly as
    (let ((c (cons 1 2)))
        (edo () ((long-variable-name 1 (+ 1 (* 3 :@)))
                 ((car c) (* 2 :@) (* 3 :@)))  ; Clauses end here
                 ....... ))

Note that every appearance of :@ at any depth will be replaced. Therefore if you actually want to refer to :@ itself then you have to assign it to a variable; quoting it won't work.

    (let ((colon-at :@))
        (edo () ((i 1 (+ 1 i))
                 (var ':@)
                 (var2 colon-at))
                ((eql i 2))
            (format t "i is ~s    var is ~s    var2 is ~s~%" i var var2)))
prints
    i is 1    var is VAR    var2 is :@

When the form of a clause is a compound form, it will be evaluated when the initial bindings and assignments for the clauses happen and at every repetition of the loop. For example:

    (defun (setf car-with-side-effect) (newval cons)
        (format t "This is a side effect~%")
        (rplaca cons newval)
        newval  ; See 5.1.2.9 in CLHS
    )

   (let ((c (cons 1 2)))
        (edo () (((car-with-side-effect c) 3 (+ 1 (car c))))
                ((equalp c '(5 . 2)))
            (format t "c is ~s~%" c)))
prints
    This is a side effect
    c is (3 . 2)
    This is a side effect
    c is (4 . 2)
    This is a side effect

NOTE

There are only 2 differences between edo and edo* : how they establish their bindings, as if by LET vs as if by LET* ,respectively; and how they update their forms, as if by PSETF vs as if by SETF ,respectively.

LOOP DESIGNATORS

A loop designator chooses a LEM which edoloop-encloses a point A in the code. If the loop designator is an argument to a LCM then A is the point where the LCM invocation itself appears whereas for edo-user-info it is the point where the env argument passed to edo-user-info was obtained {the examples further down will make this clear}; if env is NIL then it's as if there are no LEMs which are edoloop-enclosing.

If the loop designator argument is a symbol then the LEM chosen is the innermost one which edoloop-encloses point A and whose name is EQ to the symbol; if loop designator is a number n then the LEMs which edoloop-enclose A will be numbered starting from the innermost which gets number 0 and the n-th one will be chosen. So for example if there are m edoloop-enclosing LEMs and you want to choose the outermost by number rather than by name you should pass m-1 as a loop designator argument. The examples will show how this can be done without counting by hand.

In the case of a LCM, if a LEM cannot be found according to these rules, an error will be signalled.

LCMs

An invocation of a LCM alters the flow of control.

edo-exit returns as if by RETURN-FROM from the LEM chosen by its loop-designator argument with return value retval .

edo-break transfers control as if by GO to the result-form's of the LEM chosen by its loop-designator argument; if no such result-form's exist it's as if there was the single result form (PROGN) .If the LEM chosen is a forever then edo-break behaves like edo-exit with retval NIL .

edo-cont transfers control as if by GO to an event of the LEM chosen by the loop-designator argument. The event is chosen based on the event argument passed to edo-cont and possibly also the SOED of the chosen loop. For an event of :B , :C , :U ,the event chosen will be body, condition, update, respectively. If event is :B+ , :C+ , :U+ then the event chosen will be the event following body, condition, update, respectively according to the SOED of the chosen LEM and taking into account the fact that events happen in a cyclic manner. If event is :B- , :C- , :U- then the event chosen will be the event right before body, condition, update, respectively according to the SOED of the chosen LEM. Finally, if event is 1 , 2 , 3 then the event chosen will be the 1st , 2nd , 3rd, respectively according to the SOED of the chosen LEM. So for example, if the loop chosen has SOED :UBC then (edo-cont ARG ...) jumps to an event of the loop as follows:

   ARG      Event

   :B      body; SOED irrelevant
   :C      condition; SOED irrelevant
   :U      update; SOED irrelevant
   :B+     condition
   :C+     update
   :U+     body
   :B-     update
   :C-     body
   :U-     condition
   1       update
   2       body
   3       condition

Loop designators only know about blocks established by LEMs. Contrast the following 2 pieces of code:

    (edo (nice-block :BCU) () (t)
        (format t "AAA~%")
        (block nice-block (return-from nice-block nil))
        (format t "BBB~%"))
prints
    AAA
    BBB
whereas
    (edo (nice-block :BCU) () (t)
        (format t "AAA~%")
        (block nice-block (edo-exit nil nice-block))
        (format t "BBB~%"))
prints
    AAA

As implied by the SCOPES subsection, the init-form's of edo and edo* clauses are not in the ES of the corresponding macro. So for example the following

    (let ((i (some-computation)))
        (edo (block) ((j (if (> i 0) (edo-exit nil block) (+ i 1))))
                      ...... ))

will signal an error because the edo-exit does not appear in the ES of the edo .But you could instead write

    (let ((i (some-computation)))
        (edo (block) ((j (if (> i 0) (return-from block nil) (+ i 1))))
                      ...... ))

The reason for this restriction is that ESs also apply to edo-cont and edo-break which transfer control to a part of the loop where the bindings established by the clauses have to be valid but at the points where the init-form's of clauses get evaluated not all the bindings have, in general, been established yet. I saw no point in complicating things by introducing one scope for edo-exit and another for edo-cont and edo-break .

edo-user-info

The edo-user-info function can be used to obtain information either about a single LEM or all LEMs of the ES determined by the env argument. For this subsection, BLI {basic loop information} for a LEM will be a list containing all the data which appear after :user-info in that loop invocation or the empty list if the invocation of the LEM did not have a :user-info subform; ELI {extended loop information} will mean a list (A B C D) where A is the kind of macro {i.e. edo , edo-for , etc.}, B is name, C is SOED and D is the BLI for the loop. So for example, with

(edo (myloop :D) (:user-info "hello" 'world) ... )

the ELI is the list (edo myloop :D ("hello" 'world)) .

If extended-info? is NIL then {if no loop-designator gets passed then the return value will be a list containing the BLI information of all edoloop-enclosing LEMs, innermost first and moving outwards; if a loop-designator gets passed then there will be 2 return values: if there is a LEM corresponding to loop-designator then the first return value will be the BLI for that LEM and the second return value T, otherwise both return values will be NIL }. Otherwise, if no loop-designator gets passed then the return value will be a list containing the ELI information of all edoloop-enclosing LEMs, innermost first and moving outwards; if a loop-designator gets passed then if there is a LEM corresponding to loop-designator then the return value will be the ELI for that LEM, otherwise the return value will be NIL .Example:

    (defmacro print-user-info (&environment env &aux expansion)
    ; Using an &environment parameter is the way to get an environment
    ; object ; see 3.4.4 in CLHS.

       (push 'progn expansion)
        (push `(format t "Extended information about all the ~
                         surrounding loops is~%~s~%"
                         (quote ,(edo-user-info :env env :extended-info? t)))
              expansion)
        (push '(format t "Now we're going to print basic information about all ~%~
                          the surrounding loops , one by one , starting from the ~%~
                          innermost:~%")
              expansion)
        (dotimes (i (length (edo-user-info :env env)))
            (push `(format t "Loop ~a : ~s~%"
                             ,i (quote ,(edo-user-info :env env :loop-designator i)))
                   expansion))
        (reverse expansion))

   (edo-for (outer) (:user-info 1 "1" A) () () ()
        (edo   (second :BUC)
               ; No  (:user-info ...)  form
               () (t)
            (forever (innermost :CBU) (:user-info bcd)
                (print-user-info)
                (edo-exit nil outer))))
prints
    Extended information about all the surrounding loops is
    ((FOREVER INNERMOST :CBU (BCD)) (EDO SECOND :BUC NIL)
     (EDO-FOR OUTER :CBU (1 "1" A)))
    Now we're going to print basic information about all
    the surrounding loops , one by one , starting from the
    innermost:
    Loop 0 : (BCD)
    Loop 1 : NIL
    Loop 2 : (1 "1" A)

Modifying an object returned by edo-user-info is undefined behaviour.

EXAMPLES

    (edo-for   (nice-name :D)
               (:user-info You "can put any Lisp" 'object you like here 1234)
               (:let i (j 0) (k "123") (declare (symbol i))
                                       (declare (type (integer 0) j) (string k)))
               ()  ; Empty init-form
               ((<= j 2)   ; Loop for as long as j is <= 2
                    (format t "Xiting xit~%") ; A result form
                    (list i j k)              ; Another result form
               )
               ((unless i) (setq i :abc) (incf j)) ; Update form
        (format t "~s~%" (list i j k)))
prints
    (NIL 0 "123")
    (:ABC 1 "123")
    (:ABC 2 "123")
    Xiting xit
and returns
    (:ABC 3 "123")

   (defmacro edo-exit-outer (retval &environment env
                              &aux (depth (length (edo-user-info :env env))))
    "Exits the outermost LEM with return value retval"
        (when (eql depth 0) (error "No surrounding loops"))
        `(edo-exit ,retval ,(- depth 1)))

   (forever ()
        (forever ()
            (forever ()
                (edo-exit-outer 111))))
returns 111.

   (defun print-hash-table (hash-table)
    "Prints all the key-value entries of a hash table"
        (with-hash-table-iterator (next-entry hash-table)
            (edo-for (nil :UCB) (:let more? key value)
                     () (more?)
                     ((multiple-value-setq (more? key value) (next-entry)))
                (format t "~s  =>  ~s~%" key value))))

   (let ((hash (make-hash-table :test 'equalp)))
        (dotimes (i 10)
            (setf (gethash (format nil "~r" i) hash) i))
        (print-hash-table hash))
prints
    "zero"  =>  0
    "one"  =>  1
    "two"  =>  2
    "three"  =>  3
    "four"  =>  4
    "five"  =>  5
    "six"  =>  6
    "seven"  =>  7
    "eight"  =>  8
    "nine"  =>  9

We want to construct 2 macros, dovector and vecinfo .The first will be similar to DOLIST ;it will accept as a first argument the name of a variable var and as a second an expression which will evaluate to a vector. If the vector is not empty then the macro will enter a loop where in each repetition var will take the value of successive elements of the vector. vecinfo can only be used within the body or result forms of a dovector invocation; it will accept as an argument the name of the variable of a lexically enclosing dovector and will expand into an expression which will return 3 values, the index in the vector at the current repetition, the size of the vector and the vector itself, of the corresponding dovector.

    (defmacro dovector ((var vector-form &rest result-forms) &body body
                        &aux (index-var (gensym "dv-index-var"))
                             (vector (gensym "dv-vector"))
                             (limit (gensym "dv-limit")))
        `(edo-for  () (:user-info (dovector ,var ,index-var ,limit ,vector))
                   (:let* (,vector ,vector-form) (,limit (array-dimension ,vector 0))
                          (,index-var 0)
                          (,var (if (eql 0 ,limit) (return) (aref ,vector 0)))
                          (declare (type (and (integer 0) fixnum) ,limit ,index-var)
                                   (type vector ,vector)))
                   () (t ,@result-forms)
                   ((incf ,index-var) (when (eql ,index-var ,limit) (edo-break))
                    (setq ,var (aref ,vector ,index-var)))
             ,@body))

   (defmacro vecinfo (varname &environment env
                       &aux user-info exists? (i 0))
        (forever ()
            (multiple-value-setq (user-info exists?)
                                 (edo-user-info :env env :loop-designator i))
            (unless exists?
                (error "No enclosing dovector loop found with ~
                        variable name ~s" varname))
            (dolist (el user-info)
                (when (and (listp el) (eq (first el) 'dovector) (eq (second el) varname))
                    (return-from vecinfo `(values ,(third el) ,(fourth el)
                                                  ,(fifth el)))))
            (incf i)))

   (dovector (var1 #(1 2 3))
        (format t "~%")
        (dovector (var2 (map 'vector (lambda (a) (+ a var1)) #(4 5 6)))
            (format t "Information for inner loop is ~s    element is ~s~%"
                      (multiple-value-list (vecinfo var2)) var2)
            (format t "Information for outer loop is ~s    element is ~s~%"
                      (multiple-value-list (vecinfo var1)) var1)))
prints
    Information for inner loop is (0 3 #(5 6 7))    element is 5
    Information for outer loop is (0 3 #(1 2 3))    element is 1
    Information for inner loop is (1 3 #(5 6 7))    element is 6
    Information for outer loop is (0 3 #(1 2 3))    element is 1
    Information for inner loop is (2 3 #(5 6 7))    element is 7
    Information for outer loop is (0 3 #(1 2 3))    element is 1

   Information for inner loop is (0 3 #(6 7 8))    element is 6
    Information for outer loop is (1 3 #(1 2 3))    element is 2
    Information for inner loop is (1 3 #(6 7 8))    element is 7
    Information for outer loop is (1 3 #(1 2 3))    element is 2
    Information for inner loop is (2 3 #(6 7 8))    element is 8
    Information for outer loop is (1 3 #(1 2 3))    element is 2

   Information for inner loop is (0 3 #(7 8 9))    element is 7
    Information for outer loop is (2 3 #(1 2 3))    element is 3
    Information for inner loop is (1 3 #(7 8 9))    element is 8
    Information for outer loop is (2 3 #(1 2 3))    element is 3
    Information for inner loop is (2 3 #(7 8 9))    element is 9
    Information for outer loop is (2 3 #(1 2 3))    element is 3

Obviously, a "proper" implementation of dovector would allow for user declarations {which is why we only conditionally initialise var ; the value it gets must be consistent with any user supplied declarations}, would require a more reliable "namespacing" scheme for the information the macro itself enters in the user-info subform so that it does not clash with any information the user might enter in the user-info subforms of enclosing LEMs, would allow for a user-info subform for dovector itself and a user supplied block name {in which case vecinfo might also accept as argument a block name instead of a variable name}. But the example gives an idea of the possibilities the user-info mechanism provides.

CONTACT

For bug reports contact the author, Spiros Bousbouras, at

    (let ((a (copy-seq ".cobmaigou@lspim")))
        (rotatef (subseq a 0 3) (subseq a 12 15))
        (rotatef (subseq a 4 7) (subseq a 8 11))
        a)