Paolo Amoroso's Journal

Interlisp

Medley is a residential environment for Interlisp and Common Lisp development.

With some effort it's possible to use Medley as a traditional file based Common Lisp environment. But in specific cases a better approach is to bring in Medley's residential environment Common Lisp sources created in file based environments.

In this post I explain the latter, i.e. how to use TextModules to import Common Lisp files into the residential environment. I go over the steps for converting an example program, the database of CD music tracks in Chapter 3 Practical: A Simple Database of Peter Seibel's book Practical Common Lisp.

Motivation

Using the tools and facilities of the residential environment, such as the File Manager, is the normal way of developing new Lisp programs. To run existing Common Lisp code you don't plan to change often, you can also use Medley as a traditional file based environment.

For existing code written in file based environments you want to use and further develop in Medley, a better option is to import the code into the residential environment and continue working from there. This is what TextModules helps to do.

What is TextModules

TextModules is a Medley tool for bringing Common Lisp sources into the residential environment and place them under the control of the File Manager. It can also do the reverse, i.e. export the File Manager descriptions and metadata to Common Lisp sources accessible from file based environments.

Using TextModules is a one time process. You run the tool once to import the code, then use and modify it with the tools and facilities of the residential environment.

The documentation of textModules starts from page 305 (page 335 of the PDF) of the Lisp Library Modules manual.

Preparing the Common Lisp files

This example involves two of the source files of Seibel's book, packages.lisp and simple-database.lisp in directory practicals-1.0.3/Chapter03 of the code archive.

Unlike CL:LOAD on Medley, TextModules doesn't require any special formatting of Common Lisp source files. For example, they don't need to begin with a semicolon character.

However, the Common Lisp implementation of Medley is incomplete and not ANSI compliant, so be sure to remove or adapt any unsupported forms. This is the case of the database example: packages.lisp makes current the package CL-USER which is missing from Medley. The fix is to substitute xcl-user for cl-user in the file, as XCL-USER is the Medley equivalent of CL-USER.

Another source of incompatibility is the LOOP macro. In Medley it's only a stub that runs an infinite loop no matter what clauses a call specifies.

To import with TextModules code that contains LOOP it would normally be necessary to replace any calls with equivalent expressions. Since this post focuses on TextModules I just use the LOOP calls intended to run an infinite loop, and ignore the others.

Running TextModules

As noted, importing with TextModules is the one time process of running the tool for every source file. Once in the residential environment, you save and manipulate the code as any other code under the File Manager.

First off, load TextModules by evaluating (FILESLOAD TEXTMODULES) at an Interlisp Exec. All its exported symbols are in package TM. Next, call the function TM:LOAD-TEXTMODULE for every Common Lisp file, which is similar to CL:LOAD with some additional processing.

Most Common Lisp programs comprise a file packages.lisp with package definitions, and a number of additional .lisp files that contain the bulk of the code. This dependency requires passing the files to TM:LOAD-TEXTMODULE in the proper order.

The file packages.lisp of the database defines the package for simple-database.lisp, so start with the former. At a Xerox Common Lisp (XCL) Exec with prompt > evaluate:

> (tm:load-textmodule "packages.lisp" :module "SIMPLEDB" :package (find-package "XCL-USER") :install t)
IL:SIMPLEDB

The only required argument is the input file packages.lisp. However, by default TM:LOAD-TEXTMODULE uses the same input file name as the name of the program for the File Manager. It wouldn't make much sense to call a database PACKAGES.LISP. A better choice is to pass the :module parameter with the more descriptive name SIMPLEDB.

The code in packages.lisp begins with an in-package form. To make sure the in-package symbol is accessible without qualifier, it should be read in a package such as XCL-USER that imports the standard Common Lisp symbols. Hence the argument :package (find-package "XCL-USER") in the call.

The argument :install t installs the definitions in the running system. Although not strictly necessary, it's useful for diagnostic purposes and because you likely want to continue working on the imported code.

Next, process simple-database.lisp by evaluating at a XCL Exec:

> (tm:load-textmodule "simple-database.lisp" :module "SIMPLEDB" :package (find-package "XCL-USER") :install t)
IL:SIMPLEDB

Again, the file begins with in-package and the reason for passing the :package argument is the same as for packages.lisp.

Saving the imported code

At this point the imported code is in the running Lisp image and the File Manager is ready to manipulate it. You can check the File Manager noticed the imported definitions by calling FILES? at an Interlisp Exec with prompt :

← (FILES?)
To be dumped:
SIMPLEDB ...changes to VARS: SIMPLEDBCOMS
                       VARIABLES: COM.GIGAMONKEYS.SIMPLE-DB::*DB*
                       FUNCTIONS: COM.GIGAMONKEYS.SIMPLE-DB::MAKE-CD, 
         COM.GIGAMONKEYS.SIMPLE-DB::ADD-RECORD, 
         COM.GIGAMONKEYS.SIMPLE-DB::DUMP-DB, 
         COM.GIGAMONKEYS.SIMPLE-DB::PROMPT-READ, 
         COM.GIGAMONKEYS.SIMPLE-DB::PROMPT-FOR-CD, 
         COM.GIGAMONKEYS.SIMPLE-DB::ADD-CDS, 
         COM.GIGAMONKEYS.SIMPLE-DB::SAVE-DB, 
         COM.GIGAMONKEYS.SIMPLE-DB::LOAD-DB, 
         COM.GIGAMONKEYS.SIMPLE-DB::CLEAR-DB, 
         COM.GIGAMONKEYS.SIMPLE-DB::SELECT, 
         COM.GIGAMONKEYS.SIMPLE-DB::WHERE, 
         COM.GIGAMONKEYS.SIMPLE-DB::MAKE-COMPARISONS-LIST, 
         COM.GIGAMONKEYS.SIMPLE-DB::MAKE-COMPARISON-EXPR, 
         COM.GIGAMONKEYS.SIMPLE-DB::UPDATE, 
         COM.GIGAMONKEYS.SIMPLE-DB::DELETE-ROWS

To save the definitions to the symbolic file SIMPLEDB call MAKEFILE from an Interlisp Exec:

← (MAKEFILE 'SIMPLEDB)
{DSK}<home>medley>il>SIMPLEDB.;1

Calling TM:LOAD-TEXTMODULE for every source file, and saving the result to a symbolic file with MAKEFILE, completes the import process.

You may terminate the session and resume later. When you're ready to proceed you can load, run, and modify the imported program as any other code under File Manager control.

Loading and running the imported code

In a new Medley session evaluate (FILESLOAD TEXTMODULES) at an Interlisp Exec. The tool must be in memory whenever you work with imported code, as TextModules sets up a special file environment and readtable the code needs to be read in.

Next, load the symbolic file of the database program by evaluating at a XCL Exec:

> (load "SIMPLEDB")

; Loading {DSK}<home>medley>il>SIMPLEDB.;1
; File created 14-Feb-2024 02:44:46
; IL:SIMPLEDBCOMS
IL:|{DSK}<home>medley>il>SIMPLEDB.;1|

One of the main entry points of the program is the function add-cds to add new records to the database, one record for each music track of a CD. A typical run from a XCL Exec looks like this:

> (setf *package* (find-package "COM.GIGAMONKEYS.SIMPLE-DB"))
#<Package COM.GIGAMONKEYS.SIMPLE-DB>
> (add-cds)
Title: Punch My Cards
Artist: The Fortrans
Rating: 6
Ripped [y/n]: n
Another? [y/n]: y
Title: Lisp n Roll
Artist: The Garbage Collectors
Rating: 8
Ripped [y/n]: y
Another? [y/n]: y
Title: Cdr Care Less
Artist: The Garbage Collectors
Rating: 7
Ripped [y/n]: y
Another? [y/n]: n
NIL

Since all the symbols of the program are in the package COM.GIGAMONKEYS.SIMPLE-DB, and none are exported, for convenience make the package current by setfing *package* as above.

I intentionally didn't rename the package or create nicknames. This is to show that Common Lisp code may be imported and used with minimal or no changes.

The program provides select and where to query the database. But where uses CL:LOOP features Medley doesn't support. To stay close to the original code, instead of modifying where you can use another function in Seibel's book. Define and call the specialized query function select-by-artist to search the database by artist:

> (defun select-by-artist (artist)
    (remove-if-not
     #'(lambda (cd) (equal (getf cd :artist) artist))
     *db*))
SELECT-BY-ARTIST
> (select-by-artist "The Garbage Collectors")
((:TITLE "Cdr Care Less" :ARTIST "The Garbage Collectors" :RATING 7 :RIPPED T) (:TITLE "Lisp n Roll" :ARTIST "The Garbage Collectors" :RATING 8 :RIPPED T))

Continuing the development

The imported code works. Now you can load, compile, run, edit, and save it as any other program developed in the residential environment under the File Manager.

You no longer need the original files packages.lisp and simple-database.lisp because you work only with SIMPLEDB. But remember to load TextModules with (FILESLOAD TEXTMODULES) in every session in which you use SIMPLEDB. It's a minor inconvenience but, for automating the task, you may add a loading command to the INIT initialization file or to a script that loads the program.

#CommonLisp #Interlisp #Lisp

Discuss... Email | Reply @amoroso@fosstodon.org

Managing Lisp code in the residential environment of Medley differs from similar tasks in traditional file based Common Lisp systems.

In a previous post I explained how the residential environment of Medley works, discussed some of its facilities and tools, and introduced a workflow for managing Common Lisp code under the residential environment. This leverages the way Medley is designed to work and minimizes friction.

In this post I explain how to use Medley as a file based environment, albeit with some friction and reduced functionality.

I show how to edit, organize, and load pure Common Lisp files, i.e. source files not under the control of the File Manager. Pure files are ordinary text files that contain only Common Lisp code, with no metadata or font control commands like the code databases the File Manager maintains.

Motivation

Letting the residential environment track and control code is the most convenient and productive way of using Medley for Lisp development.

But you can still manually manage pure Common Lisp files. For example, you may want to use from Medley some code you mainly intend to load into the environment and don't change often. This is the case of external libraries and programs developed elsewhere, which you call from new code or run in Medley.

Using Common Lisp code not developed for the pre ANSI implementation of Medley may need adaptation. So let's see how to edit source files.

Editing source files

To create or modify pure Common Lisp files use the TEdit word processor of Medley, not the SEdit structure editor. SEdit doesn't directly manipulate the arbitrary, unstructured text of pure files.

TEdit can read and write ordinary text files, but not by default. To save a file as ASCII execute the command Put > Plain-Text. Execute Get > Unformatted Get to load such a file.

The main downside of using TEdit for Common Lisp is the program is not Lisp aware. It doesn't support automatic indentation, prettyprinting, perenthesis matching, and code evaluation. Another major problem is cursor keys don't work in Medley, which severely limits text editing with TEdit. The Medley team is well aware of the issue.

An alternative is to use your favorite Lisp editor to edit the Common Lisp sources outside of Medley and then load the files from Medley.

Format of Pure Common Lisp files

Although Medley can load pure Common Lisp sources, these files do have some minimal formatting requirements. A pure Common Lisp file must begin with a semicolon ; character, otherwise the system assumes it's an Interlisp file.

Other than that a pure file can contain any Common Lisp feature Medley supports, in any order the language allows.

Loading source files

Once a pure Lisp file is available, load it in Medley with the usual file loading forms such as CL:LOAD. The way symbols are interned depends on whether the file defines or makes current any Common Lisp packages.

Let's go over the main cases: the code has no packages; a single file contains both package and function definitions; the code is split between a file for the package and another for the function definitions.

No package definition

The simplest type of pure file contains no package forms, just function definitions and other Common Lisp expressions.

In this case Medley interns symbols in the USER package without exporting them. The Medley implementation of Common Lisp is closer to CLtLx than ANSI, so it provides the standardized packages LISP (nicknames: COMMON-LISP and CL) and USER instead of COMMON-LISP and COMMON-LISP-USER.

For example, suppose this file SQUARE.LISP defines a function calc-square to compute the square of its argument:

;;; SQUARE.LISP

(defun calc-square (x)
  "Return the square of X."
  (* x x))

After evaluating (load "SQUARE.LISP"), at the > prompt of a Common Lisp Exec you call the function using the proper package qualifier:

> (user::square 3)
9

If the USER package isn't adequate for your code, on Medley CL:LOAD also accepts the keyword parameter :package to supply a package to which *package* is bound when reading the file. For example, at the > prompt of a Xerox Common Lisp (XCL) Exec you can pass the package XCL-USER with a form like:

> (load "SQUARE.LISP" :package (find-package "XCL-USER"))

You must pass an actual package object to :package, not a package designator like :xcl-user.

Since XCL-USER is the default package of XCL Execs no qualifier is required for calling the function:

> (square 3)
9

Package and definitions in one file

Not using packages may be adequate for very small files. However, typical Common Lisp programs do define their own packages.

This example file PKGDEMO.LISP defines the package PKGDEMO with nickname PD that exports the function FUN, which just prints a message:

;;; PKGDEMO.LISP


(in-package "XCL-USER")

(defpackage "PKGDEMO"
  (:use "LISP" "XCL")
  (:nicknames "PD")
  (:export "FUN"))


(in-package "PKGDEMO")

(defun fun ()
  (format t "Hello from PKGDEMO:FUN."))

The file holds both the package definition and the rest of the code. It's important that the first form in the file be in-package to make current a known package like XCL-USER. XCL-USER is the Medley equivalent of COMMON-LISP-USER and uses the LISP package with the standard Common Lisp symbols.

After loading PKGDEMO.LISP from an XCL Exec with (load "PKGDEMO.LISP") you can call the exported function like this:

> (pkgdemo:fun)
Hello from PKGDEMO:FUN.
NIL

or via the package nickname:

> (pd:fun)
Hello from PKGDEMO:FUN.
NIL

Separate package and definition files

Unlike PKGDEMO.LISP in the previous example, most Common Lisp programs split the code between at least two files, one holding the package definition and the other the function definitions. Let's rewrite PKGDEMO.LISP by storing the package in the file PKGSPLIT-PKG.LISP:

;;; PKGSPLIT-PKG.LISP


(in-package "XCL-USER")

(defpackage "PKGDEMO"
  (:use "LISP" "XCL")
  (:nicknames "PD")
  (:export "FUN"))

The function definition goes into the file PKGSPLIT-FUN.LISP that begins with an appropriate in-package followed by the rest of the code:

;;; PKGSPLIT-FUN.LISP


(in-package "PKGDEMO")

(defun fun ()
  (format t "Hello from PKGDEMO:FUN."))

Finally, make sure to load the package first with (load "PKGSPLIT-PKG.LISP"), then the function with (load "PKGSPLIT-FUN.LISP"). Loading the files creates the same objects as the single file example PKGDEMO.LISP and you can call the function the same way:

> (pkgdemo:fun)
Hello from PKGDEMO:FUN.
NIL
> (pd:fun)
Hello from PKGDEMO:FUN.
NIL

#CommonLisp #Interlisp #Lisp

Discuss... Email | Reply @amoroso@fosstodon.org

One of the cool features of Lisp is examining and modifying a running program.

This allows, for example, to correct a bug by inspecting, editing, fixing, and resuming a program that breaks and lands in the debugger because of an error. To gain familiarity with the process, I recorded a screencast of a Medley session in which I use the debugger to fix a bug in a running Interlisp function and conclude the computation.

Medley provides advanced debugging facilities and tools in the Break Package, including the Break Window which is the main user interface of the debugger. “Package” as in module or subsystem, not Common Lisp package.

In the recorded session I fix this broken Interlisp function to compute the square of the argument:

(DEFINEQ (CALC.SQUARE (X)
  (* Return the square of argument X.)
  (TIMES X Y)))

The bug is trivial, a typo. The call to the TIMES multiplication operator passes Y as the second argument instead of the function parameter X.

I begin the recorded session by defining the function with the SEdit Lisp editor. Next, from the Exec (a Lisp REPL) I call the function (CALC.SQUARE 3) and get the error Y is an unbound variable. Then I execute the RETRY Exec command. RETRY evaluates the latest expression and forces entering the debugger if it yields an error.

The topomost few backtrace entries in the Break Window are internal functions called by CALC.SQUARE. From the Break Window's middle-click menu I invoke the REVERT command to move the point of execution back to the CALC.SQUARE call before the error.

This selects the CALC.SQUARE frame where the bug is most likely to be. Inspecting the bindings of the frame provides a clue.

X has the expected value 3 and I confirm it by evaluating X at the debugger REPL. But Y, the variable the error references, doesn't show up in the stack frame. Evaluating Y yields the same error. It's a hint Y shouldn't probably be there and is likely a typo. Therefore, I execute the debugger's EDIT command to open the code of the current function in SEdit. From SEdit I fix the typo and evaluate the modified definition.

The new definition is now in the Lisp image and the current stack frame is still the CALC.SQUARE call in the backtrace. The OK debugger command continues execution from the point of the break, letting the program run the corrected code. The Exec from which I originally called the buggy function finally returns the expected value of the square of 3: 9.

The REVERT, EDIT, and OK commands may be typed at the debugger REPL but I invoked them from the menu to make the menu itself and its options explicit. Similarly, the Done & Close SEdit menu command has the associated keychord C-M-x.

#Interlisp #Lisp

Discuss... Email | Reply @amoroso@fosstodon.org

My exploration of Medley as a Common Lisp development environment proceeds with setting up a workflow for writing and saving code.

The workflow consists of a series of steps in a specific order using appropriate Lisp REPLs and tools. It supports writing the simplest type of Common Lisp software, i.e. programs or libraries in a single package that exports some symbols. I'll eventually extend the workflow to more complex cases such as programs with more than one package.

Since the steps are not intuitive, especially for Medley novices, in this post I'll describe the workflow in detail. But first there are a few concepts to introduce.

Why do you need a workflow in the first place? Because the differences between Medley and modern environments constrain how and in what order Common Lisp code may be written and managed. Before examining the constraints let's describe the differences.

The residential environment of Medley

Writing single package programs is straightforward in modern file based Common Lisp environments. In a new file you just define the package with DEFPACKAGE, then in the rest of the file or at the top of a new one you place a matching IN-PACKAGE followed by the code.

Although it's technically possible to do the same in Medley this doesn't take advantage of its facilities, and you may actually need to fight the system to accomplish what you want. Indeed, Medley is not an ordinary environment. Not only it predates current Common Lisp implementations, it supports a different development process.

In file based Common Lisps you directly edit source files and evaluate or load the code into the running Lisp image.

Medley instead is a “residential environment” in which you edit and evaluate Lisp objects that reside in the image — hence “residential”. Then you save the code to files that are more like code databases than traditional source files.

You don't edit the code databases, which Medley calls “symbolic files”. Rather, you use the SEdit structure editor to modify the code in memory and the “File Manager” to save the code to disk. The beginning of a symbolic file defines metadata, the “file environment”, which describes the Common Lisp package and readtable associated with the code.

The File Manager, also known as “File Package” (not to be confused with Common Lisp packages), is a facility that coordinates the development tools and code management tasks. It notices the changes to Lisp objects edited with SEdit or manipulated in memory, tracks what changed functions and objects need to be saved to symbolic files, and carries out the actions for building programs such as compiling or listing them. The File Manager has some of the functionality of Unix Make.

Figuring what changed is easy in modern Common Lisp environments, as you know which files you edited and need action like saving or compiling. System definition tools like ASDF can track this for you. In Medley it's the File Manager which tracks the changes that take place in the running image and need to be synchronized to disk.

Motivation

How do the peculiarities of Medley constrain writing Common Lisp code and require a tailored workflow?

The functions and objects of Interlisp programs usually live in the same namespace of Interlisp and its tools. As a consequence, functions and Lisp objects may be mostly defined in any order and accessed without package qualifiers. No special handling is necessary with the File Manager either.

With Common Lisp code, however, a subtle complication arises due to packages and exported symbols.

A good explanation of why things are different, and how the File Manager and file environment interact, is in the documentation of TextModules, a tool for importing Common Lisp code created outside of Medley. Although in the context of TextModules, these remarks give an overview of the same issues the File Manager faces with other Common Lisp code. The TextModules chapter of the Lisp Library Modules manual says on page 311 (page 341 of the PDF):

It is important to separate the environment of the file from its contents because the File Manager (not TextModules) first reads all the forms in the file, and then evaluates them. Text based source files sometimes change the package as needed. This cannot work for the File Manager since the file's forms are all read and then executed, i.e. the package changes would not occur until after the entire file had been read, and forms after any IN-PACKAGE form would have been read incorrectly.

In other words, unless you define packages and access symbols in the proper order, you'll get subtle errors. The solution is a workflow that avoids such errors.

More information on dealing with packages in Medley is in the sources referenced in section “Documentation” of my post on using Common Lisp on Medley.

The workflow

How does the workflow order the development tasks to achieve its goal?

At any one time the workflow accesses only defined symbols and ensures the running Lisp image stays synchronized with the symbolic file. It's not the only or the best possible workflow, just one that works. I put it together after extensively reading the documentation and experimenting.

As I said in the overview of Medley as a Common Lisp environment, when coding in Common Lisp I use two Executives (Lisp REPLs), a Xerox Common Lisp (XCL) Exec and an Interlisp one.

The former is for testing, running, and evaluating Common Lisp code. I use the Interlisp Exec for running system tools and interacting with the File Manager. Since the tools and File Manager facilities are in the Interlisp package, referencing them from a Common Lisp Exec would require qualifying all symbols with the IL: package.

What follows assume you're familiar with basic Interlisp and File Manager features such as SEdit, file coms, FILES?, and MAKEFILE. If not I recommend reading the Medley primer, particulary Chapter 7 “Editing and Saving”. Also, unless otherwise noted, you should carry out the steps in sequence in the same Medley session.

Let's start.

Defining the file environment and a minimal package

Suppose you want to write a Common Lisp program or library stored in the file SINGLEPKG. The package SINGLEPKG, nicknamed SP, will export the two functions FUN1 and FUN2.

From now on, denotes the prompt of an Interlisp Exec and > that of a Xerox Common Lisp (XCL) Exec. Sometimes I'll tell you in which Exec to evaluate expressions.

The first step is to define the file environment. At an Interlisp Exec evaluate:

← (XCL:DEFINE-FILE-ENVIRONMENT SINGLEPKG :PACKAGE (DEFPACKAGE "SINGLEPKG" (:USE "LISP" "XCL")) :READTABLE "XCL")

For now don't use other packages or export any symbols, just enter the form as is.

Although the file environment references package SINGLEPKG, the package doesn't exist yet in the running image. To synchronize the image with the file environment evaluate the definition of a minimal package from an Interlisp Exec:

← (DEFPACKAGE "SINGLEPKG" (:USE "LISP" "XCL"))

Next, from an Interlisp Exec evaluate (FILES?) and, when asked where the SINGLEPKG file info should go, respond yes, enter SINGLEPKG as the file name, and confirm the creation of the file.

Defining the first function

Everything is ready to define the first function FUN1. At an Interlisp Exec call SEdit with (ED 'SINGLEPKG::FUN1 '(FUNCTIONS :DONTWAIT)) and select DEFUN from the menu. Enter the code of FUN1:

(DEFUN FUN1 ()
  (FORMAT T "Hello from FUN1."))

Save and exit with Ctrl-Alt-X and test the function at an XCL Exec (an Interlisp Exec will do too):

> (SINGLEPKG::FUN1)
Hello from FUN1.
NIL

It works, so at an Interlisp Exec evaluate (FILES?) to associate FUN1 with the file SINGLEPKG.

Completing the package definition

The Lisp image contains the new symbol FUN1 in package SINGLEPKG but there's no symbolic file yet, let alone an exported symbol in the file. Therefore, to keep things in sync you need to update the package definition by exporting the function name. This is also an opportunity for adding the SP nickname to the package.

At an Interlisp Exec evaluate (DC SINGLEPKG) to open the file coms in SEdit. Just after the XCL:FILE-ENVIRONMENTS form enter:

(P (DEFPACKAGE "SINGLEPKG"
     (:USE "LISP" "XCL")
     (:NICKNAMES "SP")
     (:EXPORT SINGLEPKG::FUN1)))

Both colon characters : are required in the function name. The P File Manager command tells the system to execute the following Lisp expressions at load time, so loading SINGLEPKG will define the package and export the symbol.

Save and exit with Ctrl-Alt-X and, at an Interlisp Exec, save the file with (MAKEFILE 'SINGLEPKG) to reflect the updated definitions. This creates the file SINGLEPKG on disk.

Before doing anything else it's better to make sure the package is properly defined and the function exported.

The most reliable way is to exit the Medley session with (IL:LOGOUT), start a new session and, from an Interlisp Exec, evaluate (LOAD 'SINGLEPKG). If there are errors you will have to go back and check whether you went through all the steps correctly. You may need to delete the latest or all versions of the file SINGLEPKG.

Assuming there are no errors the package is properly defined and the function exported. To double check, from an XCL Exec call the exported function like this:

> (SINGLEPKG:FUN1)
Hello from FUN1.
NIL

Since it works you may proceed development by editing the existing function or defining new ones. In the former case you employ SEdit, FILES?, and MAKEFILE as usual. Just make sure to pass to ED the package-qualified function name like (ED 'SINGLEPKG:FUN1 :DONTWAIT).

Things change if you want to define new functions.

Defining more functions

As planned you proceed to define a second function FUN2 exported from package SINGLEPKG. If you're still in the Medley session in which you've just loaded the file SINGLEPKG, continue from there. Otherwise start a new session and load the file with (LOAD 'SINGLEPKG) from any Exec.

At an Interlisp Exec define the new function with (ED 'SINGLEPKG::FUN2 '(FUNCTIONS :DONTWAIT)) and select DEFUN from the menu:

(DEFUN FUN2 ()
  (FORMAT T "Hello from FUN2."))

Save and exit with Ctrl-Alt-X and test the function at an XCL Exec:

> (SINGLEPKG::FUN2)
Hello from FUN2.
NIL

Call FILES? to associated FUN2 with file SINGLEPKG.

As already done for FUN1 you need to modify the package definition to export the new function. At an Interlisp Exec edit the file coms with (DC SINGLEPKG) and add the symbol FUN2 to the export clause, which will now look like this:

(P (DEFPACKAGE "SINGLEPKG"
     (:USE "LISP" "XCL")
     (:NICKNAMES "SP")
     (:EXPORT SINGLEPKG:FUN1 SINGLEPKG::FUN2)))

SEdit replaced the double colon of SINGLEPKG::FUN1 with a single colon as in SINGLEPKG:FUN1. But you still have to type :: for FUN2 because FUN2 hasn't been exported yet.

Save and exit with Ctrl-Alt-X and, at an Interlisp Exec, save the file with (MAKEFILE 'SINGLEPKG).

To synchronize the Lisp image with the symbolic file, which will allow to call the exported function as (SINGLEPKG:FUN2), either load the file SINGLEPKG from a fresh session or, just after editing the file coms, evaluate a revised package definition at an Interlisp Exec:

← (DEFPACKAGE "SINGLEPKG"
    (:USE "LISP" "XCL")
    (:NICKNAMES "SP")
    (:EXPORT SINGLEPKG::FUN1 SINGLEPKG::FUN2))

Either way, to check that everything works evaluate at an XCL Exec:

> (SINGLEPKG:FUN2)
Hello from FUN2.
NIL

Success!

For every new function or Lisp object you want to export from the package, go through the steps of this section again, making sure the Lisp image and the symbolic file stay synchronized. The steps are, in order:

  1. edit the new function
  2. call FILES? to tell the File Manager about the function
  3. edit the file coms to update the package
  4. save the file with MAKEFILE
  5. evaluate the revised package definition

That's all, you're finally ready to develop single package programs. This workflow may seem convoluted at first but things will come more natural as you gain experience with Medley.

#CommonLisp #Interlisp #Lisp

Discuss... Email | Reply @amoroso@fosstodon.org

Medley is the first Lisp system I experimented with on my new Raspberry Pi 400, here is the AArch64 version on the Raspberry Pi OS desktop:

SDL version of Medley Interlisp on a Raspberry Pi 400 under Raspberry Pi OS.

The online version of Medley runs well and smoothly enough in Chromium. There are no major or unexpected performance differences between the Pi 400 and my Chromebox.

But the AArch64 .deb package of the prebuilt Medley distribution is really, really slow on the Pi 400. For example, when launching the program I can see the characters being printed one by one to the Exec window. The problem is the AArch64 release is built for X-Windows, which on the Wayland based Raspberry Pi OS Bookworm goes through Xwayland with degraded performance.

Therefore I built from source the SDL version of Medley that doesn't rely on X-Windows.

Building everything, i.e. compiling Maiko and making the loadups, takes less than five minutes on the Pi 400. Although the process is simple, it took me some trial and error to figure the exact steps from the limited documentation. Nick Briggs helped me troubleshoot the build and advised on how to create a missing makefile.

The SDL version is snappier than the default AArch64 Debian package. Now Medley starts up almost instantly. The only inconvenience common to both versions is they don't recognize my Italian keyboard as they work only with the English US layout.

#Interlisp #pi400

Discuss... Email | Reply @amoroso@fosstodon.org

Although Medley's documentation and published material is scattered across several archives and sources, and doesn't cover all the system facilities, once I find the right document I get most of the information I need on a specific feature.

But sometimes the information is not up to date because Medley is still under development after so many decades.

I bumped into such a case when experimenting with TableBrowser, Medley's tabular data browser tool on which the system file browser is built. Other applications can access and control from Lisp TableBrowser, which is a reusable component with an API.

Before program development with the API, TableBrowser requires setting up a specific environment by loading a source file with some declarations, which are not needed when running compiled applications. The Lisp Library Packages manual explains how to set up the environment in section “Installation” of the “TABLEBROWSER” chapter on page 283 of the PDF. Here the manual provides the File Manager commands to add to the source file of an application that uses TableBrowser:

(FILES (SYSLOAD) TABLEBROWSER)
(DECLARE: EVAL@COMPILE DONTCOPY
     (FILES (SOURCE) TABLEBROWSERDECLS)

But in spite of following these instructions I got an error when calling TableBrowser functions.

It turns out that, as a result of the recent modernization work the Medley Interlisp Project is doing, some source files were rearranged and moved. So now all it takes to set up the Lisp environment for development with TableBrowser is the form (LOADCOMP 'TABLEBROWSER), for example in the coms of the program under development.

#Interlisp #Lisp

Discuss... Email | Reply @amoroso@fosstodon.org

Since encountering Medley I gained considerable experience with Interlisp. Medley Interlisp is a project for preserving, reviving, and modernizing the Interlisp-D software development environment of the Lisp Machines Xerox created at PARC.

Nine months later I know enough to find my way around and confidently use most of the major system tools and features.

I read all the available documentation, books, and publications, so I know where to look for information. And I undertook Interlisp programming projects such as Stringscope, Braincons, Sysrama, and Femtounit.

Now I'm ready to explore Medley as a Common Lisp development environment.

Although most of the system, facilities, and tools are written in and designed around Interlisp, the companies that maintained and marketed Medley over time partially implemented Common Lisp and integrated it with the environment. The completion level of the implementation is somewhere between CLtL1 and CLtL2, plus CLOS via Portable Common Loops (PCL).

Motivation

I want to widen this experience to Common Lisp.

I'll leverage the more advanced Lisp dialect and interface with Interlisp's facilities as an application platform that comprises a rich set of libraries and tools such a window system, graphics primitives, menu facilities, and GUI controls for building applications. Each world can interoperate with the other, so Common Lisp functions can call Interlisp ones and the other way around.

Developing Common Lisp programs with Medley is both my goal and a way of achieving it through practice. Medley is an ideal self-contained computing universe for my personal projects and Common Lisp greatly enchances its toolbox.

Tools

The main tools for developing Common Lisp code are the same as for Interlisp: the SEdit structure editor for writing code; the File Manager, a make-like tool for tracking changes to Lisp objects in the running image and saving them to files; and the Executive (or Exec), the Lisp listener.

However, the workflow is subtly different.

In some cases taking advantage of the integration with Medley involves different steps for Common Lisp code. For example, defining and changing packages so that the File Manager notices and tracks them needs to be done in a certain order. And there are Medley extensions to the package forms.

When working with Common Lisp I open at least two Execs, a Common Lisp and an Interlisp one. The former is for testing, running, and evaluating Common Lisp code.

The Interlisp Exec is for launching system tools and interacting with the File Manager. Since all the symbols of SEdit, the File Manager, and other system tools are in the IL Interlisp package, in an Interlisp Exec it's not necessary to add package qualifiers to symbols all the time.

Exec commands such as DIR and CD work the same in both Execs.

Documentation

Medley's Common Lisp features aren't documented in the Interlisp Reference Manual, the main information source about the system. The reason is the companies that distributed and maintained the product ceased operations before the work on implementing and documenting Common Lisp was completed.

I found only a couple of good sources on Common Lisp under Medley.

The implementation notes and the release notes of Lyric, the music-themed codename of one of Interlisp-D's versions, provide an overview of the integration between Common Lisp and Medley. The release notes of Medley 1.0, a later version, expand on this. Issue 5 of HOTLINE!, a newsletter Xerox published for its Lisp customers, has useful step by step examples of creating and managing Common Lisp packages the Medley way.

Some of the system code of Medley is written in Common Lisp and may be a source of usage examples and idioms. I'm also writing Common Lisp code snippets to test my understanding of the integration with Medley.

#CommonLisp #Interlisp #Lisp

Discuss... Email | Reply @amoroso@fosstodon.org

Planet Python carried out my request to remove my blog from the aggregator. Now their feed no longer syndicates my posts about Python, which I'll no longer write much about.

Planet Python is an aggregator of blogs, podcasts, and other resources of interest to the Python community. In late 2019 I submitted the feed of Python posts of my old blog, later updating it to point to my new blog.

I was learning the language and sharing on the blog my experience with coding projects and other experiences. But although I had great fun with Python and accomplished a lot I'm proud of, my interest waned as I rediscovered my old love Lisp.

I encountered Scheme in the early 1990s at an introductory computer science class based on Structure and Interpretation of Computer Programs, fell in love with the Lisp family of languages, and learned Common Lisp and Emacs Lisp. Lisp became my only language until the early 2010s when real life claimed my time and attention. Near the end of the decade, intrigued by Python and its massive ecosystem, I decided to learn it.

At the beginning of 2023 I discovered Medley Interlisp and got hooked.

Using Interlisp and its environment made me realize Lisp is the language that comes most natural to me, I'm most productive with, and gives me joy and not just fun. I never mastered and enjoyed other languages to the level of Lisp. And my projects turned out not to need Python's batteries.

I'll still maintain a reading knowledge of Python and keep up with its ecosystem. But this journey made me readjust my focus on Lisp, now my only language.

It's good to be back home.

#Interlisp #Python #Lisp #blogging

Discuss... Email | Reply @amoroso@fosstodon.org

I integrated Femtounit with the File Manager by defining the new type TESTS for Femtounit tests and redefining DEFTEST in terms of it.

It turns out it's not a good idea as the tests get duplicated. The DEFTEST macro expands into a DEFINEQ function definition and the File Manager notices both, the DEFTEST form of type TESTS and the function of type FNS.

The fix seemed simple, assigning tests to an existing type such as FNS. First I removed TESTS with (DELDEF 'TESTS 'DEFINE-TYPES), then replaced the type argument TESTS with FNS in the XCL:DEFDEFINER form. But when I tried to edit a sample test TEST.PLUS for the PLUS function with (ED 'TEST.PLUS :DONTWAIT), SEdit quit with the error:

Warning: Couldn't find a hash-table for FNS definitions.
One will be created.
Could not find fns definition for TEST.PLUS.
Could not find fns definition for
TEST.PLUS

Back to the drawing board.

#femtounit #Interlisp #Lisp

Discuss... Email | Reply @amoroso@fosstodon.org

I integrated Femtounit, my Interlisp unit test framework, with the File Manager and the SEdit structure editor.

To achieve this I defined the new File Manager type TESTS for Femtounit tests and redefined DEFTEST in terms of it, then tweaked the code formatting of tests in SEdit. Now the system notices and keeps track of new and modified tests.

Here's the File Manager type menu, which SEdit pops up when opening a test, with the DEFTEST option highlighted under TESTS:

Interlisp File Manager menu with the Femtounit types.

Also, now SEdit handles and properly formats test definitions like this of a function SQUARE to compute the square of its argument:

SEdit editing a Femtounit unit test definition on Interlisp.

Implementing the new features was much easier than expected.

The traditional way of adding types to the File Manager, described in the Interlisp Reference Manual, involves writing a dozen functions for most of which it's not clear how they're supposed to work. Medley 1.0 added support for two easy to use File Manager type defining forms that replace all that, XCL:DEF-DEFINE-TYPE and XCL:DEFDEFINER. This code is all it took for Femtounit's new TESTS type:

(XCL:DEF-DEFINE-TYPE TESTS "Femtounit unit tests")

(XCL:DEFDEFINER (DEFTEST (:PROTOTYPE
                            (LAMBDA (NAME)
                              (AND (LITATOM NAME)
                                   `(DEFTEST ,NAME ("Arg List")
                                      "Body")))))
                TESTS
                (NAME PARAMETERS &BODY BODY)
   `(DEFINEQ (,NAME ,PARAMETERS
      (LET ((FTU.TEST.NAME (APPEND FTU.TEST.NAME (LIST ',NAME]
        ,@BODY))))

This additional tweak makes SEdit format and indent DEFTEST unit test definitions the same way as DEFUN function definitions in Common Lisp:

(SEDIT:DEF-LIST-FORMAT DEFTEST CL:DEFUN)

The simplified process is not discussed in the Interlisp Reference Manual, where I'd have expected, but in a more obscure source, the Medley 1.0 release notes from page 52 of the PDF document. My proactive reading of all the documentation paid off as it let me spot such crucial information and save a lot of work.

#femtounit #Interlisp #Lisp

Discuss... Email | Reply @amoroso@fosstodon.org