Adventures in MAKEINIT and Lisp Loadups

[masinter]

The processes for buildilng sysouts have been quite manual and slow. There are steps documented as taking hours that now take seconds to complete. And there's quite a bit of deadwood, because (apparently) there was quite a bit of time where XAIS/Envos/Venue & PARC were trying to support running on D-machines at the same time, or using the same lisp compiled code on Maiko and on DMachines.

Unwrapping the loadups

The process of unwrapping the tangled messes and cleaning out deadwood is proceeding in reverse order.

Making an init

This proceeds in several phases in the function MAKEINITGREET (on file MAKEINITGREET).

1. producing code that can load compiled lisp into a memory image file instead of memory. Called DORENAME
2. running that code to produce an initial sysout init.sysout. Called MAKEINIT
3. fixing the sysout so that it can load on a machine that can't run BCPL and needs the microcode at the head of the file (DLFIXINIT). This makes an initial image file which can be used as a launch point (INIT.DLINIT)

This used to only run on a Dorado. It was slow (2.5 hours). It now completes in less than 5 seconds on a mac mini.
I've managed to get through the process enough times and have a medley branch 'makeinit6' with 'loadup-instructions'.
It's ugly and needs cleanup.

There were some patches and changes to files that didn't survive the MAKEINIT process and I've temporarily put them aside to get through -- ATBL, LLKEY where I've moved the newer versions to ADIR.4 and ADIR.LCOM.7 (naming convention).
This is still in process.

making a LISP.SYSOUT

We don't have an INIT.DLINIT from which to build a Lisp, so we've been patching a Lisp sysout we have from Venue (lisp.venusysout in the loadups directory, by reloading files that are newer (by their FILECREATED header dates, now that we're using github which doesn't preserve file system dates).

I'm getting to the place where it's trying to load DFASL.DFASL because there isn't a DFASL.LCOM (which there used to be, but we thought it was a mistake that there were both. I suspect that the reason there were both is because the FASL files work better with packages enabled?

I'm not sure the initialization of X is complete. And there's no reason to do the loadup blind now, since we could make a headless lisp without windows or maybe in the xmas code? to run without windowworld or processworld enabled.

I've been loading up without ethernet sources so far successfully

loadup FULL cleanup from Lisp

loading up a "full" sysout which has useful tools, text editor, that is comfortable to keep running as an environment

This used to take a while and so the site admins might choose to include a common set of packages
now can be done by 'loadup-full.sh' scripts (moving from makesysout to scripts directory soon)

making other files needed by runtime

These are in the file internal/library/MEDLEYUTILS.LCOM

1. MAKE-WHEREIS-HASH function for re-creating loadups/WHEREIS.HASH (for finding source files you haven't loaded)
2. MAKE-EXPORTS-ALL loading file library/EXPORTS.ALL which contained macro, record and other compile-time declarations needed by some packages to compile. (the file SYSEDIT loads this and sets some other parameters useful for editing low-level system files)
3. FIX-MEDLEY-DATES This should be run whenever you checkout / clone a medley repository to make the file modification date match the FILECREATED date. Useful for looking at Filebrowser.
4. FIX-MEDLEY-LINKS This should also be run whenever you checkout / clone a repository to put back the hard links between a file and its version numbered file.~NN~ since git doesn't preserve hard links.

there's also a script for changing your github diff so that git diff will treat CR as LF and ignore Control-A to Control-F so that new versions will compare more usefully ./scripts/install-diff-filter.sh.

[masinter]

Just to follow up on the MAKEINIT front: I got a LISP.SYSOUT with some manual tweaking and file renaming.
It's all in the makeinit6 branch, which I'm not planning to release, but introduce changes step by step starting again from master.

Questions? What belongs in the loadup? Ethernet? Pup? XNS? Press and Interpress? Postscript?
Which loadups should be there?