15. XEmacs from the Inside

Internally, XEmacs is quite complex, and can be very confusing. To simplify things, it can be useful to think of XEmacs as containing an event loop that “drives” everything, and a number of other subsystems, such as a Lisp engine and a redisplay mechanism. Each of these other subsystems exists simultaneously in XEmacs, and each has a certain state. The flow of control continually passes in and out of these different subsystems in the course of normal operation of the editor.

It is important to keep in mind that, most of the time, the editor is “driven” by the event loop. Except during initialization and batch mode, all subsystems are entered directly or indirectly through the event loop, and ultimately, control exits out of all subsystems back up to the event loop. This cycle of entering a subsystem, exiting back out to the event loop, and starting another iteration of the event loop occurs once each keystroke, mouse motion, etc.

If you’re trying to understand a particular subsystem (other than the event loop), think of it as a “daemon” process or “servant” that is responsible for one particular aspect of a larger system, and periodically receives commands or environment changes that cause it to do something. Ultimately, these commands and environment changes are always triggered by the event loop. For example:

• The window and frame mechanism is responsible for keeping track of what windows and frames exist, what buffers are in them, etc. It is periodically given commands (usually from the user) to make a change to the current window/frame state: i.e. create a new frame, delete a window, etc.
• The buffer mechanism is responsible for keeping track of what buffers exist and what text is in them. It is periodically given commands (usually from the user) to insert or delete text, create a buffer, etc. When it receives a text-change command, it notifies the redisplay mechanism.
• The redisplay mechanism is responsible for making sure that windows and frames are displayed correctly. It is periodically told (by the event loop) to actually “do its job”, i.e. snoop around and see what the current state of the environment (mostly of the currently-existing windows, frames, and buffers) is, and make sure that state matches what’s actually displayed. It keeps lots and lots of information around (such as what is actually being displayed currently, and what the environment was last time it checked) so that it can minimize the work it has to do. It is also helped along in that whenever a relevant change to the environment occurs, the redisplay mechanism is told about this, so it has a pretty good idea of where it has to look to find possible changes and doesn’t have to look everywhere.
• The Lisp engine is responsible for executing the Lisp code in which most user commands are written. It is entered through a call to eval or funcall, which occurs as a result of dispatching an event from the event loop. The functions it calls issue commands to the buffer mechanism, the window/frame subsystem, etc.
• The Lisp allocation subsystem is responsible for keeping track of Lisp objects. It is given commands from the Lisp engine to allocate objects, garbage collect, etc.

etc.

The important idea here is that there are a number of independent subsystems each with its own responsibility and persistent state, just like different employees in a company, and each subsystem is periodically given commands from other subsystems. Commands can flow from any one subsystem to any other, but there is usually some sort of hierarchy, with all commands originating from the event subsystem.

XEmacs is entered in main(), which is in ‘emacs.c’. When this is called the first time (in a properly-invoked ‘temacs’), it does the following:

1. It does some very basic environment initializations, such as determining where it and its directories (e.g. ‘lisp/’ and ‘etc/’) reside and setting up signal handlers.
2. It initializes the entire Lisp interpreter.
3. It sets the initial values of many built-in variables (including many variables that are visible to Lisp programs), such as the global keymap object and the built-in faces (a face is an object that describes the display characteristics of text). This involves creating Lisp objects and thus is dependent on step (2).
4. It performs various other initializations that are relevant to the particular environment it is running in, such as retrieving environment variables, determining the current date and the user who is running the program, examining its standard input, creating any necessary file descriptors, etc.
5. At this point, the C initialization is complete. A Lisp program that was specified on the command line (usually ‘loadup.el’) is called (temacs is normally invoked as temacs -batch -l loadup.el dump). ‘loadup.el’ loads all of the other Lisp files that are needed for the operation of the editor, calls the dump-emacs function to write out ‘xemacs’, and then kills the temacs process.

When ‘xemacs’ is then run, it only redoes steps (1) and (4) above; all variables already contain the values they were set to when the executable was dumped, and all memory that was allocated with malloc() is still around. (XEmacs knows whether it is being run as ‘xemacs’ or ‘temacs’ because it sets the global variable initialized to 1 after step (4) above.) At this point, ‘xemacs’ calls a Lisp function to do any further initialization, which includes parsing the command-line (the C code can only do limited command-line parsing, which includes looking for the ‘-batch’ and ‘-l’ flags and a few other flags that it needs to know about before initialization is complete), creating the first frame (or window in standard window-system parlance), running the user’s init file (usually the file ‘.emacs’ in the user’s home directory), etc. The function to do this is usually called normal-top-level; ‘loadup.el’ tells the C code about this function by setting its name as the value of the Lisp variable top-level.

When the Lisp initialization code is done, the C code enters the event loop, and stays there for the duration of the XEmacs process. The code for the event loop is contained in ‘cmdloop.c’, and is called Fcommand_loop_1(). Note that this event loop could very well be written in Lisp, and in fact a Lisp version exists; but apparently, doing this makes XEmacs run noticeably slower.

Notice how much of the initialization is done in Lisp, not in C. In general, XEmacs tries to move as much code as is possible into Lisp. Code that remains in C is code that implements the Lisp interpreter itself, or code that needs to be very fast, or code that needs to do system calls or other such stuff that needs to be done in C, or code that needs to have access to “forbidden” structures. (One conscious aspect of the design of Lisp under XEmacs is a clean separation between the external interface to a Lisp object’s functionality and its internal implementation. Part of this design is that Lisp programs are forbidden from accessing the contents of the object other than through using a standard API. In this respect, XEmacs Lisp is similar to modern Lisp dialects but differs from GNU Emacs, which tends to expose the implementation and allow Lisp programs to look at it directly. The major advantage of hiding the implementation is that it allows the implementation to be redesigned without affecting any Lisp programs, including those that might want to be “clever” by looking directly at the object’s contents and possibly manipulating them.)

Moving code into Lisp makes the code easier to debug and maintain and makes it much easier for people who are not XEmacs developers to customize XEmacs, because they can make a change with much less chance of obscure and unwanted interactions occurring than if they were to change the C code.

This document was generated by Aidan Kehoe on December 27, 2016 using texi2html 1.82.