Elisp Compatibility Package

Owner: ???

Effort: ???

Dependencies: ???

Abstract: ???

A while ago I created a package called Sysdep, which aimed to be a forward compatibility package for Elisp. The idea was that instead of having to write your package using the oldest version of Emacs that you wanted to support, you could use the newest XEmacs API, and then simply load the Sysdep package, which would automatically define the new API in terms of older APIs as necessary. The idea of this package was good, but its design wasn't perfect, and it wasn't widely adopted. I propose a new package called Compat that corrects the design flaws in Sysdep, and hopefully will be adopted by most of the major packages.

In addition, this package will provide macros that can be used to bracket code as necessary to disable byte compiler warnings generated as a result of supporting the APIs of different versions of Emacs; or rather the Compat package strives to provide useful constructs to make doing this support easier, and these constructs have the side effect of not causing spurious byte compiler warnings. The idea here is that it should be possible to create well-written, clean, and understandable Elisp that supports both older and newer APIs, and has no byte compiler warnings. Currently many warnings are unavoidable, and as a result, they are simply ignored, which also causes a lot of legitimate warnings to be ignored.

The approach taken by the Sysdep package to make sure that the newest API was always supported was fairly simple: when the Sysdep package was loaded, it checked for the existence of new API functions, and if they weren't defined, it defined them in terms of older API functions that were defined. This had the advantage that the checks for which API functions were defined were done only once at load time rather than each time the function was called. However, the fact that the new APIs were globally defined caused a lot of problems with unwanted interactions, both with other versions of the Sysdep package provided as part of other packages, and simply with compatibility code of other sorts in packages that would determine whether an API existed by checking for the existence of certain functions within that API. In addition, the Sysdep package did not scale well because it defined all of the functions that it supported, regardless of whether or not they were used.

The Compat package remedies the first problem by ensuring that the new APIs are defined only within the lexical scope of the packages that actually make use of the Compat package. It remedies the second problem by ensuring that only definitions of functions that are actually used are loaded. This all works roughly according to the following scheme:

Part of the Compat package is a module called the Compat generator. This module is actually run as an additional step during byte compilation of a package that uses Compat. This can happen either through the makefile or through the use of an eval-when-compile call within the package code itself. What the generator does is scan all of the Lisp code in the package, determine which function calls are made that the Compat package knows about, and generates custom compat code that conditionally defines just these functions when the package is loaded. The custom compat code can either be written to a separate Lisp file (for use with multi-file packages), or inserted into the beginning of the Lisp file of a single file package. (In the latter case, the package indicates where this generated code should go through the use of magic comments that mark the beginning and end of the section. Some will say that doing this trick is bad juju, but I have done this sort of thing before, and it works very well in practice).
The functions in the custom compat code have their names prefixed with both the name of the package and the word compat, ensuring that there will be no name space conflicts with other functions in the same package, or with other packages that make use of the Compat package.
The actual definitions of the functions in the custom compat code are determined at run time. When the equivalent API already exists, the wrapper functions are simply defined directly in terms of the actual functions, so that the only run time overhead from using the Compat package is one additional function call. (Alternatively, even this small overhead could be avoided by retrieving the definitions of the actual functions and supplying them as the definitions of the wrapper functions. However, this appears to me to not be completely safe. For example, it might have bad interactions with the advice package).
The code that wants to make use of the custom compat code is bracketed by a call to the construct compat-execute. What this actually does is lexically bind all of the function names that are being redefined with macro functions by using the Common Lisp macro macrolet. (The definition of this macro is in the CL package, but in order for things to work on all platforms, the definition of this macro will presumably have to be copied and inserted into the custom compat code).

In addition, the Compat package should define the macro compat-if-fboundp. Similar macros such as compile-when-fboundp and compile-case-fboundp could be defined using similar principles). The compat-if-fboundp macro behaves just like an (if (fboundp ...) ...) clause when executed, but in addition, when it's compiled, it ensures that the code inside the if-true sub-block will not cause any byte compiler warnings about the function in question being unbound. I think that the way to implement this would be to make compat-if-fboundp be a macro that does what it's supposed to do, but which defines its own byte code handler, which ensures that the particular warning in question will be suppressed. (Actually ensuring that just the warning in question is suppressed, and not any others, might be rather tricky. It certainly requires further thought).

Note: An alternative way of avoiding both warnings about unbound functions and warnings about obsolete functions is to just call the function in question by using funcall, instead of calling the function directly. This seems rather inelegant to me, though, and doesn't make it obvious why the function is being called in such a roundabout manner. Perhaps the Compat package should also provide a macro compat-funcall, which works exactly like funcall, but which indicates to anyone reading the code why the code is expressed in such a fashion.

If you're wondering how to implement the part of the Compat generator where it scans Lisp code to find function calls for functions that it wants to do something about, I think the best way is to simply process the code using the Lisp function read and recursively descend any lists looking for function names as the first element of any list encountered. This might extract out a few more functions than are actually called, but it is almost certainly safer than doing anything trickier like byte compiling the code, and attempting to look for function calls in the result. (It could also be argued that the names of the functions should be extracted, not only from the first element of lists, but anywhere symbol occurs. For example, to catch places where a function is called using funcall or apply. However, such uses of functions would not be affected by the surrounding macrolet call, and so there doesn't appear to be any point in extracting them).

Ben Wing

Conform with <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
Automatically validated by PSGML