XEmacs Spring 2001 Release: Draft Proposal

From: "Stephen J. Turnbull" <turnbull@sk.tsukuba.ac.jp>
Sender: owner-xemacs-review@xemacs.org
To: xemacs-review@xemacs.org
Message-ID: <14911.16142.556985.689923@turnbull.sk.tsukuba.ac.jp>
Subject: DRAFT: Release Proposal (target date March 1)
Date: Tue, 19 Dec 2000 19:57:18 +0900
X-Mailer: VM 6.84 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid

After all my smoke and flame, I decided I ought to do something
constructive.  I've checked around with those have open projects, and
a few other crucial personnel, and it seems to me that now is a good
time to try to organize a release.  However it works out, I think that
just putting together the information is a contribution.

The March 1 date was suggested by Martin.  I picked it up because
winter is a good time for me to work (another heavy teaching term
starts in April), and Andy and Bill at least would like to get some
closure on the widgets and GTK by then.

I'd appreciate it if everybody on the board could take a look at the
proposal over the holidays.  I know the schedule is tight even without
a break; I also know it's unrealistic to expect everybody to review it
carefully within a couple of days at this time of year (in particular
Vin is already flown away).  I'd like to have at least "thumbs
up/down" reactions by say January 4 so I can whip up a next draft and
maybe have some discussion and decision by January 9.  Any further
discussion would of course be very welcome, especially if there are
projects or showstopper issues that I've missed.

I would like People with Projects (specifically

    Bill - GTK/XEmacs
    Andy - gutters/widgets
    Olivier - portable dumper

and maybe

    Jan - custom themes, if you want in)

to try to give scheduling a close look, especially with respect to
test schedules.  If you can give me a pretty specific idea of how
schedules and code status look, that would be a big bonus.  Anybody
else who has a project you're working on, and would like to include in
the next release, now's the time to speak up.  The window will
definitely close in early January, tentatively the 9th.

I know this is pretty presumptuous of me.  However, on the one hand we
don't have an explicit process for setting a release (see below).  On
the other, I have contacted and received replies from about half the
board concerning the idea of a release.  The replies I have gotten
were all positive; some were very enthusiastic.  No one explicitly
opposed the March 1 date.

Of course, nobody has yet specifically endorsed the proposal below.
None of the statements I attribute to various people below count as
commitments yet; I was just sounding things out.  Everything in the
proposal is subject to change, of course.  But I have to take some
risks if the target is going to be met.

I hope this proposal will meet with your approval and cooperation.

I think it should be clear that I'm volunteering for the role labelled
"Public Release Engineer" in the proposal.  I don't really think, in
the best of all worlds, that I would be the best candidate for the
position.  But I'm the only one I _know_ is available for the period
in question, so I'm backing up the proposal with that commitment.  I
personally propose Martin as the obvious candidate.

I plan to continue to revise the proposal and flesh out some details
over the holidays.  I will post these on my personal web site

      http://turnbull.sk.tsukuba.ac.jp/Tools/XEmacs/21.2-Draft/

until the board has adopted some target; at that point I will move
the proposal/draft plan to www.xemacs.org as the community information
center.  I will continue to maintain it.

Although it's not directly relevant to the release, I would be happy
to act as a clearinghouse for RFCs that are not aimed at this release,
but perhaps at the next one.  This would be sort of a "librarian"
function.  I am specifically _not_ including any "process" proposals
not specifically relevant to the release here, and all processes in
this proposal are ad hoc, only for the duration until the release
stabilizes.  I hope over the next few months we can discuss process
and governance issues, but I am going to try to stay low key (believe
it or not) as long as I have any hand in release management.

Sincere regards, a Merry Christmas and happy holidays to everybody!!

Steve

Here's the proposal:

PREAMBLE

There have been recent rumblings on xemacs-beta and on xemacs-review
about the possibility of a release in the short term.  I think it's
time; I'm going to pick up the ball and try to run with it for a
while.

STATE OF THE PROJECT

On May 14, 1999, after a gestation of about 10 months, XEmacs 21.1.1
was released to the net, followed on the same day by 21.1.2.  Since
then, the 21.1 "stable branch" has had updates on average of every two
months.  As of today, December 19, 2000, Vin is done with 21.1.13 (but
scheduling release for first week of January, immediately after he
returns from vacation).  XEmacs is starting to penetrate the Windows
market, and Mule is considered sufficiently stable and efficient that
several Linux vendors ship it by default.  The package architecture
has proved popular with users, although the separation of the SUMO
tarball of packages from the main distribution occasions several FAQs,
and instabilities in EFS and PGP have occasionally caused problems
with the package user interface.

The first snapshot of XEmacs 21.2 was released on July 19, 1998 as
version 21.2-beta1.  The current snapshot was released by Martin about
two weeks ago, and is 21.2.38 implying a period of about 3 weeks
between snapshots.  A number of coding projects were proposed and
implementation started.  As of today, several are fairly complete, and
have achieved varying degrees of stability.  However, in the summer of
2000 several core developers changed employment status or otherwise
became inactive.  At present a number of developers continue to work
more or less actively on bug-fixing or individual projects.

Recent discussion on XEmacs Review and in private email, with some
leakage to open lists, makes me believe there is a widespread feeling
among testers and developers that the project has lost momentum.
Although partly this is due to the temporary inactivity of core
developers already mentioned, I attribute much of the problem to the
fact that there is no release target.  We don't have a target date, we
haven't decided what the feature goals are, and we don't even have a
clear process for deciding these issues.

I have polled the core developers with active projects.  Those who
have responded so far are unanimously in favor of planning and
implementing a public release.

TARGET DATE

Several developers have scheduling contingencies that make a March 1,
2001 target attractive.  The remainder of those responding so far all
agree that it is a plausible date to target, although a couple have
pointed out that scheduling will have to be very aggressive.

This is a very ambitious target.  However, I believe that a great
degree of stability in the code base as a whole can be achieved in 2.5
months, since several of the features currently being implemented are
basically complete.  Most of these have already been through several
cycles of testing and bug fixing.

RELEASE THEME

In my opinion, most of the projects implemented during the 21.2 cycle
can be called developer-oriented in nature.  Although Bill's GTK
XEmacs, Andy's gutter and widgets, and Jan's custom themes will all be
quite user-visible, achieving full impact will require a lot of work
porting old code and adding new features to take advantage of these
capabilities.  Therefore I propose that the theme of the release be
consolidation and integration of these current projects.  Trying to
actually make use of these features pervasive cannot be done in such a
short period of time.

PERSONNEL ISSUES

The project currently lacks a strong leader.  Whether there should be
such a person is subject to differences of opinion, but right now we
have no explicit mechanism to replace the "benevolent dictator".  In
my opinion, for a release to work, some one person is going to have to
be given responsibility for the release, and a certain amount of
authority.

I propose that we appoint a Public Release Engineer/Product Manager as
an explicitly term-limited position.  See the "job description" for
this position below for rationale.  Martin is an obvious candidate.
Ben might be expected to come forward, but he says that he prefers to
concentrate implementation of Mule on the Windows platform.  Martin
has proposed me.  Other candidates are welcome to step forward.

The term of the Public Release Engineer is up when Vin (or his
successor) accepts handover of the release as "stable."  All
experience with free software suggests that no matter what we call it,
the first version or so after the public release will hardly be
"stable."  Vin has been doing an excellent job so far on 21.1.  I want
to avoid screwing the system up by asking him to take responsibility
for the 21.2 body of code before _he_ thinks it's time.

Vin has expressed some reservations due to his own time constraints.
My impression is that he is willing and (as far as he knows) able to
continue as Stable Release Manager until the transition has been
achieved.

RELATED, BUT OUT-OF-SCOPE, ISSUES

There are several activities of the XEmacs project that are not
directly related to new development, including the stable release
series, the Web site, and the packages.  I consider these all to be
successful, ongoing activity, and do not mean to imply that they are
unimportant by not mentioning them in this proposal.  Rather, I
consider their success and independence to be signs that the
reorganization of the project that Steve Baur spearheaded has been in
large part successful.

Several people have voiced concerns about the "drift" in the project.
Making "consolidation" the theme of the release is disappointingly
unambitious to me.  It suggests a lack of any long term goals in the
project.  I think a release is feasible, as described above, and
desirable to help recover momentum.  However, a release is
insufficient.  We should adopt some goals more specific than "further
development" by the time the release is complete.  But what those
should be is out of the scope of this proposal and its discussion.

The background of the proposal for appointing a Public Release
Manager/Meta-Maintainer Pro Tem clearly implies certain issues about
governance of the project, in particular revising jobs.html.  We also
need to clarify the role of the board in processes like this one, and
of the notorious "veto" which board members may exercise.  These are
out of the scope of this proposal, except to note that if they aren't
settled by the time of the public release, I expect the project will
once again lose momentum and settle into a malaise at that point.

PUBLIC RELEASE ENGINEER RESPONSIBILITIES

Develop/jobs.html says that one responsibility of the Beta Release
Maintainer is "(c) to decide when it's time to do an external release.
At this point, he is responsible for initiating a code freeze and
assuming quasi-dictatorial power over the development process, doing
whatever it takes to ensure that a relatively stable, bug-free release
gets put out in a timely fashion.  If some important job is not
getting done during the critical pre-release period, it is ultimately
the duty of the beta release maintainer (in conjunction with the
meta-maintainer) to ensure that this gets done, for example, by
finding somebody else who is willing to take over this position, or
(last resort) doing the job himself."

However, Martin has explicitly repudiated responsibility for
management of public releases, at least as part of his long-term
position as Beta Release Maintainer.  Furthermore, neither Ben nor
Martin is currently functioning as meta-maintainer.  Neither one seems
likely to do so in the foreseeable future for a number of reasons.
Ben hopes to be active in development, especially with respect to
implementation of Mule on the Windows platform, but his time and
energy will be limited, partly for health reasons.

In my opinion, this means that "quasi-dictatorial power" is not an
appropriate description of what will be available.  What I propose to
do is to get the Review Board as a whole to endorse a release effort
which will be more or less ad hoc, managed by a "Public Release
Engineer."  For this release, the Public Release Engineer should also
function as Meta-Maintainer Pro Tem, attempting to fill the permanent
meta-maintainer position at least.

In terms of the release process, the Public Release Engineer will be
responsible for

1.  Chairing discussion of the release process, including discussion
    of decisions about including or postponing features to the next
    cycle.

2.  Keeping status information up to date, including nagging People
    With Projects to communicate their projects' statuses.  Initially,
    this means polling known People With Projects, checking the BTS
    and the mailing lists for outstanding issues we should try to deal
    with, and reviewing Ben's Mule on Windows code (because he is
    already out of touch, and will not be back in touch until after
    the date by which a status report on this code should be presented
    to the board).

3.  Communicating with non-reviewer developers, beta testers, and the
    general community about the release process.  This includes
    drafting and editing RFPs and RFCs, incorporating them into a
    "release bulletin board" to monitor progress, and informing the
    Review Board if a "Person With Project" seems to have disappeared,
    implying that their feature may have to be backed out.

4.  Producing timely (weekly?) beta test tarballs.

5.  Deciding when to go to "gamma" status.

6.  Writing NEWS and similar descriptive stuff.

7.  Negotiating promotion to "stable" status with the Stable Release
    Maintainer.

8.  Possibly kicking off a new release cycle, with a (probably) new
    Public Release Engineer.

The Public Release Engineer may of course delegate any of these
responsibilities; he just has to make sure they get done.  In
particular, Martin has said he will continue to actually roll the
tester tarballs.  And I will be doing the research in the BTS and
Ben's code regardless of whether I end up as Public Release Engineer
or not.  I will volunteer to continue doing that kind of spadework
for whoever is named Release Engineer.

FEATURES/PROJECTS PROPOSED FOR INCLUSION IN THE NEXT RELEASE

1. GTK XEmacs (Bill P) is fairly stable as a branch from 21.1.  It is
achieving some popularity on Linux, and is being distributed by one or
more of the Linux distributions as an alternative to FSFmacs and
XEmacs.

Bill says that the basic port to 21.2 is done, but it core dumps more
or less immediately.  He believes it can be stable by March 1.  He
proposes it as a configure option for the public release.  (Making it
a default is clearly a big change in policy.)

2. The portable dumper (OG) is (conservatively) 80-90% finished.  It
works for me, and for several people on the NT platform.  There is a
bug reported with respect to problems with buffer local variables (I
don't understand the details).

Olivier has said he doesn't know about anything else, but he (again
conservatively) won't rule out other problems.  I haven't received a
specific statement from him about feasibility for March 1.  However,
this feature is considered extremely important for the Windows
platform, and truly reverting this feature to the status quo ante
would require reimplementing pure space, which seems like too much
work to be done by March.  So defaulting it off on non-Windows
platforms seems to be the feasible fallback option.

I propose at least running through the first part of the test cycle
with the pdumper on by default.

3. The gutters/widgets (Andy) are apparently stable for Windows NT
(cygwin, mingwin, native).

Andy now says (conservatively) he doesn't know enough to say what will
happen with the X version; he would be happy to default them to off in
a release on X platforms.  The progress bar and buffer tabs are the
only applications still.  On the other hand, Olivier (among others)
likes them, and the people he supports are unanimously in favor.
They're apparently rather popular on Windows.

I propose having them on by default and requesting the testers to turn
them back on if they've explicitly turned them off, and try to get the
widgets working properly on X.  Bill sees interactions with GTK, and
Ben says he _may_ have time to help with them starting in January.

4. Mule on Windows (Ben).  I have contacted Ben by phone.  He has a
lot of partially finished code.  He has abstracted a lot of the
file-coding stuff, and is doing other general Mule clean-up.  He is
currently traveling but expects to be back to work in mid-January.
His condition was quite poor in summer and fall, but it seems to have
been stable since October.  He hopes that his current treatment regime
will enable him to work more than he was able to in spring and early
summer.

He says that he does not intend to function as "meta-maintainer" or
"architect" at this point.

He says that the work is "basically done" and, health permitting, he
will be able to integrate this work and have it stable for a March 1
release.  I explicitly propose relaxing the deadlines given below by a
week in this case.  (My personal interest is showing here, of course,
but several other reviewers have expressed interest in this feature.)
I do plan to review the code he already has written and report before
he returns; I ask that the Board postpone discussion of the
feasibility of the schedule of this feature until then.  (This code is
available as CVS branches dated around June, with "ben" in the tag,
and as tarballs on ftp.xemacs.org.  It does not currently build and
run on Linux.)
 
5. Hisashi Miyashita (himi@m17n) is, I believe, a colleague of
Martin's at ETL.  He's working on improving CCL and synching it to the
recent FSF, but there is no spec for this so I don't know what it
actually means.  I will be communicating with him, but as far as I
know this work is proceeding in such a way that it does not change
APIs or non-error semantics of currently correct code.  Besides
general cleanup and some extensions amounting to "syntactic sugar", it
also permits use of his Mule-UCS package on very recent XEmacsen to
read and write Unicode streams.

6. Yoshiki Hayashi has been working on various Mule issues.  Yoshiki
has done some work on case-tables.  These now work for Latin-2.  They
should work for all Mule charsets, except that the necessary tables
have not yet been coded.

This work should be included in the release.  I hope we can find some
volunteers to cons up tables for various charsets we support that need
case tables.

7. Steve Baur's implementation of PostgreSQL may not be The Right
Thing[tm] but it seems to be solid and working.  I propose including
the feature, and autodetecting by default.  Rationale: I have looked
at MySQL and ODBC briefly, and there simply seems to be no alternative
to DBMS-specific C interfaces.  Furthermore, these interfaces are
sufficiently different that Lisp-level code must be written in a
DBMS-specific way.  Of course, the queries themselves will be in SQL
and need not be DBMS-specific, except that different DBMSes implement
different parts of SQL, and some implement some functions more or less
efficiently.  I am not a specialist, but my impression is that all
this means that supporting a different Lisp interface for each
database is necessary in any case.

8. We should consider reversing Steve's decision to not distribute the
SUMO Lisp in the main tarball.  I don't know how much work this would
involve.  It could be as little as a change to INSTALL ("we don't
support any means of installation that doesn't involve untarring the
SUMO in $prefix/lib/xemacs; if you want to avoid that, read
README.packages").  It could involve somehow including the SUMO in the
core distribution.  Or something in between (don't ask me what yet).

The packages are popular, but the typical user of the core tarball
wants the kitchen sink.  The package system is used to keep up-to-date
on heavily used packages, or packages with important new features.
People who want a subset typically are either capable implementing
that themselves anyway, or should let their Linux distro deal with the
issue, or can remove unwanted packages ex-post install via package-ui.

9. Documentation.  I hope there will be lots of documentation added!

PROJECTS THAT AREN'T PLANNED FOR INCLUSION

Some projects that have been proposed that I think should be postponed
until a future release:

1. Custom themes (Jan) are in unknown state.  Jan has not replied to
my inquiry to him, nor has he been participating in discussion about
them in other fora according to inquiries I have received.  I propose
that the related code be removed from the release version, and
inclusion postponed to the next release cycle.

2. External modules (J. Kean Johnston) have been static for a while.
Stable?  I don't know.  The base64 module works for me.  Jerry James
has a project that uses them.  But I don't believe they're getting
thorough testing, the sample module never got implemented, and the
eudc and zlib modules are partial implementations AFAICT.

Kean has not responded to an inquiry about status.  I know that Hrvoje
at least objected to the external modules on the basis that they expose
the C interfaces to "user" code, restricting developer flexibility in
changing C code.  Unless there is sufficiently strong support to
override Hrvoje's objection immediately, I propose that the code be
left in and advertised as a feature available for experimentation, but
the autoconf feature be off by default.  There is insufficient time
and energy for extensive discussion of this matter for a March 1
release.

3. Yoshiki Hayashi is planning to do some work on the ISO 8559-X
coding systems in response to Hrvoje's long-running complaints about
decoding ISO 2022 escape sequences.

This work has not yet been started.  Time is short and it definitely
involves user-visible changes in semantics.  Although these changes
are dear to Hrvoje's heart, and I know Yoshiki supports them, I
reluctantly propose that ISO 2022 decoder fixes be postponed until the
following release cycle.

4. UTF-2000 (Unicode as internal encoding).  I've been puttering away
at this, but Tomo keeps getting farther and farther away from a
production design, and I keep finding things that just suck about it.

5. fill.el.  This is just plain broken right now.  I've been working
on this, but will not have time to get it right.  It has potential for
being annoyingly broken for a long time.  This has to wait.

6. External Unicode support via ucs-conv or Mule-UCS.  I plan to do
something about this lack, and there is internal support for
recognizing Unicode encodings.  However, the actual encoding and
decoding routines, related tables, etc, can be done as a package.
Both ucs-conv and Mule-UCS suffer from certain Japanese biases as far
as I can tell.  Putting them in the core at this time is not a good
idea, although some kind of Unicode support needs to be in the core
soon.

TENTATIVE TIMETABLE

December 19:  Submit draft proposal to the board; request detailed
              status reports from people with projects (PWPs); feature
              slush/RFP to board.  RFP means no completely new
              features, only features with some history of discussion,
              design complete, and preferably prototype code, are
              acceptable.

January 2:    Post revised draft on www.xemacs.org; RFC to xemacs-beta;
              nag PWPs.

January 9:    Close board-level RFPs == full feature freeze;
              re-revise and post draft proposal on comp.emacs,
              comp.emacs.xemacs; post known bug/issue list to
              www.xemacs.org.

January 16:   Maybe open temporary devel branch for the totally
              unreconstructed.  Weekly tarballs that build and run
              from now until release.  Bug/issue list updated at least
              weekly.

February 1:   Code expected to be complete.  Interim reports from
              PWPs; theoretically triage the features (unfortunately,
              I don't think any of our current devel features are
              easily reverted, no way to go but forward).  Reports
              include known issues for testers to bang on.

              Announce explicit testing goals (bugs, issues, features)
              to xemacs-beta.  Update frequently.

              Since GTK is not going to be the default in this
              release, I propose that GTK code and its basic debugging
              be complete at this point, and that GTK be turned on as
              autodetect for beta testers on X11 platforms until February
              15.  (I _have not_ cleared this with Bill Perry.  It
              seems like a plausible way to deal with the "GTK is not
              on by default" issue; discussion welcomed.)

February 15:  Final reports from PWPs: code complete and design bugs
              cleared.  Triage features not meeting this criterion.
              Revert tester default to not-GTK.

March 1:      "Release" 21.3.1 as "gamma"; move devel code to trunk as
              21.4, open 21.3 as new stable branch, branch 21.1 off
              trunk.  21.1 will continue to be supported as the
              "stable" branch, with hopefully no more releases until:

April 2:      Acceptance of 21.3.x as stable by Vin; mothball 21.1.
              My birthday.  ;-)


-- 
University of Tsukuba                Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Institute of Policy and Planning Sciences       Tel/fax: +81 (298) 53-5091
_________________  _________________  _________________  _________________
What are those straight lines for?  "XEmacs rules."
Conform with <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
Automatically validated by PSGML