Request for Comments: 4228 The Measurement Factory
Category: Informational December 2005
Requirements for an IETF Draft Submission Toolset
Status of this Memo
This memo provides information for the Internet community. It does
not specify an Internet standard of any kind. Distribution of this
memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2005).
Abstract
This document specifies requirements for an IETF toolset to
facilitate Internet-Draft submission, validation, and posting.
Table of Contents
1. Introduction ....................................................2
2. Scope ...........................................................2
3. Notation and Terminology ........................................3
4. Status Quo ......................................................4
5. Overall Toolset Operation .......................................6
6. Upload Page .....................................................9
7. Check Action ....................................................9
7.1. Preprocessing .............................................10
7.2. Processing ................................................11
7.3. Storage ...................................................11
7.4. Extraction ................................................12
7.5. Validation ................................................13
7.5.1. Absolute Requirements ..............................14
7.5.2. Desirable Features .................................15
7.5.3. DoS Thresholds .....................................17
7.5.4. WG Approval ........................................17
8. Check Page .....................................................18
8.1. External Meta-Data ........................................19
9. Post Now Action ................................................20
9.1. Receipt Page ..............................................20
10. Adjust Action .................................................21
11. Adjust Page ...................................................21
12. Post Manually Action ..........................................22
13. Receipt Page ..................................................22
14. Bypassing the Toolset .........................................22
15. Email Interface ...............................................23
16. Implementation Stages .........................................25
17. Testing .......................................................26
18. Security Considerations .......................................27
19. Compliance ....................................................27
Appendix A. Comparison with Current Procedures ....................28
Appendix B. Acknowledgements ......................................29
Normative References ..............................................30
Informative References ............................................30
1. Introduction
Public Internet-Drafts are the primary means of structured
communication within the IETF. Current Internet-Draft submission and
posting mechanisms hinder efficient and timely communication while
creating an unnecessary load on the IETF Secretariat. The IETF Tools
team recommends formalization and automation of the current
mechanisms. This document contains specific automation requirements.
The IETF Secretariat and many IETF participants have long been
proponents of automation. This document attempts to reflect their
known needs and wishes, as interpreted by the Tools team.
2. Scope
The Draft Submission Toolset discussed in this document is about
getting a single new version of an Internet-Draft from an IETF
participant to the IETF draft repository. A single draft version may
include several formats, and dealing with those formats is in scope
for the Toolset. Definition and sources of draft meta-information
(to be used in Secretariat databases and elsewhere) are in scope.
Submitter authentication and submission authorization are in scope.
Draft posting may result in various notifications sent to interested
parties. While this document recommends a subset of notification
targets, details of notifications are out of scope.
Creation of new drafts or new draft versions as well as manipulation,
visualization, and interaction with the drafts already in the
repository are out of scope. Draft expiration and archiving of old
draft versions are out of scope.
The set of requirements in this document is not meant to be
comprehensive or final. Other IETF documents or procedures may
require additional functionality from the Toolset. For example, it
is possible that the Toolset will be required to handle draft source
formats other than plain text and XML.
3. Notation and Terminology
The following terms are to be interpreted according to their
definitions below.
posted draft: A draft accepted into the public IETF draft repository
and, hence, publicly available from the IETF web site. Posting of
a draft does not imply any IETF or IESG review and endorsement.
draft version: A meant-to-be-public snapshot of an Internet-Draft
with a meant-to-be-unique version number. Also known as "draft
revision".
draft format: Any draft source or presentation format, including
original and preprocessed XML, original or generated plain text as
well as PDF, PostScript, and HTML formats.
primary draft format: The first available draft format from the
following list: plain text, PDF, PostScript, or XML.
WG-named draft: A draft for which identifier (a.k.a. filename) is
known and starts with "draft-SPECIAL-", where SPECIAL is one of
the following strings: "ietf", "iab", "iesg", "rfc-editor", or
"irtf". Abbreviated as "WGN draft". Exceptions notwithstanding,
WG-named drafts are usually controlled by IETF working groups or
similar IETF-related bodies (and vice versa). The handling of
such naming exceptions is outside of this document’s scope.
individual draft: A draft other than a WGN draft.
submitter: A human or software agent initiating submission of an
Internet-Draft version for validation or posting. In some cases,
the Secretariat staff does the actual submission, but always on
behalf of a submitter. In some cases (including but not limited
to malicious attacks), the submitter is not the draft author.
expected submitter: A submitter that is authorized by IETF rules to
post a given draft. This includes a draft author or editor
(listed in the draft text), a corresponding WG Chair, or an IESG
member.
authorized submitter: An expected submitter authenticated by the
Toolset. Authentication is initially limited to verifying
submitter access to submitter’s email address.
immediately: Without human interaction or artificial software delays
and within a few seconds.
The Toolset is specified using a set of normative requirements.
These requirements are English phrases ending with an "(Rnnn/s)"
indication, where "nnn" is a unique requirement number, and "s" is a
single-letter code ("a", "b", or "c") specifying the implementation
stage for the requirement. Implementation stages are documented in
Section 16.
This document specifies the interface and functionality of the
Toolset, not the details of a Toolset implementation. However,
implementation hints or examples are often useful. To avoid mixup
with Toolset requirements, such hints and examples are often marked
with a "Hint:" prefix. Implementation hints do not carry any
normative force, and a different implementation may be the best
choice.
4. Status Quo
This section summarizes the process for draft submission and posting
as it exists at the time of writing.
To get an Internet-Draft posted on the IETF web site, an IETF
participant emails the draft text to the IETF Secretariat, along with
an informal note asking the Secretariat to post the draft.
Secretariat staff reads the note, reviews the draft according to a
checklist, and then approves or rejects the submission. Draft
approval triggers the corresponding announcement to be sent to
appropriate IETF mailing lists. Every 4 hours, approved drafts are
automatically copied to the IETF drafts repository and become
available on the IETF web site.
Collectively, IETF participants submit thousands of Internet-Drafts
per year (in the year 2000, about 3,000 drafts were submitted; 2002:
5k; 2004: 7k [secretariat]). About 30-50% of posted drafts are
WG-named drafts (among some 2,100 drafts, there were about 380 new
and 290 updated WGN drafts posted in 2003). While no rejection
statistics are available, the vast majority of submitted drafts are
approved by the Secretariat for posting.
It usually takes the Secretariat a few minutes to review a given
draft. However, since the Secretariat staff does not work 24/7, does
not work in all time zones, and has other responsibilities, and since
approved drafts are posted in batches every 4 hours, it may take from
several hours to several days to get a draft posted. Due to much
higher demand and fixed processing capacity, postings during the last
weeks before IETF face-to-face meetings take much longer, creating a
long queue of unprocessed drafts that are then announced nearly
simultaneously.
To give IETF face-to-face meeting participants time to review
relevant documents, the Secretariat does not accept Internet-Draft
submissions close to IETF meetings (regardless of whether a draft is
relevant to the upcoming meeting or not).
Many Working Groups have come up with ad hoc solutions to cope with
posting delays. For example, many draft snapshots are "temporarily"
published on personal web sites or sent (completely or in part) to
the group list. Alternative means of publication may effectively
replace official IETF interfaces, with only a few major draft
revisions ending up posted on the IETF web site.
Informal interfaces for submitting and posting drafts discourage
automation. Lack of submission automation increases Secretariat
load, complicates automated indexing and cross-referencing of the
drafts, and, for some authors, leads to stale drafts not being
updated often enough.
Beyond a short Secretariat checklist, submitted drafts are not
checked for compliance with IETF requirements for archival documents,
and submitters are not notified of any violations. As a result, the
IESG and RFC Editor may have to spend resources (and delay approval)
resolving violations with draft authors. Often, these violations can
be detected automatically and would have been fixed by draft authors
if the authors knew about them before requesting publication of the
draft.
Technically, anybody and anything can submit a draft to the
Secretariat. There is no reliable authentication mechanism in place.
Initial submissions of WGN drafts require WG Chair approval, which
can be faked just like the submission request itself. No malicious
impersonations or fake approvals have been reported to date, however.
Lack of authentication is not perceived as a serious problem,
possibly because serious falsification are likely to be noticed
before serious damage can be done. Due to the informal and manual
nature of the submission mechanism, its massive automated abuse is
unlikely to cause anything but a short denial of draft posting
service and, hence, is probably not worth defending against.
However, future automation may result in a different trade-off.
5. Overall Toolset Operation
This section provides a high-level description for the proposed
Toolset. The description is meant to show overall operation and
order; please refer to other sections for details specific to each
step.
A typical submitter goes through a sequence of 2-4 web pages and
associated actions. The number of pages depends on the draft
validation and meta-data extraction results. For example, validating
the draft without posting it requires interacting with two web pages:
Upload and Check. The common case of posting a valid draft without
manual meta-data adjustments takes three web pages (Upload, Check,
Receipt).
Here is a brief overview of pages and actions:
Upload page: The interface to copy a draft from the submitter’s
computer to the Toolset staging area (Section 6). Multiple
formats are accepted. The draft is sent to the Check action.
Check action: Stores the draft in the Toolset staging area, extracts
draft meta-data, validates the submission (Section 7). Produces
the Check page.
Check page: Displays draft interpretation and validation results
(Section 8). A draft preview may also be given on this page.
After reviewing the draft interpretation and validation results,
the submitter has four basic choices: (a) auto-post draft "as is"
now; (b) make manual corrections and submit the draft to
Secretariat for manual posting later; (c) cancel submission; or
(d) do nothing. The automated posting option may not be available
for drafts with validation errors.
Automated posting: If the submitter decides to proceed with automated
posting from the Check page, the system authenticates the
submitter and may also check whether the submitter is allowed to
post the draft. If the submitter is authorized, the draft is
immediately posted, deleted from the staging area, and the
submitter is notified of the result via email and a Receipt page
(Section 9).
Manual adjustment and posting: If the submitter decides to adjust the
meta-data, the draft remains in the Toolset staging area, and the
Adjust action (Section 10) presents the submitter with an Adjust
page (Section 11). When the submitter makes the adjustments and
proceeds with manual posting, a pointer to the stored draft and
its adjusted meta-data is sent to the Secretariat for manual
processing (Section 12). The submitter is notified of the pending
Secretariat request via email and a Receipt page.
Cancellation: If the submitter decides to explicitly cancel the
submission, the submission state (including the draft) is
immediately deleted from the Toolset staging area and an
appropriate Receipt page is generated without further actions
(R123/a). Cancellation of posted drafts is out of this document
scope.
Receipt page: Contains details of a successful or failed draft
submission and informs the submitter of the next appropriate
step(s) related to submission result.
The following informal diagram illustrates the basic submission
logic:
/---> Post Now
/
Upload --> Check -+-----> Adjust ---> Send to Secretariat
\
\---> Cancel
If the submitter does nothing while the Toolset is expecting some
response, the abandoned submission times out (R124/a). The timeout
value depends on the submission state. Hint: A timeout value of one
hour is probably large enough unless the Toolset is waiting for some
kind of a 3rd party confirmation (e.g., WG Chair approval). Doing
nothing is functionally equivalent to explicitly canceling the
submission, except that explicit cancellation requires immediate
removal of submission state while the state of submissions marked as
abandoned is garbage-collected.
The staging area maintenance algorithms must keep the area in a
consistent, correct state in the presence of denial-of-service (DoS)
attacks attempting to overwhelm the area with fake submissions in
various stages (R67/a). Hint: denial of service to legitimate users
is acceptable under DoS attack conditions, but corruption of the
storage area is not.
The "web pages" this text is referring to are distinct dialogs that
may be visible to the submitter under the same or different URL and
that are supported by a single or several server-side programs.
The Toolset must handle multiple submitters simultaneously submitting
the same draft (R72/a) and a single submitter simultaneously
submitting two drafts (R73/a). The latter might happen, for example,
when the submitter is using several browser windows to submit several
drafts or is submitting drafts via email interface. The term
"simultaneously" means that submission processing times overlap.
Hint: Except for the Upload page, pages contain a submission session
identifier to provide actions with access to stored information. The
identifier is specific to the submission rather than the draft
version or the submitter. While the nature of the web interface
allows the session identifier to be invisible to the submitter, email
communication would need to identify the session so that the
recipient (and Toolset) know the context.
Hint: A single action may correspond to multiple server-side programs
and, vice versa, a single program may implement several actions.
This document does not mandate any specific technology (e.g., Common
Gateway Interface (CGI), PHP, and/or Java servlets) to implement
server-side support. The implementer experience, code reuse across
web and email interfaces, and other factors will determine the right
technology choice.
Hint: Actions preserve and exchange state by storing it along with
the draft. Grouping all submission-specific information in one
subdirectory named using the session identifier may increase
robustness and simplify debugging. Session creation and destruction
can then be logged in a global index.
Ways to partially or completely bypass the Toolset are documented in
Section 14.
It must be possible to transfer the Toolset from one management team
to another, to incorporate work by volunteers, and to allow for
public review of the developed code. To meet these goals, the
Toolset source codes should be publicly available (R152/b) and there
should be an interface to report bugs and request enhancements
(R145/b). Development should be structured to avoid lock-in to
proprietary platforms or backends. The Tools team believes that
developing the Toolset sources under one or more open source licenses
following the Open Source Definition [OSD] would provide an effective
way of meeting these requirements at reasonable cost. Care should be
taken that the licenses selected allow code from different
implementers to be mixed.
Hint: Placing the Toolset source repository at an
open-source-friendly project management site like SourceForge.net
would provide the IETF community with a decent, ready-to-use
interface to access the code, documentation, bug reports, and
discussion forums. Establishing and documenting a simple interface
between the Toolset and external software (e.g., the Secretariat
draft posting scripts) would facilitate availability checks.
The Toolset is meant to be compatible with the Secretariat’s tools
for handling drafts. Hint: Such compatibility can be achieved by
appropriately implementing the Toolset or, in some cases, by
modifying existing Secretariat tools.
6. Upload Page
To upload a draft, the submitter goes to a well-known page on the
IETF web site (R1/b). There, the draft text can be uploaded using an
HTML file upload form. This form provides fields to upload the plain
text format of the draft (R2/a) and all other formats allowed by IETF
draft publication rules (R3/b). At the time of writing, these
formats are: XML ([RFC2629] and [writing-rfcs]), PDF, and PostScript.
Submitted forms are handled by the Check action documented in
Section 7.
The Upload page also has a validate-only flag, indicating that an
uploaded draft must not be posted and may be deleted immediately
after the validation (R74/b). Regardless of the validation results,
the stored draft meta-data is marked so that validation-only drafts
can be identified and deleted first by garbage collector for the
Toolset staging area (R75/b). Drafts uploaded in a validate-only
mode cannot be posted (R76/b); they would need to be uploaded again,
without the validate-only flag, and the validation results page
should explain that (R77/b). This flag is useful for tools using
online validation, especially for bulk draft processing. Hint: it
may be better to implement this flag as a hidden HTML input field to
simplify the interface for human submitters.
7. Check Action
The Check action preprocesses a submission, generates plain text
format (if needed, see R70), stores the submitted draft (all formats)
in the staging area, and then extracts meta-data and validates each
format (R6/a). Errors and warnings are indicated to the submitter in
the response via computer-friendly tag(s) and human-friendly text
(R7/a).
If any error is found, automated posting becomes impossible (R113/a).
This rule applies to all errors, even those that do not refer to R113
and do not explicitly prohibit automated posting. If automated
posting is not possible, the Toolset still gives the submitter an
option of sending the draft for manual validation and posting
(R114/a). Since each submission is treated in isolation, the
submitter also has an option of correcting the problem and
resubmitting the draft for automated posting.
The manual validation and posting route is a Toolset bypass mechanism
(see Section 14) not meant for fixing problems with the draft itself.
The Secretariat does not generally correct submitted drafts. If the
draft needs tweaking to match submitter’s intent, then the draft
should be corrected by the submitter and resubmitted.
It is an error to submit a draft that has neither plain text nor XML
source format (R68/a). XML source is acceptable without accompanying
plain text only if the Toolset successfully generates a draft in
plain text format from the XML source, as a part of the processing
step documented below (R69/b). These rules imply that PDF- or
PostScript-only drafts cannot be auto-posted. Moreover, even manual
Secretariat involvement cannot help with posting these drafts, as the
IETF policy is to always require a plain text format in addition to
PDF or PostScript. Furthermore, drafts containing PDF or PostScript
format must not be auto-posted until the Toolset can validate that
their content matches the plain text format (R143/a).
The draft format acceptance rules above are meant to decrease the
chances that multiple posted draft formats for a single draft contain
substantially different documents. With experience, the rules may be
simplified so that, for example, only submissions containing nothing
but XML or plain text sources can be posted without Secretariat
involvement and all other submissions require manual actions to match
formats or extract meta-data.
7.1. Preprocessing
Submitting compressed drafts is a desirable feature, especially for
submitters behind slow or content-altering links. Compressed draft
formats may be accepted (R150/c). Compressed formats, if any, must
be decompressed during the preprocessing step (R151/c) so that other
processors do not have to deal with compressed formats. Hint: While
this specification does not document a list of supported compression
standards, it is expected that such popular methods as "zip" and
"gzip" should be accepted if compression is supported. Accepting a
collection of draft formats within a single compressed archive may
also be desirable.
XML source containing XML processor <rfc? include="..."> instructions
(PIs) is preprocessed to include references (R8/b). This step is
needed to remove external dependencies from XML sources and to
simplify tools processing posted XML. This document refers to such
XML processor instructions as "include PIs".
The XML preprocessor uses public database(s) to resolve PI references
(R85/b). The Toolset documentation specifies what databases are used
and how PIs are mapped to database entries (R86/b). The Toolset must
not rely on PIs’ existence (R87/b) because some XML sources will be
preprocessed before the submission or will be written without PIs.
Hint: Local up-to-date copies of Marshall Rose’s reference databases
at xml.resource.org can be used.
Both original and preprocessed XML sources may be posted later. The
original source with include PIs may be useful to the RFC Editor and
generation of diffs (against future or past original sources). The
preprocessed source without include PIs becomes the default public
XML source of the posted draft (R10/b). If any of the include PIs
known to the Toolset cannot be handled, an error is recorded (R11/b),
and the submitter is encouraged to do the preprocessing locally,
before submitting the draft (R111/b).
Uncompressed draft formats other than XML are not preprocessed.