7.2. Processing
When no plain text format of the draft is submitted, but XML sources
are available, the Toolset attempts to generate plain text format
from submitted XML sources (R70/b).
If XML sources are available, the Toolset generates HTML draft format
(R112/c). HTML generation failures should result in warnings, not
errors (R115/c). HTML generation is not meant to be implemented
until the Enhancement Stage is reached (R130/a). In general, HTML
generation is desirable because HTML drafts are usually easier to
navigate than plain text drafts due to improved overall readability
and links. As any Enhancement Stage feature, HTML generation may be
dropped or drastically changed to reflect then-current IETF consensus
and the experience of the first two implementation stages.
Hint: The Toolset implementers should not assume that draft formats
generated by the same tool from the same source format have
essentially the same content. The generation tool may have options
that allow authors to generate content exclusive to a specific
generated format. Such options might be abused.
7.3. Storage
The Check action needs to store all draft formats so that
successfully validated drafts can later be auto-posted at submitter
request. The action stores all submitted formats of the draft in a
staging area dedicated to the Toolset (R12/a). If, after garbage
collection, the staging area is full (i.e., the total used size has
reached the configured maximum capacity), the submitter and the
Secretariat are notified of a fatal error (R13/a).
7.4. Extraction
The Toolset extracts meta-data from the following stored draft
formats: plain text (R131/a), XML (R132/b), and other (R133/c). If a
meta-data extraction fails, the Toolset records an error (R15/a).
Meta-data extraction is necessary to validate and post the draft.
Extraction from all formats is necessary to validate that all
meta-data matches across all formats (in addition to and before the
Toolset can validate that the contents matches as well).
Section 16 documents a non-obvious implementation schedule related to
the above requirements. When only partial support for format
interpretation is available, only interpreted formats are subject to
extraction and validation requirements. In other words, if the
Toolset does not yet support interpretation of a given format, then
the corresponding information is stored and made available "as is",
regardless of the actual content.
The draft interpreter extracts the following meta-data from each
draft format (R16/a):
identifier: Also known as draft "filename". For example,
"draft-ietf-sieve-vacation-13".
version: A non-negative integer number representing draft version
number (also known as draft revision number). For example, the
number 7 in "draft-ietf-sieve-vacation-07". The number is usually
rendered using two digits, padding with "0" if necessary.
name: The common part of all draft identifiers for all versions of
the same draft. In other words, a draft identifier without the
version component. For example, "draft-ietf-sieve-vacation" in
"draft-ietf-sieve-vacation-07".
WG ID: Working Group identifier. For example, "sieve" in
"draft-ietf-sieve-vacation-07" is a WG ID. The WG ID value is
empty for drafts that are not WG-named drafts.
WG flag: True for WGN drafts and false for all other drafts. For
example, "true" for "draft-ietf-sieve-vacation-13". This flag
only influences the further handling of initial (version 00) draft
submissions, as far as the current document is concerned.
title: A human-friendly draft title. For example, the title of this
document is "Requirements for an IETF Draft Submission Toolset".
authors: A list of all draft authors. Each author’s name and email
address are extracted.
abstract: The draft abstract text.
creation date: The draft version creation date.
expiration date: The draft version expiration date.
size: The number of pages and octets in the primary format of the
draft. The definition of a page depends on the format and may be
imprecise or arbitrary for some formats.
Failure to extract any field results in error (R95/a).
The Toolset requires author email addresses because they are
essential for notifying co-authors that their draft has been posted.
If there are no such notifications, a submitter adding a co-author to
the draft without the co-author’s consent may not be caught for a
while. Such "surprise" co-authorships have happened in the past and
can be quite annoying. However, since the Toolset does not solicit
co-authors’ consent to post a valid draft (and such solicitation
would not go beyond email control verification anyway), it is not
possible to stop a malicious submitter from adding co-authors without
their knowledge.
Like other meta-data items above, draft creation and expiration dates
are extracted from the draft; their values do not depend on the
actual submission time (i.e., the time the Check action starts).
However, the validation procedure (see Section 7.5) may declare any
extracted date invalid after taking into consideration current (i.e.,
submission) time, IETF draft expiration rules, and other factors
external to the draft.
7.5. Validation
Drafts need to be validated to catch broken submissions. Validation
also helps educate or warn authors of problems that may become
show-stoppers when the draft is sent for IETF Last Call and IESG
review. IETF standards have to follow a set of syntax and semantics
requirements (see the "ID-NITS" document at
<http://www.ietf.org/ID-Checklist.html>. Most of those requirements
are not enforced for Internet-Drafts. However, following them may
improve draft quality, reduce the IESG load, and increase the chances
of the draft being approved as an RFC.
When validating a given draft, it is important to distinguish between
absolute requirements and desirable draft properties. Both
categories are checked for, but violations have different effects
depending on the category. The two categories are detailed in the
following subsections.
When a valid draft is being posted and submitter authorization or
co-author notification is performed, validation results should be
included in the email (R81/b) so that the submitter can see meta-data
extraction and validation warnings. Note that these results cannot
include errors since only valid drafts can be posted.
7.5.1. Absolute Requirements
Violating any of these requirements would prevent a draft from being
automatically posted (R17/a). The offending draft would have to be
fixed or submitted for manual posting, with an explanation as to why
the absolute requirements need to be violated (or why the Validator
mis-detected violations). These explanations may speed up the
Secretariat posting decision and may help the Secretariat to improve
the Toolset implementation.
1. All available meta-data entries must match across all submitted
draft formats (R18/a). For example, if the interpreter managed
to extract a draft title from both the plain text and the PDF
format, both titles must match. This requirement prevents
accidental submission of mismatching formats.
2. Version 00 of a Working Group draft has a corresponding Working
Group approval (R20/a). This approval can be relayed before or
after the first draft submission, by a Chair or Secretary of the
WG. See Section 7.5.4 for related requirements.
3. The draft ID must be correct (R22/a), including the draft version
number value and format. Single-digit draft version numbers must
be left-padded with "0". Draft version numbers must start with
zero and increase by one with every new version. To satisfy this
requirement, the Toolset would have to consult the repository of
already posted drafts, including expired ones. If the IETF
infrastructure cannot handle version numbers greater than 99, the
Toolset must reject them (R158/a).
4. An IETF IPR Statement and other boilerplate required for drafts
according to [RFC3978] and [RFC3979] (or successors) must appear
in the draft text (R23/a).
5. The extracted creation date of the draft version must be within 3
days of the actual submission date (R159/a). Hint: Implementers
should be careful to handle creation dates that appear to be in
the past or in the future, due to possible time zone differences.
Making the most forgiving/permissive assumption about the time
zone should suffice.
6. The draft version expiration date obeys IETF draft expiration
rules.
7. No IETF submission blackout period applies. Hint: IETF blackouts
must be enforced based on submission time, not possible draft
creation time.
8. Posting the draft must not result in any DoS attack threshold to
be crossed (R97/a). Specific thresholds are documented in
Section 7.5.3.
9. XML sources (if available) are valid with respect to the XML
format [XML] (R153/c) and XML Document Type Definition (DTD) for
IETF drafts (R154/c). Note that during the first two
implementation stages, the corresponding validation failures
result in warnings and not errors (see Section 7.5.2).
The XML DTD for IETF drafts is documented in [RFC2629] with recent
changes available in [writing-rfcs]. Hint: Bill Fenner’s "RFC 2629
validator" at http://rtg.ietf.org/~fenner/ietf/xml2rfc-valid/ (or its
derivative) may be useful for XML format and DTD validation.
Hint: If the extracted meta-data differs in the submitted draft
formats, the Toolset should use the meta-data from the most "formal"
format when populating the form entries for manual submission. On
the other hand, if most extracted entries come from a less "formal"
format, the Toolset may choose that format instead. For example, XML
source can be considered more "formal" than plain text format. The
Toolset may also offer the submitter an option to specify which
format should be used for populating the form. It is probably a bad
idea to mix-and-match the conflicting entries extracted from multiple
formats. Instead, either one format should be chosen when populating
the form or the form should contain several meta-data sections, one
for each format. The error messages will contain the exact mismatch
information.
Hint: The Toolset should accept dates without the day of the month,
as long as IETF rules do not prohibit them. The Toolset should make
the most forgiving/permissive assumption about the actual day of the
month when validating day-less dates.
7.5.2. Desirable Features
Violating any of the following requirements does not prevent the
submitter from auto-posting the draft (R24/a) but results in a
warning (R160/a). Each warning explains the corresponding violation
and provides advice on how to comply (R161/b). Hint: To ease
maintenance and encourage 3rd party updates, detailed explanations
and/or advice may be available as a resource separate from the
Toolset.
1. All automatically testable nits in the "ID-NITS" document at
<http://www.ietf.org/ID-Checklist.html> (R116/b) and
automatically testable guidelines at
<http://www.ietf.org/ietf/1id-guidelines.txt> (R157/b). The
Toolset should use external tools to check these nits and
guidelines rather than embed checking code (R117/a). Hint:
Henrik Levkowetz’s idnits tool can be used
(http://tools.ietf.org/tools/idnits/) and other tools can be
written or adopted.
2. New draft versions are expected (R21/b). For example, version 00
of an individual draft is always expected, while posting a new
version of a draft already under the IESG review should generate
a warning.
3. If both XML and plain text formats are submitted, the submitted
plain text matches what can be generated based on submitted XML
(R146/b).
4. The previous version, if any, was posted at least 24 hours ago
(R96/b). This warning may prevent some human errors, especially
when multiple authors may post the same draft.
5. XML sources (if available) are valid with respect to the XML
format (R155/b) and XML DTD for IETF drafts (R156/b). These
requirements become absolute after the second implementation
phase. See Section 7.5.1 for related information.
When comparing generated and submitted plain text formats to satisfy
R146, a standard word-based diff is sufficient for initial Toolset
implementations (R147/b). However, a custom fuzzy matching function
can be developed (R148/c) to minimize false warnings due to, for
example, draft text formatting differences. When differences are
detected, a complete diff may be provided on a separate page
(R149/c), in addition to the warning.
Hint: When comparing generated and submitted plain text formats, the
Toolset may try several recent xml2rfc versions for plain text
generation, to eliminate warnings due to differences among xml2rfc
versions.
7.5.3. DoS Thresholds
The following table documents DoS attack thresholds for various draft
categories. Daily limits correspond to all drafts (and all draft
formats) within the category. Other thresholds may be introduced and
these initial thresholds may be adjusted as necessary. The
thresholds are likely to become more smart/dynamic with experience.
DoS attack thresholds:
+---------------------------------+--------------+-----------+
| category | versions/day | MB/day |
+---------------------------------+--------------+-----------+
| drafts with the same draft name | 3 | 5 |
| drafts with the same submitter | 10 | 15 |
| WGN drafts with the same WG ID | 30 | 45 |
| all drafts | 400 | 200 |
+---------------------------------+--------------+-----------+
The thresholds are meant to limit destructive effects of DoS attacks
(e.g., full disks cause other tasks to fail), allow for capacity
planning (e.g., how much storage space the Toolset needs), and limit
annoying side effects of "too many" drafts being posted (e.g., when a
person receives posting notifications about a given draft or a given
working group). The Toolset should warn the Secretariat if total
submissions are approaching any threshold (R134/b). Hint: Bandwidth
available for submissions may need to be throttled (on a network
subnet basis?) to make reaching the daily size quota (with malicious
intent) difficult.
7.5.4. WG Approval
For version 00 of a WGN draft, the Toolset checks for an existing WG
approval (R125/a). If (a) no approval exists, and (b) the Toolset
does not support the "waiting for WG approval" feature, the Toolset
records an error (R135/a).
If (a) no approval exists, (b) the Toolset supports the "waiting for
WG approval" feature, and (c) the draft cannot be posted even if WG
approval is received, then the Toolset records a warning that a WG
approval would be required once all errors preventing draft from
posting are fixed (R137/b).
If (a) no approval exists, (b) the Toolset supports the "waiting for
WG approval" feature, and (c) the draft can be posted if WG approval
is received, then the Toolset explains the situation to the submitter
and asks whether an explicit approval from the WG should be solicited
or expected (R126/b). If the approval should be solicited, it is
solicited by the software or the submitter. If appropriate, the
Toolset puts the submission into a "waiting for WG approval" state
until the expected approval is available (R127/b). Otherwise, the
Toolset records a "no WG approval is expected" error (R138/b).
The details of manual or automated solicitation for WG approval is
outside the scope of this document. Hint: Initially, the submitter
will be responsible for soliciting a WG Chair approval, but this
process should eventually be automated.
Details of the approval recording and access interfaces as well as
the mechanism to resume the submission upon approval are out of this
document’s scope.
8. Check Page
The Check page, created by the Check action, displays extracted draft
meta-data and validation results (R25/a). The purpose of the page is
to allow the submitter to verify whether the stored draft and
automatically extracted meta-data match the submitter’s intent and to
be informed of validation problems.
Meta-data items specified in Section 7.4 that failed validation
checks must be marked specially (rather than silently omitted or
ignored) (R26/b). Hint: rendering those items in red, with links to
corresponding validation errors or warnings, may force authors to pay
attention.
Validation messages include both errors and warnings. Each
validation message refers to normative document(s) containing the
corresponding validation rules (R27/b).
The Check page allows the submitter to enter external meta-data
(Section 8.1) (R28/a). If validation was successful, an
"automatically post the draft now" button is provided (R29/a).
Regardless of validation results, "adjust and post manually" and
"cancel" buttons are provided (R30/a).
The Check page provides a preview of the draft plain text format
(R31/a), with a link to see how the entire draft (with all its
formats) would look if posted (R82/b). Hint: the Check page preview
should be sufficiently long to let authors detect obvious draft
mismatch or misinterpretation errors but short enough to avoid
dominating the page. Displaying the first line of the draft through
the last line of the abstract may be sufficient.
For draft updates, the Check page reports the time and the submitter
of the last update (R83/b). This information is especially useful
when multiple authors are working on the same draft. The page also
provides a link to generate a diff against the last posted version
(R84/c).
8.1. External Meta-Data
The Check page solicits the following meta-data from the submitter.
This information must be supplied by submitter because it cannot be
extracted from the draft:
Submitter email address (R32/a). When submitter is not an
expected submitter (see Section 3), automated posting is not
possible and the draft has to go through the Secretariat (R98).
Hint: A set of checkboxes next to extracted author names along
with a "none of the above" checkbox with an input field would
suffice.
A list of drafts replaced by this draft (R33/c). This is useful
to make replaced drafts invisible. This document does not specify
any actions necessary to actually replace an existing draft
because existing draft manipulation is out of scope, and because
security concerns and other complications of such actions would be
better addressed by a separate specification.
Primary email address for discussion of this draft (R71/b). Hint:
The Toolset can suggest the WG mailing list address for WGN
drafts, (submitting) author address for individual drafts, or even
the first email address in draft text. Offering a few likely
addresses instead of relying exclusively on user input would also
reduce the number of typos.
Except for the submitter email address, external meta-data is
optional (R109/a).
If a given submitter email address belongs to an expected submitter
(i.e., belongs to one of the categories below), the Toolset performs
submitter authentication during a Post Now action (R19/a).
Otherwise, an error is reported (R118/a).
The following possible expected submitters are identified by the
Toolset, without any Secretariat intervention:
For version 00 of a draft, any submitter (R119/a).
For version N+1 of a draft, an author of version N of the same
draft (R120/a). This requirement only needs to be satisfied for
drafts for which Nth version was posted using the Toolset; other
drafts may not have the meta-information available that is
required to reliably get a list of authors.
For a WGN draft, a Chair of the corresponding WG (R121/b).
For any draft, an IESG member (R122/c).
9. Post Now Action
The Post Now action checks that the draft has been successfully
validated (R34/a), validates external meta-data (including submitter
email address) (R35/a), and posts the draft (R36/a). The submitter
is notified of the action progress and the final result (R37/a).
The external meta-data contains the submitter’s email address. As a
part of the validation procedure, the Post Now action authorizes the
submitter. The initial action implementation checks that the
submitter has access to email sent to that address (R38/a).
Eventually, the Toolset should accept client certificates signed by
IETF, PGP-signed email, and/or other forms of client-side
authentication to eliminate the weak and annoying email access check
(R110/c). If submitter authentication fails, the submission
eventually and silently times out (R39/a).
The Toolset provides both web (R99/a) and email (R139/b) interfaces
for confirming email access. Hint: To check submitter’s access to
email, the tool can email a hard-to-guess cookie or token to the
submitter’s address. To continue with the submission, the submitter
is requested to paste the cookie at the specified URL, go to the
token-holding URL, or respond to the email.
Immediately after sending an email to the submitter, the Post Now
action generates an intermediate Receipt page that explains Toolset
expectations and provides the submitter with the submission ID
(R100/a). That number allows the Secretariat to troubleshoot stuck
submissions (R101/a) and can also be used for checking submission
status without Secretariat involvement (R140/b).
Immediately after posting the draft, the Toolset notifies all authors
(with known email addresses) of the posting (R102/a). The
notification email contains the information available on the
"successful posting" Receipt page described below (R103/a).
If draft posting is successful, the submission state is marked as
available for deletion (R105/a) so that the garbage collection
routine eventually deletes it.
9.1. Receipt Page
A successful Post Now action reports at least the following
information on the final Receipt page (R104/a):
o the draft ID and a link to the draft status page.
o the draft title, authors, and abstract.
o the submission ID.
o a link to the draft submission status page (when status queries
are supported, see R140).
o the submitter’s name and email address.
The primary purpose of the Receipt page is to inform all draft
authors that (supposedly) their draft has been posted. The secondary
purpose is to let authors create a permanent record of the event and
troubleshoot postings. The same information should be sent to other
parties interested in the draft (e.g., to the WG mailing list), but
3rd-party notification specifics are out of this document’s scope.
10. Adjust Action
The Adjust action generates the Adjust page (R40/a), populating it
with available extracted meta-data and external meta-data, as well as
validation results and a preview. Some information may be missing,
depending on draft interpretation and the success of preview
generation.
11. Adjust Page
The Adjust page includes the same information as the Check page, but
allows the submitter to adjust all extracted draft meta-data (and,
naturally, external meta-data) at will (R41/a). Such adjustment is
necessary when automated extraction failed to extract correct
information. To avoid any mismatch between draft and its meta-data,
adjusted drafts cannot be automatically posted and require manual
validation by the Secretariat (R42/a). Secretariat staff can post