specify an entire system, instead of simple component protocols,
leading to feature bloat and applicability only to a narrow range of
applications (see also Section 2.4). On the other hand, others
believe that the IESG has approved simple component protocols without
an adequate understanding of the systems and contexts in which the
protocols might be used. These problems appear to be two additional
aspects of the general problem that the IETF has with handling large
and/or complex systems.
2.4. Three Stage Standards Hierarchy not properly Utilized
The current hierarchy of Proposed, Draft, and Full Standard maturity
levels for specifications is no longer being used in the way that was
envisioned when the stratification was originally proposed. In
practice, the IETF currently has a one-step standards process that
subverts the IETF’s preference for demonstrating effectiveness
through running code in multiple interoperable implementations. This
compresses the process that previously allowed specifications to
mature as experience was gained with actual implementations:
o Relatively few specifications are now progressed beyond Proposed
Standard (PS) to Draft Standard (DS) level, and even fewer to Full
Standard (FS).
o It is widely perceived that the IESG has ’raised the (quality)
bar’ that standards have to pass to be accorded a PS status.
Protocol developers may be required to specify a complete system
rather than an interface in order for their specification to be
approved as a PS (see also Section 2.3).
o In spite of the apparently higher quality hurdle, implementation
or deployment experience is still not required, so the IETF’s
guiding principle of ’rough consensus and running code’ has less
of a chance to be effective.
o There appears to be a vicious circle in operation where vendors
tend to deploy protocols that have reached PS as if they were
ready for full production, rather than accepting that standards at
the PS level are still under development and could be expected to
be altered after feature, performance, and interoperability tests
in limited pilot installations, as was originally intended. The
enthusiasm of vendors to achieve a rapid time to market seems to
have encouraged the IETF in general and the IESG in particular to
attempt to ensure that specifications at PS are ready for prime
time, and that subsequent modifications will be minimal as it
progresses to DS and FS, assuming effort can be found to create
the necessary applicability and interoperability reports that are
needed.
o The three stage hierarchy is, accordingly, seen to be excessive.
o There is no formal bug reporting or tracking system in place for
IETF specifications.
o The periodic review of protocols at PS and DS levels specified in
[1] are not being carried out, allowing protocols to persist in
these lower maturity levels for extended periods of time, whereas
the process would normally expect them to progress or be relegated
to Historic status.
o No individual or body is given the task of ’maintaining’ a
specification after the original WG has closed down.
Specifications are generally only updated when a need for a new
version is perceived. No attempt is normally made to correct bugs
in the specification (whether they affect operation or not) and
the specification is not updated to reflect parts of the
specification that have fallen into disuse or were, in fact, never
implemented. This is, in part, because the current procedures
would require a standard to revert to the PS maturity level, even
when specification maintenance is carried out. This occurs even
if the changes can be demonstrated to have no or minimal effect on
an existing protocol at the DS or FS level.
2.5. The IETF’s Workload Exceeds the Number of Fully Engaged
Participants
There are a number of respects in which IETF participants and
contributors appear to have become less fully engaged with the IETF
processes, for example:
o Although there may be large attendance at many WG meetings, in
many cases, 5% or less of the participants have read the drafts
under discussion or that have a bearing on the decisions to be
made.
o Commitments to write, edit, or review a document are not carried
out in a timely fashion.
o Little or no response is seen when a request for ’last-call’
review is issued, either at WG or IETF level.
This might be because contributors have less time available in their
work schedule during the downturn of the Internet business climate
between 2001 and 2003. Yet, this is not the whole story, as there
were signs of this effect back at the height of the Internet’s boom
in 2000.
This problem exacerbates the problems the IETF has had with timely
delivery and may weaken the authority of IETF specifications if
decisions are seen to be taken by badly informed participants and
without widespread review.
2.5.1. Lack of Formal Recognition
Beyond RFC Authorship, WG Chair positions, Directorate positions, or
IESG and Internet Architecture Board (IAB) membership, the IETF does
not offer formal recognition of contributions to the IETF. This
potentially acts as a disincentive to continued engagement and can
lead to useful and effective participants leaving because they cannot
obtain any recognition (the only currency the IETF has to pay
participants), which they use to fuel their own enthusiasm and help
justify their continued attendance at IETF meetings to cost
constrained employers. Note: Using Leadership positions as rewards
for good work would probably be damaging to the IETF. This paragraph
is meant to indicate the need for other types of rewards.
2.6. The IETF Management Structure is not Matched to the Current Size
and Complexity of the IETF
The management and technical review processes currently in place were
adequate for the older, smaller IETF, but are apparently not scalable
to the current size of the organization. The form of the
organization has not been significantly modified since 1992, since
when the organization has undergone considerable further growth. The
scope of IETF activities has also been extended as the Internet has
become more complex.
2.6.1. Span of Authority
Overt authority in the IETF is concentrated in the small number of
people sitting on the IESG at that time. Existing IETF processes
work to funnel tasks on to this small number of people (primarily the
Area Directors (ADs) in the IESG). This concentration slows process
and puts a very large load of responsibility on the shoulders of
these people who are required to act as the senior management for
Working Group (WG) chairs, as well as acting as quality backstops for
the large number of documents issued by the IETF. The situation has
not been helped by the widening of the scope of the IETF, which has
resulted in somewhat more WGs and a need for a very broad spectrum of
knowledge within the set of ADs.
2.6.2. Workload of the IESG
With the current structure of the IETF and IESG, the workload imposed
on each of the ADs is almost certainly well beyond the capabilities
of a single person.
The current job description for an AD encompasses at least the
following tasks:
o Interacting with WGs
o Understanding network and computer technology in general, and
their own area in detail
o Cross-pollinating between groups
o Coordinating with other areas
o Potentially, managing their Area Directorate team
o Effectively providing technical management, people-management, and
project supervision for their WGs
o Reading (or at least skimming) every formal document which the
IETF produces, and having an opinion on all of them, as well as
all the Internet Drafts produced by the WGs in the area, and
understanding the interactions between all these specifications.
Given the number of WGs which are now active, the increasing
complexity of both the work being undertaken and the technology in
general, together with the volume of documents being produced, makes
it clear that only superhumans can be expected to do this job well.
To make matters worse, these tasks are, in theory, a ’part time’
occupation. ADs will normally have a conventional job, with the IETF
activities as just one part of their job specification. This view
has been reinforced by recent resignations from the IESG, citing the
size of the workload as a primary factor. The IETF also has no
mechanisms to nominate a temporary replacement or an assistant should
an AD be incapacitated wholly or partially for a period.
The malign effects of this overload include:
o Wear on the IESG: The IESG members are overworked which is bad
for their health, humor, and home life, and may also result in
conflicts with their employers if the IETF work impacts the IESG
member’s performance of their ’day job’.
o Unhappiness in the IETF: IETF stakeholders perceive that IESG
members are responding slowly, are not fully up-to-date with their
technology, fail to pro-actively manage problems in their WGs, and
are unable to keep communication channels with other groups open.
o Recruiting shrinkage: The number of people who can imagine taking
on an IESG post is steadily decreasing. It is largely limited to
people who work for large companies who can afford to send IESG
members to the IETF for the duration of their appointments. In
the current business climate, fewer companies are able to justify
the preemption of an important engineering and business resource
for a significant period of time, and are more likely to put
forward ’standards professionals’ than their best engineers.
2.6.3. Procedural Blockages
The current procedural rules combined with the management and quality
roles of the ADs can lead to situations where WGs or document authors
believe that one or two ADs are deliberately blocking the progress of
a WG document without good reason or public justification. Appeal
processes in these circumstances are limited and the only sanction
that could be applied to the relevant ADs is recall, which has almost
always been seen to be out of scale with the apparent offense and
hence almost never invoked. This perception of invulnerability has
led to a view that the IESG in general and the ADs in particular are
insufficiently accountable for their actions to their WGs and the
IETF at large, although the recent introduction of the Internet Draft
Tracker tool makes it easier to determine if and how a document has
become blocked, and hence to take appropriate steps to release it.
2.6.4. Consequences of Low Throughput in IESG
If documents are inappropriately (or even accidentally) delayed or
blocked as a result of IESG (in)action, this can cause much
frustration inside the organization, a perception of disunity seen
from outside the organization, and delay of standards, possibly to
the point where they are too late to match market requirements: work
which has been properly authorized as being within the scope of the
IETF and properly quality checked during development, should almost
never come up against such a blockage.
Delay in authorizing a BOF or chartering a new WG can delay the start
of the process with similar effects.
It also appears that IESG delays are sometimes used to excuse what is
actually slow work in WGs.
2.6.5. Avoidance of Procedural Ossification
The systems and processes used by the IETF are generally designed
around having firm general principles and considerable IESG
discretion within those principles. It appears that the IETF is
showing a disturbing tendency to turn IESG ’rules of convenience’
into rigid strictures that cannot be violated or deviated from.
Up to now, IETF discussions of procedures have been driven by a model
in which the procedural BCPs construct a framework for doing work,
but the details of the framework are left for the IESG to fill in.
When issues or crises have arisen, the IETF has generally avoided
making specific procedural changes to compensate, instead realizing
that we could not anticipate all cases and that ’fighting the last
war’ is not a good way to proceed.
This can only continue to work if the participants continue to trust
the IESG to act fairly in filling in the details and making
appropriate exceptions, without a great deal of debate, when it is
clearly desirable. At present, the IETF appears to have lost sight
of this flexibility, and is entangling itself in procedures that
evolve from organizational conveniences into encumbrances.
2.6.6. Concentration of Influence in Too Few Hands
Until the last couple of years, successive IETF Nominating Committees
have chosen to give heavy weighting to continuity of IESG and IAB
membership. Thus, the IETF appeared to have created an affinity
group system which tended to re-select the same leaders from a
limited pool of people who had proved competent and committed in the
past.
Members of this affinity group tend to talk more freely to each other
and former members of the affinity group - this may be because the
affinity group has also come to share a cultural outlook which
matches the dominant cultural ethos of the IETF (North American,
English speaking). Newcomers to the organization and others outside
the affinity group are reluctant to challenge the apparent authority
of the extended affinity group during debates and consequently
influence remains concentrated in a relatively small group of people.
This reluctance may also be exacerbated if participants come from a
different cultural background than the dominant one. Such
participants also tend to find it more difficult to follow the rapid
and colloquial speaking style of native English speakers, and may
consequently be effectively excluded from the discussion, even if
maximum assistance is available by such means as real time Jabber
logs and extensive text on presentation slides. Even on mailing
lists, people from other cultures may be reluctant to be as
forthright as is often the case in discussions between North
Americans; also, a person whose first language is not English may be
daunted by the volume of mail that can occur on some mailing lists
and the use of colloquialisms or euphemisms may cause
misunderstandings if correspondents are not aware of the problem.
A further instance of the problems of concentration of influence
potentially occurs when, from time to time, ADs have acted as WG
chairs: conflict of interest might well arise in discussions between
the IESG and any WG with an AD as its chair. Whilst care is usually
taken to have a newly selected AD vacate any WG chair positions which
might be held in his or her own area, the conflict can arise on the
occasions when an AD has been used as the chair of a WG because it is
clearly the right (or only possible) solution for the WG from an
engineering and know-how position. Furthermore, given the known
problem of workload for IESG members, there must be doubts as to
whether an AD can or ought to be taking on this extra load.
2.6.7. Excessive Reliance on Personal Relationships
The IETF is an intensely personal and individualistic organization.
Its fundamental structure is based on individuals as actors, rather
than countries, organizations, or companies as in most other SDOs.
This is also reflected in how the IETF gets its work done: the NOMCOM
process, the WG Chair selection processes, and the activities of WGs
are all reliant on personal knowledge of the capabilities of other
individuals and an understanding built on experience of what they can
be expected to deliver, given that there are almost no sanctions that
can be applied beyond not asking them to do a similar task again.
The relationship works best when it is two way - the person being
asked to perform a task needs to be able to rely on the behavior of
the person doing the asking.
In essence, the IETF is built on a particular kind of one-to-one
personal trust relationship. This is a very powerful model but it
does not scale well because this trust is not transitive. Just
because you trust one person, it does not mean that you trust (i.e.,
know the capabilities of and can rely on) all the people that person
trusts in turn.
The disruption caused when one set of relationships has to be
replaced by another is clearest when an AD is replaced. The IETF
does not keep personnel records or written plans, and formal process
documentation is very sparse, so that incoming ADs have little
information on which to base new relationships with WG chairs or
Directorate members not already known to them.
A new AD has to build or bring along his or her set of trusted
individuals. The AD will tend to prefer individuals from this set as
WG chairs, unless there is a suitable outsider who was part of the
team that brought the WG idea to the IETF. This tends to limit the
AD’s field of choice, particularly when asking for a ’stabilizing’,
’advising’, or ’process’ chair to work with an enthusiastic newcomer
in a difficult area. A breakdown of an established relationship
(such as between an AD and a WG chair) can be very damaging to the
work of the IETF, and it may not be immediately obvious to outsiders.
Another consequence of the reliance on personal relationships is that
the IETF has very little institutional ’memory’ outside the memories
of the people in the process at a given time. This makes it more
likely that failures will be repeated and makes process improvement
more difficult (see Section 2.2).
2.6.8. Difficulty making Technical and Process Appeals
When an individual thinks that the process has produced a result that
is harmful to the Internet or thinks that IETF processes have not
been adhered to, there is no mechanism to aid that individual in
seeking to change that result.
2.7. Working Group Dynamics can make Issue Closure Difficult
The IETF appears to be poor at making timely and reasonable decisions
that can be guaranteed to be adhered to during the remainder of a
process or until shown to be incorrect.
The problems documented in this section are probably consequences of
the non-hierarchical organization of the IETF and the volunteer
status of most participants. The enforcement measures available in a
more conventional hierarchical corporate environment are mostly not
available here, and it is unlikely that application of some well-
known procedure or practice will fix these problems.
Participants are frequently allowed to re-open previously closed
issues just to replay parts of the previous discussion without
introducing new material. This may be either because the decision
has not been clearly documented, or it may be a maneuver to try to
get a decision changed because the participant did not concur with
the consensus originally. In either case, revisiting decisions stops
the process from moving forward, and in the worst cases, can
completely derail a working group. On the other hand, the decision
making process must allow discussions to be re-opened if significant
new information comes to light or additional experience is gained
which appears to justify alternative conclusions for a closed issue.
One cause that can lead to legitimate attempts to re-open an
apparently closed issue is the occurrence of ’consensus by
exhaustion’. The consensus process can be subverted by off-topic or
overly dogmatic mail storms which can lead to the exclusion of
knowledgeable participants who are unable to devote the time needed
to counter the mail storm. The consequence may be an
unrepresentative and unsatisfactory consensus which will tend to be
re-opened, often leading to repeat discussions. Mailing lists, which
are at the heart of the IETF WG process, are becoming increasingly
ineffective at resolving issues and achieving consensus because of
this phenomenon.
A single vocal individual or small group can be a particular
challenge to WG progress and the authority of the chair. The IETF
does not have a strategy for dealing effectively with an individual
who is inhibiting progress, whilst ensuring that an individual who
has a genuine reason for revisiting a decision is allowed to get his
or her point across.
2.8. IETF Participants and Leaders are Inadequately Prepared for
their Roles
Participants and leaders at all levels in the IETF need to be taught
the principles of the organization (Mission and Architecture(s)) and
trained in carrying out the processes, which they have to use in
developing specifications, etc.
Part of the reason for the lack of training in the principles of the
organization is that there is not currently an explicit formulation
of these principles that is generally agreed upon by all
stakeholders. Section 2.1 identifies that this shortage is a major
problem.
The IETF currently has voluntary and inconsistent processes for
educating its participants, which may be why significant numbers of
participants seem to fail to conform to the proper principles when
working in the IETF context.
The people in authority have generally been steeped in the principles
of the IETF (as they see them) and first-time non-compliance by newer
participants is sometimes treated as an opportunity for abuse rather
than recognition of a training failure.
The IETF culture of openness also tends to tolerate participants who,
whilst understanding the principles of the IETF, disagree with them
and actively ignore them. This can be confusing for newer
participants, but they need to be made aware that the IETF does not
exclude such people. The IETF does not currently have a strategy for
dealing with the conflicts that can result from participants who
disagree with the principles of the organization.
Lack of training, compounded with the perceived concentration of
influence in the affinity group documented in Section 2.6.6, can lead
to newcomers being ignored during discussions, consequently being
ineffective, either in their own eyes or their employers. This may
result in their departure from the IETF.
In addition, some participants are not aware of the problems that
participants, who do not have English as their first language, may
have with rapid speaking and the use of colloquialisms in both spoken
and written communication. They are also not always aware of the
possible cultural nuances that may make full participation more
difficult for those who do not share the same outlook.
3. Security Considerations
This document does not, of itself, have security implications, but it
may have identified problems which raise security considerations for
other work. Any such implications should be considered in the
companion document which will be produced setting out how the IETF
should set about solving the identified problems.
4. Acknowledgements
Apart from the contributions of all those who provided input on the
problem statement mailing list, the final reduction of the problems
was especially assisted by the following people:
Rob Austein <sra@hactrn.net>
Marc Blanchet <Marc.Blanchet@hexago.com>
Dave Crocker <dcrocker@brandenburg.com>
Spencer Dawkins <spencer@mcsr-labs.org>
Avri Doria <avri@psg.com> (WG co-chair)
Jeanette Hoffmann <jeanette@wz-berlin.de>
Melinda Shore <mshore@cisco.com> (WG co-chair)
Margaret Wasserman <margaret@thingmagic.com>
Special thanks are due to Margaret Wasserman for extensive reviewing
of and contributions to the wording of Section 2.
The early part of the reduction of the problem statement mailing list
input was done by Harald Alvestrand and the latter part by Elwyn
Davies and the team acknowledged above. In total, there were
approximately 750 extensive and thoughtful contributions (some making
several points). The thread was started by a call for volunteers in