information into the DNS label would be impractical and unwise, and
the IAB, based in part on the comments in the ad hoc committee, saw
no reason to review that decision.
The Unicode Consortium has sometimes used the likelihood of a
combination of characters actually appearing in a natural language as
a criterion for the safety of a possible change. However, as
discussed above, DNS names are often fabrications -- abbreviations,
strings deliberately formed to be unusual, members of a series
sequenced by numbers or other characters, and so on. Consequently, a
criterion that considers a change to be safe if it would not be
visible in properly-constructed running text is not helpful for DNS
purposes: a change that would be safe under that criterion could
still be quite problematic for the DNS.
This sensitivity to changes has made it quite difficult to migrate
IDNA from one version of Unicode to the next if any changes are made
that are not strictly additive. A change in a code point assignment
or definition may be extremely disruptive if a DNS label has been
defined using the earlier form and any of its previous components has
been moved from one table position or normalization rule to another.
Unicode normalization tables, tables of scripts or languages and
characters that belong to them, and even tables of confusable
characters as an adjunct to security recommendations may be very
helpful in designing registry restrictions on registrations and
applications provisions for avoiding or identifying suspicious names.
Ironically, they also extend the sensitivity of IDNA and its
implementations to all forms of change between one version of Unicode
and the next. Consequently, they make Unicode version migration more
difficult.
An example of the type of change that appears to be just a small
correction from one perspective but may be problematic from another
was the correction to the normalization definition in 2004
[Unicode-PR29]. Community input suggested that the change would
cause problems for Stringprep, but the Unicode Technical Committee
decided, on balance, that the change was worthwhile. Because of
difficulties with consistency, some deployed implementations have
decided to adopt the change and others have not, leading to subtle
incompatibilities.
This situation leads to a dilemma. On the one hand, it is completely
unacceptable to freeze IDNA at a Unicode version level that excludes
more recently-defined characters and scripts that are important to
those who use them. On the other hand, it is equally unacceptable to
migrate from one version of Unicode to the next if such migration
might invalidate an existing registered DNS name or some of its
registered properties or might make the string or representation of
that name ambiguous. If IDNA is to be modified to accommodate new
versions of Unicode, the IETF will need to work with the Unicode
Consortium and other bodies to find an appropriate balance in this
area, but progress will be possible only if all relevant parties are
able to fairly consider and discuss possible decisions that may be
very difficult and unpalatable.
It would also prove useful if, during the course of that dialog, the
need for Unicode Consortium concern with security issues in
applications of the Unicode character set could be clarified. It
would be unfortunate from almost every perspective considered here,
if such matters slowed the inclusion of as yet unencoded scripts.
3.2. Version Changes and Normalization Issues
3.2.1. Unnormalized Combining Sequences
One of the advantages of the Unicode model of combining characters,
as with previous systems that use character overstriking to
accomplish similar purposes, is that it is possible to use sequences
of code points to generate characters that are not explicitly
provided for in the character set. However, unless sequences that
are not explicitly provided for are prohibited by some mechanism
(such as the normalization tables), such combining sequences can
permit two related dangers.
o The first is another risk of character confusion, especially if
the relationship of the combining character with characters it
combines with are not precisely defined or unexpected combinations
of combining characters are used. That issue is discussed in more
detail, with an example, in Section 2.2.3.
o These same issues also inherently impact the stability of the
normalization tables. Suppose that, somewhere in the world, there
is a character that looks like a Roman-derived lowercase "i", but
with three (not one or two) dots above it. And suppose that the
users of that character agree to represent it by combining a
traditional "i" (U+0069) with a combining diaeresis (U+0308). So
far, no problem. But, later, a broader need for this character is
discovered and it is coded into Unicode either as a single
precomposed character or, more likely under existing rules, by
introducing a three-dot-above combining character. In either
case, that version of Unicode should include a rule in NFKC that
maps the "i"-plus-diaeresis sequence into the new, approved, one.
If one does not do so, then there is arguably a normalization that
should occur that does not. If one does so, then strings that
were valid and normalized (although unanticipated) under the
previous versions of Unicode become unnormalized under the new
version. That, in turn, would impact IDNA comparisons because,
effectively, it would introduce a change in the matching rules.
It would be useful to consider rules that would avoid or minimize
these problems with the understanding that, for reasons given
elsewhere, simply minimizing it may not be good enough for IDNA. One
partial solution might be to ban any combination of a base character
and a combining character that does not appear in a hypothetical
"anticipated combinations" table from being used in a domain name
label. The next subsection discusses a more radical, if impractical,
view of the problem and its solutions.
3.2.2. Combining Characters and Character Components
For several reasons, including those discussed above, one thing that
increases IDNA complexity and the need for normalization is that
combining characters are permitted. Without them, complexity might
be reduced enough to permit easier transitions to new versions. The
community should consider the impact of entirely prohibiting
combining characters from IDNs. While it is almost certainly
unfeasible to introduce this change into Unicode as it is now defined
and doing so would be extremely disruptive even if it were feasible,
the thought experiment can be helpful in understanding both the
issues and the implications of the paths not taken. For example, one
consequence of this, of course, is that each new language or script,
and several existing ones, would require that all of its characters
have Unicode assignments to specific, precomposed, code points.
Note that this is not currently permitted within Unicode for Latin
scripts. For non-Latin scripts, some such code points have been
defined. The decisions that govern the assignment of such code
points are managed entirely within the Unicode Consortium. Were the
IETF to choose to reduce IDNA complexity by excluding combining
characters, no doubt there would be additional input to the Unicode
Consortium from users and proponents of scripts that precomposed
characters be required. The IAB and the IETF should examine whether
it is appropriate to press the Unicode Consortium to revise these
policies or otherwise to recommend actions that would reduce the need
for normalization and the related complexities. However, we have
been told that the Technical Committee does not believe it is
reasonable or feasible to add all possible precomposed characters to
Unicode. If Unicode cannot be modified to contain the precomposed
characters necessary to support existing languages and scripts, much
less new ones, this option for IDN restrictions will not be feasible.
3.2.3. When does normalization occur?
In many Unicode applications, the preferred solution is to pick a
style of normalization and require that all text that is stored or
transmitted be normalized to that form. (This is the approach taken
in ongoing work in the IETF on a standard Unicode text form
[net-utf8]). IDNA does not impose this requirement. Text is
normalized and case-reduced at registration time, and only the
normalized version is placed in the DNS. However, there is no
requirement that applications show only the native (and lower-case
where appropriate) characters associated with the normalized form in
discussions or references such as URLs. If conventions used for
all-ASCII DNS labels are to be extended to internationalized forms,
such a requirement would be unreasonable, since it would prohibit the
use of mixed-case references for clarity or market identification.
It might even be culturally inappropriate. However, without that
restriction, the comparison that will ultimately be made in the DNS
will be between strings normalized at different times and under
different versions of Unicode. The assertion that a string in
normalized form under one version of Unicode will still be in
normalized form under all future versions is not sufficient.
Normalization at different times also requires that a given source
string always normalizes to the same target string, regardless of the
version under which it is normalized. That criterion is much more
difficult to fulfill. The discussion above suggests that it may even
be impossible.
Ignoring these issues with combining characters entirely, as IDNA
effectively does today, may leave us "stuck" at Unicode 3.2, leading
either to incompatibility differences in applications that otherwise
use a modern version of Unicode (while IDN remains at Unicode 3.2) or
to painful transitions to new versions. If decisions are made
quickly, it may still be possible to make a one-time version upgrade
to Version 4.1 or Version 5 of Unicode. However, unless we can
impose sufficient global restrictions to permit smooth transitions,
upgrading to versions beyond that one are likely to be painful (e.g.,
potentially requiring changing strings already in the DNS or even a
new Punycode prefix) or impossible.
4. Framework for Next Steps in IDN Development
4.1. Issues within the Scope of the IETF
4.1.1. Review of IDNA
The IETF should consider reviewing RFCs 3454, 3490, 3491, and/or
3492, and update, replace, or supplement them to meet the criteria of
this paragraph (one or more of them may prove impractical after
further study). Any new versions or additional specifications should
be adapted to the version of Unicode that is current when they are
created. Ideally, they should specify a path for adapting to future
versions of Unicode (some suggestions below may facilitate this).
The IETF should also consider whether there are significant
advantages to mapping some groups of characters, such as code points
assigned to font variations, into others or whether clarity and
comprehensibility for the user would be better served by simply
prohibiting those characters. More generally, it appears that it
would be worthwhile for the IETF to review whether the Unicode
normalization rules now invoked by the Stringprep profile in Nameprep
are optimal for the DNS or whether more restrictive rules, or an even
more restrictive set of permitted character combinations, would
provide better support for DNS internationalization.
The IAB has concluded that there is a consensus within the broader
community that lists of code points should be specified by the use of
an inclusion-based mechanism (i.e., identifying the characters that
are permitted), rather than by excluding a small number of characters
from the total Unicode set as Stringprep and Nameprep do today. That
conclusion should be reviewed by the IETF community and action taken
as appropriate.
We suggest that the individuals doing the review of the code points
should work as a specialized design team. To the extent possible,
that work should be done jointly by people with experience from the
IETF and deep knowledge of the constraints of the DNS and application
design, participants from the Unicode Consortium, and other people
necessary to be able to reach a generally-accepted result. Because
any work along these lines would be modifications and updates to
standards-track documents, final review and approval of any proposals
would necessarily follow normal IETF processes.
It is worth noting that sufficiently extreme changes to IDNA would
require a new Punycode prefix, probably with long-term support for
both the old prefix and the new one in both registration arrangements
and applications. An alternative, which is almost certainly
impractical, would be some sort of "flag day", i.e., a date on which
the old rules are simultaneously abandoned by everyone and the new
ones adopted. However, preliminary analysis indicates that few, if
any, of the changes recommended for consideration elsewhere in this
document would require this type of version change. For example,
suppose additional restrictions, such as those implied above, are
imposed on what can be registered. Those restrictions might require
policy decisions about how labels are to be disposed of if they
conformed to the earlier rules but not to the new ones. But they
would not inherently require changes in the protocol or prefix.
4.1.2. Non-DNS and Above-DNS Internationalization Approaches
The IETF should once again examine the extent to which it is
appropriate to try to solve internationalization problems via the DNS
and what place the many varieties of so-called "keyword systems" or
other Internet navigational techniques might have. Those techniques
can be designed to impose fewer constraints, or at least different
constraints, than IDNA and the DNS. As discussed elsewhere in this
document, IDNA cannot support information about scripts, languages,
or Unicode versions on lookup. As a consequence of the nature of DNS
lookups, characters and labels either match or do not match; a near-
match is simply not a possible concept in the DNS. By contrast,
observation of near-matching is common in human communication and in
matching operations performed by people, especially when they have a
particular script or language context in mind. The DNS is further
constrained by a fairly rigid internal aliasing system (via CNAME and
DNAME resource records), while some applications of international
naming may require more flexibility. Finally, the rigid hierarchy of
the DNS --and the tendency in practice for it to become flat at
levels nearest the root-- and the need for names to be unique are
more suitable for some purposes than others and may not be a good
match for some purposes for which people wish to use IDNs. Each of
these constraints can be relaxed or changed by one or more systems
that would provide alternatives to direct use of the DNS by users.
Some of the issues involved are discussed further in Section 5.3 and
various ideas have been discussed in detail in the IETF or IRTF.
Many of those ideas have even been described in Internet Drafts or
other documents. As experience with IDNs and with expectations for
them accumulates, it will probably become appropriate for the IETF or
IRTF to revisit the underlying questions and possibilities.
4.1.3. Security Issues, Certificates, etc.
Some characters look like others, often as the result of common
origins. The problem with these "confusable" characters, often
incorrectly called homographs, has always existed when characters are
presented to humans who interpret what is displayed and then make
decisions based on what is seen. This is not a problem that exists
only when working with internationalized domain names, but they make
the problem worse. The result of a survey that would explain what
the problems are might be interesting. Many of these issues are
mentioned in Unicode Technical Report #36 [UTR36].
In this and other issues associated with IDNs, precise use of
terminology is important lest even more confusion result. The
definition of the term ’homograph’ that normally appears in
dictionaries and linguistic texts states that homographs are
different words that are spelled identically (for example, the
adjective ’brief’ meaning short, the noun ’brief’ meaning a document,
and the verb ’brief’ meaning to inform). By definition, letters in
two different alphabets are not the same, regardless of similarities
in appearance. This means that sequences of letters from two
different scripts that appear to be identical on a computer display
cannot be homographs in the accepted sense, even if they are both
words in the dictionary of some language. Assuming that there is a
language written with Cyrillic script in which "cap" is a word,
regardless of what it might mean, it is not a homograph of the
Latin-script English word "cap".
When the security implications of visually confusable characters were
brought to the forefront in 2005, the term homograph was used to
designate any instance of graphic similarity, even when comparing
individual characters. This usage is not only incorrect, but risks
introducing even more confusion and hence should be avoided. The
current preferred terminology is to describe these similar-looking
characters as "confusable characters" or even "confusables".
Many people have suggested that confusable characters are a problem
that must be addressed, at least in part, directly in the user
interfaces of application software. While it should almost certainly
be part of a complete solution, that approach creates it own set of
difficulties. For example, a user switching between systems, or even
between applications on the same system, may be surprised by
different types of behavior and different levels of protection. In
addition, it is unclear how a secure setup for the end user should be
designed. Today, in the web browser, a padlock is a traditional way
of describing some level of security for the end user. Is this
binary signaling enough? Should there be any connection between a
risk for a displayed string including confusable characters and the
padlock or similar signaling to the user?
Many web browsers have adopted a convention, based on a "whitelist"
or similar technique, of restricting the display of native characters
to subdomains of top-level domains that are deemed to have safe
practices for the registration of potentially confusable labels.
IDNs in other domains are displayed as Punycode. These techniques
may not be sufficiently sensitive to differences in policies among
top-level domains and their subdomains and so, while they are clearly
helpful, they may not be adequate. Are other methods of dealing with
confusable characters possible? Would other methods of identifying
and listing policies about avoiding confusing registrations be
feasible and helpful?
It would be interesting to see a more coordinated effort in
establishing guidelines for user interfaces. If nothing else, the
current whitelists are browser specific and both can, and do, differ
between implementations.
4.1.4. Protocol Changes and Policy Implications
Some potential protocol or table changes raise important policy
issues about what to do with existing, registered, names. Should
such changes be needed, their impact must be carefully evaluated in
the IETF, ICANN, and possibly other forums. In particular, protocol
or policy changes that would not permit existing names to be
registered under the newer rules should be considered carefully,
balancing their importance against possible disruption and the issues
of invalidating older names against the importance of consistency as
seen by the user.
4.1.5. Non-US-ASCII in Local Part of Email Addresses
Work is going on in the IETF related to the local part of email
addresses. It should be noted that the local part of email addresses
has much different syntax and constraints than a domain name label,
so to directly apply IDNA on the local part is not possible.
4.1.6. Use of the Unicode Character Set in the IETF
Unicode and the closely-related ISO 10646 are the only coded
character sets that aspire to include all of the world’s characters.
As such, they permit use of international characters without having
to identify particular character coding standards or tables. The
requirement for a single character set is particularly important for
use with the DNS since there is no place to put character set
identification. The decision to use Unicode as the base for IETF
protocols going forward is discussed in [RFC2277]. The IAB does not
see any reason to revisit the decision to use Unicode in IETF
protocols.
4.2. Issues That Fall within the Purview of ICANN
4.2.1. Dispute Resolution
IDNs create new types of collisions between trademarks and domain
names as well as collisions between domain names. These have impact
on dispute resolution processes used by registries and otherwise. It
is important that deployment of IDNs evolve in parallel with review
and updating of ICANN or registry-specific dispute resolution
processes.
4.2.2. Policy at Registries
The IAB recommends that registries use an inclusion-based model when
choosing what characters to allow at the time of registration. This
list of characters is in turn to be a subset of what is allowed
according to the updated IDNA standard. The IAB further recommends
that registries develop their inclusion-based models in parallel with
dispute resolution process at the registry itself.
Most established policies for dealing with claimed or apparent
confusion or conflicts of names are based on dispute resolution.
Decisions about legitimate use or registration of one or more names
are resolved at or after the time of registration on a case-by-case
basis and using policies that are specific to the particular DNS zone
or jurisdiction involved. These policies have generally not been
extended below the level of the DNS that is directly controlled by
the top-level registry.
Because of the number of conflicts that can be generated by the
larger number of available and confusable characters in Unicode, we
recommend that registration-restriction and dispute resolution
policies be developed to constrain registration of IDNs and zone
administrators at all levels of the DNS tree. Of course, many of
these policies will be less formal than others and there is no
requirement for complete global consistency, but the arguments for
reduction of confusable characters and other issues in TLDs should
apply to all zones below that specific TLD.
Consistency across all zones can obviously only be accomplished by
changes to the protocols. Such changes should be considered by the
IETF if particular restrictions are identified that are important and
consistent enough to be applied globally.
Some potential protocol changes or changes to character-mapping
tables might, if adopted, have profound registry policy implications.
See Section 4.1.4.
4.2.3. IDNs at the Top Level of the DNS
The IAB has concluded that there is not one issue with IDNs at the
top level of the DNS (IDN TLDs) but at least three very separate
ones:
o If IDNs are to be entered in the root zone, decisions must first
be made about how these TLDs are to be named and delegated. These
decisions fall within the traditional IANA scope and are ICANN
issues today.
o There has been discussion of permitting some or all existing TLDs
to be referenced by multiple labels, with those labels presumably
representing some understanding of the "name" of the TLD in
different languages. If actual aliases of this type are desired
for existing domains, the IETF may need to consider whether the
use of DNAME records in the root is appropriate to meet that need,
what constraints, if any, are needed, whether alternate
approaches, such as those of [RFC4185], are appropriate or whether
further alternatives should be investigated. But, to the extent
to which aliases are considered desirable and feasible, decisions
presumably must be made as to which, if any, root IDN labels
should be associated with DNAME records and which ones should be
handled by normal delegation records or other mechanisms. That