but with the DNS-imposed requirement that its results, as with all
other parts of the matching and comparison process, yield a binary
"match or no match" answer, rather than, e.g., a value on a
similarity scale that can be evaluated by the user or by user-driven
heuristic functions.
4.4 The Unicode Stability Problem
ISO 10646 basically defines only code points, and not rules for using
or comparing the characters. This is part of a long-standing
tradition with the work of what is now ISO/IEC JTC1/SC2: they have
performed code point assignments and have typically treated the ways
in which characters are used as beyond their scope. Consequently,
they have not dealt effectively with the broader range of
internationalization issues. By contrast, the Unicode Technical
Committee (UTC) has defined, in annexes and technical reports (see,
e.g., [UTR15]), some additional rules for canonicalization and
comparison. Many of those rules and conventions have been factored
into the "stringprep" and "nameprep" work, but it is not
straightforward to make or define them in a fashion that is
sufficiently precise and permanent to be relied on by the DNS.
Perhaps more important, the discussions leading to nameprep also
identified several areas in which the UTC definitions are inadequate,
at least without additional information, to make matching precise and
unambiguous. In some of these cases, the Unicode Standard permits
several alternate approaches, none of which are an exact and obvious
match to DNS needs. That has left these sensitive choices up to
IETF, which lacks sufficient in-depth expertise, much less any
mechanism for deciding to optimize one language at the expense of
another.
For example, it is tempting to define some rules on the basis of
membership in particular scripts, or for punctuation characters, but
there is no precise definition of what characters belong to which
script or which ones are, or are not, punctuation. The existence of
these areas of vagueness raises two issues: whether trying to do
precise matching at the character set level is actually possible
(addressed below) and whether driving toward more precision could
create issues that cause instability in the implementation and
resolution models for the DNS.
The Unicode definition also evolves. Version 3.2 appeared shortly
after work on this document was initiated. It added some characters
and functionality and included a few minor incompatible code point
changes. IETF has secured an agreement about constraints on future
changes, but it remains to be seen how that agreement will work out
in practice. The prognosis actually appears poor at this stage,
since UTC chose to ballot a recent possible change which should have
been prohibited by the agreement (the outcome of the ballot is not
relevant, only that the ballot was issued rather than having the
result be a foregone conclusion). However, some members of the
community consider some of the changes between Unicode 3.0 and 3.1
and between 3.1 and 3.2, as well as this recent ballot, to be
evidence of instability and that these instabilities are better
handled in a system that can be more flexible about handling of
characters, scripts, and ancillary information than the DNS.
In addition, because the systems implications of internationalization
are considered out of scope in SC2, ISO/IEC JTC1 has assigned some of
those issues to its SC22/WG20 (the Internationalization working group
within the subcommittee that deals with programming languages,
systems, and environments). WG20 has historically dealt with
internationalization issues thoughtfully and in depth, but its status
has several times been in doubt in recent years. However, assignment
of these matters to WG20 increases the risk of eventual ISO
internationalization standards that specify different behavior than
the UTC specifications.
4.5 Audiences, End Users, and the User Interface Problem
Part of what has "caused" the DNS internationalization problem, as
well as the DNS trademark problem and several others, is that we have
stopped thinking about "identifiers for objects" -- which normal
people are not expected to see -- and started thinking about "names"
-- strings that are expected not only to be readable, but to have
linguistically-sensible and culturally-dependent meaning to non-
specialist users.
Within the IETF, the IDN-WG, and sometimes other groups, avoided
addressing the implications of that transition by taking "outside our
scope -- someone else's problem" approaches or by suggesting that
people will just become accustomed to whatever conventions are
adopted. The realities of user and vendor behavior suggest that
these approaches will not serve the Internet community well in the
long term:
o If we want to make it a problem in a different part of the user
interface structure, we need to figure out where it goes in order
to have proof of concept of our solution. Unlike vendors whose
sole [business] model is the selling or registering of names, the
IETF must produce solutions that actually work, in the
applications context as seen by the end user.
o The principle that "they will get used to our conventions and
adapt" is fine if we are writing rules for programming languages
or an API. But the conventions under discussion are not part of a
semi-mathematical system, they are deeply ingrained in culture.
No matter how often an English-speaking American is told that the
Internet requires that the correct spelling of "colour" be used,
he or she isn't going to be convinced. Getting a French-speaker in
Lyon to use exactly the same lexical conventions as a French-
speaker in Quebec in order to accommodate the decisions of the
IETF or of a registrar or registry is just not likely. "Montreal"
is either a misspelling or an anglicization of a similar word with
an acute accent mark over the "e" (i.e., using the Unicode
character U+00E9 or one of its equivalents). But global agreement
on a rule that will determine whether the two forms should match
-- and that won't astonish end users and speakers of one language
or the other -- is as unlikely as agreement on whether
"misspelling" or "anglicization" is the greater travesty.
More generally, it is not clear that the outcome of any conceivable
nameprep-like process is going to be good enough for practical,
user-level, use. In the use of human languages by humans, there are
many cases in which things that do not match are nonetheless
interpreted as matching. The Norwegian/Danish character that appears
in U+00F8 (visually, a lower case 'o' overstruck with a forward
slash) and the "o-umlaut" German character that appears in U+00F6
(visually, a lower case 'o' with diaeresis (or umlaut)) are clearly
different and no matching program should yield an "equal" comparison.
But they are more similar to each other than either of them is to,
e.g., "e". Humans are able to mentally make the correction in
context, and do so easily, and they can be surprised if computers
cannot do so. Worse, there is a Swedish character whose appearance
is identical to the German o-umlaut, and which shares code point
U+00F6, but that, if the languages are known and the sounds of the
letters or meanings of words including the character are considered,
actually should match the Norwegian/Danish use of U+00F8.
This text uses examples in Roman scripts because it is being written
in English and those examples are relatively easy to render. But one
of the important lessons of the discussions about domain name
internationalization in recent years is that problems similar to
those described above exist in almost every language and script.
Each one has its idiosyncrasies, and each set of idiosyncracies is
tied to common usage and cultural issues that are very familiar in
the relevant group, and often deeply held as cultural values. As
long as a schoolchild in the US can get a bad grade on a spelling
test for using a perfectly valid British spelling, or one in France
or Germany can get a poor grade for leaving off a diacritical mark,
there are issues with the relevant language. Similarly, if children
in Egypt or Israel are taught that it is acceptable to write a word
with or without vowels or stress marks, but that, if those marks are
included, they must be the correct ones, or a user in Korea is
potentially offended or astonished by out-of-order sequences of Jamo,
systems based on character-at-a-time processing and simplistic
matching, with no contextual information, are not going to satisfy
user needs.
Users are demanding solutions that deal with language and culture.
Systems of identifier symbol-strings that serve specialists or
computers are, at best, a solution to a rather different (and, at the
time this document was written, somewhat ill-defined), problem. The
recent efforts have made it ever more clear that, if we ignore the
distinction between the user requirements and narrowly-defined
identifiers, we are solving an insufficient problem. And,
conversely, the approaches that have been proposed to approximate
solutions to the user requirement may be far more complex than simple
identifiers require.
4.6 Business Cards and Other Natural Uses of Natural Languages
Over the last few centuries, local conventions have been established
in various parts of the world for dealing with multilingual
situations. It may be helpful to examine some of these. For
example, if one visits a country where the language is different from
ones own, business cards are often printed on two sides, one side in
each language. The conventions are not completely consistent and the
technique assumes that recipients will be tolerant. Translations of
names or places are attempted in some situations and transliterations
in others. Since it is widely understood that exact translations or
transliterations are often not possible, people typically smile at
errors, appreciate the effort, and move on.
The DNS situation differs from these practices in at least two ways.
Since a global solution is required, the business card would need a
number of sides approximating the number of languages in the world,
which is probably impossible without violating laws of physics. More
important, the opportunities for tolerance don't exist: the DNS
requires a exact match or the lookup fails.
4.7 ASCII Encodings and the Roman Keyboard Assumption
Part of the argument for ACE-based solutions is that they provide an
escape for multilingual environments when applications have not been
upgraded. When an older application encounters an ACE-based name,
the assumption is that the (admittedly ugly) ASCII-coded string will
be displayed and can be typed in. This argument is reasonable from
the standpoint of mixtures of Roman-based alphabets, but may not be
relevant if user-level systems and devices are involved that do not
support the entry of Roman-based characters or which cannot
conveniently render such characters. Such systems are few in the
world today, but the number can reasonably be expected to rise as the
Internet is increasingly used by populations whose primary concern is
with local issues, local information, and local languages. It is,
for example, fairly easy to imagine populations who use Arabic or
Thai scripts and who do not have routine access to scripts or input
devices based on Roman-derived alphabets.
4.8 Intra-DNS Approaches for "Multilingual Names"
It appears, from the cases above and others, that none of the intra-
DNS-based solutions for "multilingual names" are workable. They rest
on too many assumptions that do not appear to be feasible -- that
people will adapt deeply-entrenched language habits to conventions
laid down to make the lives of computers easy; that we can make
"freeze it now, no need for changes in these areas" decisions about
Unicode and nameprep; that ACE will smooth over applications
problems, even in environments without the ability to key or render
Roman-based glyphs (or where user experience is such that such glyphs
cannot easily be distinguished from each other); that the Unicode
Consortium will never decide to repair an error in a way that creates
a risk of DNS incompatibility; that we can either deploy EDNS
[RFC2671] or that long names are not really important; that Japanese
and Chinese computer users (and others) will either give up their
local or IS 2022-based character coding solutions (for which addition
of a large fraction of a million new code points to Unicode is almost
certainly a necessary, but probably not sufficient, condition) or
build leakproof and completely accurate boundary conversion
mechanisms; that out of band or contextual information will always be
sufficient for the "map glyph onto script" problem; and so on. In
each case, it is likely that about 80% or 90% of cases will work
satisfactorily, but it is unlikely that such partial solutions will
be good enough. For example, suppose someone can spell her name 90%
correctly, or a company name is matched correctly 80% of the time but
the other 20% of attempts identify a competitor: are either likely to
be considered adequate?
5. Search-based Systems: The Key Controversies
For many years, a common response to requirements to locate people or
resources on the Internet has been to invoke the term "directory".
While an in-depth analysis of the reasons would require a separate
document, the history of failure of these invocations has given
"directory" efforts a bad reputation. The effort proposed here is
different from those predecessors for several reasons, perhaps the
most important of which is that it focuses on a fairly-well-
understood set of problems and needs, rather than on finding uses for
a particular technology.
As suggested in some of the text above, it is an open question as to
whether the needs of the community would be best served by a single
(even if functionally, and perhaps administratively, distributed)
directory with universal applicability, a single directory that
supports locally-tailored search (and, most important, matching)
functions, or multiple, locally-determined, directories. Each has
its attractions. Any but the first would essentially prevent
reverse-mapping (determination of the user-visible name of the host
or resource from target information such as an address or DNS name).
But reverse mapping has become less useful over the years --at least
to users -- as more and more names have been associated with many
host addresses and as CIDR [CIDR] has proven problematic for mapping
smaller address blocks to meaningful names.
Locally-tailored searches and mappings would permit national
variations on interpretation of which strings matched which other
ones, an arrangement that is especially important when different
localities apply different rules to, e.g., matching of characters
with and without diacriticals. But, of course, this implies that a
URL may evaluate properly or not depending on either settings on a
client machine or the network connectivity of the user. That is not,
in general, a desirable situation, since it implies that users could
not, in the general case, share URLs (or other host references) and
that a particular user might not be able to carry references from one
host or location to another.
And, of course, completely separate directories would permit
translation and transliteration functions to be embedded in the
directory, giving much of the Internet a different appearance
depending on which directory was chosen. The attractions of this are
obvious, but, unless things were very carefully designed to preserve
uniqueness and precise identities at the right points (which may or
may not be possible), such a system would have many of the
difficulties associated with multiple DNS roots.
Finally, a system of separate directories and databases, if coupled
with removal of the DNS-imposed requirement for unique names, would
largely eliminate the need for a single worldwide authority to manage
the top of the naming hierarchy.
6. Security Considerations
The set of proposals implied by this document suggests an interesting
set of security issues (i.e., nothing important is ever easy). A
directory system used for locating network resources would presumably
need to be as carefully protected against unauthorized changes as the
DNS itself. There also might be new opportunities for problems in an
arrangement involving two or more (sub)layers, especially if such a
system were designed without central authority or uniqueness of
names. It is uncertain how much greater those risks would be as
compared to a DNS lookup sequence that involved looking up one name,
getting back information, and then doing additional lookups
potentially in different subtrees. That multistage lookup will often
be the case with, e.g., NAPTR records [RFC2915] unless additional
restrictions are imposed. But additional steps, systems, and
databases almost certainly involve some additional risks of
compromise.
7. References
7.1 Normative References
None
7.2 Explanatory and Informative References
[Albitz] Any of the editions of Albitz, P. and C. Liu, DNS and
BIND, O'Reilly and Associates, 1992, 1997, 1998, 2001.
[ASCII] American National Standards Institute (formerly United
States of America Standards Institute), X3.4, 1968,
"USA Code for Information Interchange". ANSI X3.4-1968
has been replaced by newer versions with slight
modifications, but the 1968 version remains definitive
for the Internet. Some time after ASCII was first
formulated as a standard, ISO adopted international
standard 646, which uses ASCII as a base. IS 646
actually contained two code tables: an "International
Reference Version" (often referenced as ISO 646-IRV)
which was essentially identical to the ASCII of the
time, and a "Basic Version" (ISO 646-BV), which
designates a number of character positions for
national use.
[CIDR] Fuller, V., Li, T., Yu, J. and K. Varadhan, "Classless
Inter-Domain Routing (CIDR): an Address Assignment and
Aggregation Strategy", RFC1519, September 1993.
Eidnes, H., de Groot, G. and P. Vixie, "Classless IN-
ADDR.ARPA delegation", RFC2317, March 1998.
[COM-SIZE] Size information supplied by Verisign Global Registry
Services (the zone administrator, or "registry
operator", for COM, see [REGISTRAR], below) to ICANN,
third quarter 2002.
[DNS-Search] Klensin, J., "A Search-based access model for the
DNS", Work in Progress.
[FINGER] Zimmerman, D., "The Finger User Information Protocol",
RFC1288, December 1991.
Harrenstien, K., "NAME/FINGER Protocol", RFC742,
December 1977.
[IAB-OPES] Floyd, S. and L. Daigle, "IAB Architectural and Policy
Considerations for Open Pluggable Edge Services", RFC
3238, January 2002.
[IQUERY] Lawrence, D., "Obsoleting IQUERY", RFC3425, November
2002.
[IS646] ISO/IEC 646:1991 Information technology -- ISO 7-bit
coded character set for information interchange
[IS10646] ISO/IEC 10646-1:2000 Information technology --
Universal Multiple-Octet Coded Character Set (UCS) --
Part 1: Architecture and Basic Multilingual Plane and
ISO/IEC 10646-2:2001 Information technology --
Universal Multiple-Octet Coded Character Set (UCS) --
Part 2: Supplementary Planes
[MINC] The Multilingual Internet Names Consortium,
http://www.minc.org/ has been an early advocate for
the importance of expansion of DNS names to
accommodate non-ASCII characters. Some of their
specific proposals, while helping people to understand
the problems better, were not compatible with the
design of the DNS.
[NAPTR] Mealling, M. and R. Daniel, "The Naming Authority
Pointer (NAPTR) DNS Resource Record", RFC2915,
September 2000.
Mealling, M., "Dynamic Delegation Discovery System
(DDDS) Part One: The Comprehensive DDDS", RFC3401,
October 2002.
Mealling, M., "Dynamic Delegation Discovery System
(DDDS) Part Two: The Algorithm", RFC3402, October
2002.
Mealling, M., "Dynamic Delegation Discovery System
(DDDS) Part Three: The Domain Name System (DNS)
Database", RFC3403, October 2002.
[REGISTRAR] In an early stage of the process that created the
Internet Corporation for Assigned Names and Numbers
(ICANN), a "Green Paper" was released by the US
Government. That paper introduced new terminology
and some concepts not needed by traditional DNS
operations. The term "registry" was applied to the
actual operator and database holder of a domain
(typically at the top level, since the Green Paper was
little concerned with anything else), while
organizations that marketed names and made them
available to "registrants" were known as "registrars".
In the classic DNS model, the function of "zone
administrator" encompassed both registry and registrar
roles, although that model did not anticipate a
commercial market in names.
[RFC625] Kudlick, M. and E. Feinler, "On-line hostnames
service", RFC625, March 1974.
[RFC734] Crispin, M., "SUPDUP Protocol", RFC734, October 1977.
[RFC811] Harrenstien, K., White, V. and E. Feinler, "Hostnames
Server", RFC811, March 1982.
[RFC819] Su, Z. and J. Postel, "Domain naming convention for
Internet user applications", RFC819, August 1982.
[RFC830] Su, Z., "Distributed system for Internet name
service", RFC830, October 1982.
[RFC882] Mockapetris, P., "Domain names: Concepts and
facilities", RFC882, November 1983.
[RFC883] Mockapetris, P., "Domain names: Implementation
specification", RFC883, November 1983.
[RFC952] Harrenstien, K, Stahl, M. and E. Feinler, "DoD
Internet host table specification", RFC952, October
1985.
[RFC953] Harrenstien, K., Stahl, M. and E. Feinler, "HOSTNAME
SERVER", RFC953, October 1985.
[RFC1034] Mockapetris, P., "Domain names, Concepts and
facilities", STD 13, RFC1034, November 1987.
[RFC1035] Mockapetris, P., "Domain names - implementation and
specification", STD 13, RFC1035, November 1987.
[RFC1591] Postel, J., "Domain Name System Structure and
Delegation", RFC1591, March 1994.
[RFC2181] Elz, R. and R. Bush, "Clarifications to the DNS
Specification", RFC2181, July 1997.
[RFC2295] Holtman, K. and A. Mutz, "Transparent Content
Negotiation in HTTP", RFC2295, March 1998
[RFC2396] Berners-Lee, T., Fielding, R. and L. Masinter,
"Uniform Resource Identifiers (URI): Generic Syntax",
RFC2396, August 1998.
[RFC2608] Guttman, E., Perkins, C., Veizades, J. and M. Day,
"Service Location Protocol, Version 2", RFC2608, June
1999.
[RFC2671] Vixie, P., "Extension Mechanisms for DNS (EDNS0)", RFC
2671, August 1999.
[RFC2825] IAB, Daigle, L., Ed., "A Tangled Web: Issues of I18N,
Domain Names, and the Other Internet protocols", RFC
2825, May 2000.
[RFC2826] IAB, "IAB Technical Comment on the Unique DNS Root",
RFC2826, May 2000.
[RFC2972] Popp, N., Mealling, M., Masinter, L. and K. Sollins,
"Context and Goals for Common Name Resolution", RFC
2972, October 2000.
[RFC3305] Mealling, M. and R. Denenberg, Eds., "Report from the
Joint W3C/IETF URI Planning Interest Group: Uniform
Resource Identifiers (URIs), URLs, and Uniform
Resource Names (URNs): Clarifications and
Recommendations", RFC3305, August 2002.
[RFC3439] Bush, R. and D. Meyer, "Some Internet Architectural
Guidelines and Philosophy", RFC3439, December 2002.
[Seng] Seng, J., et al., Eds., "Internationalized Domain
Names: Registration and Administration Guideline for
Chinese, Japanese, and Korean", Work in Progress.
[STRINGPREP] Hoffman, P. and M. Blanchet, "Preparation of
Internationalized Strings (stringprep)", RFC3454,
December 2002.
The particular profile used for placing
internationalized strings in the DNS is called
"nameprep", described in Hoffman, P. and M. Blanchet,
"Nameprep: A Stringprep Profile for Internationalized
Domain Names", Work in Progress.
[TELNET] Postel, J. and J. Reynolds, "Telnet Protocol
Specification", STD 8, RFC854, May 1983.
Postel, J. and J. Reynolds, "Telnet Option
Specifications", STD 8, RFC855, May 1983.
[UNICODE] The Unicode Consortium, The Unicode Standard, Version
3.0, Addison-Wesley: Reading, MA, 2000. Update to
version 3.1, 2001. Update to version 3.2, 2002.
[UTR15] Davis, M. and M. Duerst, "Unicode Standard Annex #15:
Unicode Normalization Forms", Unicode Consortium,
March 2002. An integral part of The Unicode Standard,
Version 3.1.1. Available at
(http://www.unicode.org/reports/tr15/tr15-21.html).
[WHOIS] Harrenstien, K, Stahl, M. and E. Feinler,
"NICNAME/WHOIS", RFC954, October 1985.
[WHOIS-UPDATE] Gargano, J. and K. Weiss, "Whois and Network
Information Lookup Service, Whois++", RFC1834, August
1995.
Weider, C., Fullton, J. and S. Spero, "Architecture of
the Whois++ Index Service", RFC1913, February 1996.
Williamson, S., Kosters, M., Blacka, D., Singh, J. and
K. Zeilstra, "Referral Whois (RWhois) Protocol V1.5",
RFC2167, June 1997;
Daigle, L. and P. Faltstrom, "The
application/whoispp-query Content-Type", RFC2957,
October 2000.
Daigle, L. and P. Falstrom, "The application/whoispp-
response Content-type", RFC2958, October 2000.
[X29] International Telecommuncations Union, "Recommendation
X.29: Procedures for the exchange of control
information and user data between a Packet
Assembly/Disassembly (PAD) facility and a packet mode
DTE or another PAD", December 1997.
8. Acknowledgements
Many people have contributed to versions of this document or the
thinking that went into it. The author would particularly like to
thank Harald Alvestrand, Rob Austein, Bob Braden, Vinton Cerf, Matt
Crawford, Leslie Daigle, Patrik Faltstrom, Eric A. Hall, Ted Hardie,
Paul Hoffman, Erik Nordmark, and Zita Wenzel for making specific
suggestions and/or challenging the assumptions and presentation of
earlier versions and suggesting ways to improve them.
9. Author's Address
John C. Klensin
1770 Massachusetts Ave, #322
Cambridge, MA 02140
EMail: klensin+srch@jck.com
A mailing list has been initiated for discussion of the topics
discussed in this document, and closely-related issues, at
ietf-irnss@lists.elistx.com. See http://lists.elistx.com/archives/
for subscription and archival information.
10. Full Copyright Statement
Copyright (C) The Internet Society (2003). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFCEditor function is currently provided by the
Internet Society.