usually have the same interpretation in each one. For the subset of
characters that actually are ideographs or pictographs, pronunciation
is expected to vary widely while meaning is preserved. At least in
part because of that similarity of meaning, it made sense in the JET
case to permit a registration to specify multiple languages, to
verify that the characters in the label string (the requested "Base
registration") were valid for each, and then to generate variant
labels using each language in turn. For many alphabetic languages,
it may be more sensible to prohibit the label string submitted for
registration from being associated with more than one language.
Indeed, "one label, one language" has been suggested as an important
barrier against common sources of "look-alike" confusion. For
example, the imposition of that rule in a zone would prevent the
insertion of a few Greek or Cyrillic characters with shapes identical
to the Latin ones into what was otherwise a Latin-based string. For
a particular table, the list of base characters may be thought of as
the script associated with the relevant language, with the
understanding that the table design does not prevent the same
character from appearing in the tables for multiple languages.
Indeed, this notion of a script that is local and specifically
identified can be turned around: so-called "language tables" are
associated with languages only insofar as thinking about the
character structure and word forms associated with a given language
helps to inform the construction of the table. A country like
Finland, for example, might select among:
o One table each for Finnish, Swedish, and English characters and
conventions, permitting a string to be registered in one, two, or
all three languages. However, a three-language registration would
necessarily prohibit any characters that did not appear in all
three languages, since the label would make little sense
otherwise.
o One table each, but with a "one label, one language" rule for the
zone.
o A combined table based on the observation that all three writing
systems were based on Roman characters and that the possibilities
for confusion of interest to the registry would not be reduced by
"language" differentiation. This option raises an interesting
issue about language labeling as described in Section 1.4.1; see
the discussion in Section 7 below.
Regardless of what decisions were made about those languages and
scripts, they might have a separate table for registration of labels
containing Cyrillic characters. That table might contain some
Roman-derived characters (either as base characters or as variants),
just as some CJK tables do. See also Section 2, below.
Tables that present multiple languages, as described above, have
introduced confusion and discomfort among those who have failed to
understand these definitions. The consequence of these definitions
is that use of a language or script code in a registration is a
mnemonic, rather than a normative statement about the language or
script itself. When that confusion is likely to occur, it is
appropriate to simply use the registry identifier and a sequence
number to identify the registration.
As the JET Guidelines stress, no tables or systems of this type --
even if identified with a language as a means of defining or
describing the table -- can assure linguistic or even syntactic
correctness of labels with regard to that language. That assurance
may not be possible without human intervention or at least dictionary
lookups of complete proposed labels. It may even not be desirable to
attempt that level of correctness (see Section 2).
Of course, if any language-based tests or constraints, including "one
label, one language", are to be applied to limit the associated
sources of confusion, each zone must have a table for each language
in which it expects to accept registrations. The notion of a single
combined table for the zone is, in the general case, simply
unworkable. One could use a single table for the zone if the intent
were to impose only minimal restrictions, e.g., to force alphabetic
and numeric characters only, excluding symbols and punctuation. That
type of restriction might be useful in eliminating some problems,
such as those of unreadable labels, but it would be unlikely to be
very helpful with, e.g., confusion caused by similar-looking
characters.
1.5.2. Variant Selection
The area of character variants is rife with difficulties (and perhaps
opportunities). There is no universal agreement about which base
characters have variants, or if they do, what those variants are.
For example, in some regions of the world and in some languages,
LATIN SMALL LETTER O WITH DIAERESIS (U+00F6) and LATIN SMALL LETTER O
WITH STROKE (U+00F8) are variants of each other, while in other
regions, most people would think that LATIN SMALL LETTER O WITH
STROKE has no variants. In some cases, the list of variants is
difficult to enumerate. For example, it required several years for
the Chinese language community to create variant tables for use with
IDNA, and it remains, at the time of this writing, questionable how
widely those tables will be accepted among users of Chinese from
areas of the world other than those represented by the groups that
created them.
Thus, the first thing a registry should ask is whether or not any of
the characters that they want to permit to be used have variants. If
not, the registry’s work is much simpler. This is not to say that a
registry should ignore variants if they exist: adding variants after
a registry has started to take registrations will be nearly as
difficult administratively as removing characters from the list of
acceptable characters. That is, if a registry later decides that two
characters are variants of each other, and there are actively-used
names in the zones that differ only on the new variants, the registry
might have to transfer ownership of one of the names to a different
owner, using some process that is certain to be controversial.
This situation in likely to be much easier for areas and zones that
use characters that previously did not occur in the DNS at all than
it will be for zones in which non-English labels have been registered
in ASCII characters for some time, presumably because the language of
interest uses additional "Latin" characters with some conventions
when only ASCII is available. In the former case, the rules and
conventions can be established before any registrations occur. In
the latter, there may be conflicts or opportunities for confusion
between existing registrations and now-permitted Roman-based
characters that do not appear in ASCII. For example, a domain name
might exist today that uses the name of a city in Canada spelled as
"Montreal". If the zone in which it occurs changes its rules to
permit the use of the character LATIN SMALL LETTER E WITH ACUTE
(U+00E9), does the name of the city, spelled (correctly) using that
character, conflict with the existing domain name registration?
Certainly, if both are permitted, and permitted to be registered by
separate parties, there are many opportunities for confusion.
Of course, zone managers should inform all current registrants when
the registration policy for the zone changes. This includes the
times when IDN characters are first allowed in the zone, when
additional characters are permitted, and when any change occurs in
the character variant tables.
Many languages contain two variants for a character, one of which is
strongly preferred. A registry might restrict the base registration
to the preferred form, or it might allow any form for the base
registration. If the variant tables are created carefully, the
resulting bundles will be the same, but some registries will give
special status to the base registration such as its appearance in
"Whois" databases.
1.6. Variants are not a Universal Remedy
It is worth stressing that there are many obvious opportunities for
confusion that variant systems, by virtue of being based on
processing of individual characters, cannot address. For example, if
a language can be written with more than one script, or
transliterations of the language into another script are common,
variant models are insufficient to prevent conflicting registration
of the related forms. Avoiding those types of problems would require
different mechanisms, perhaps based on phonetic or natural language
processing techniques for the entire proposed base registration.
1.7. Reservations and Exclusions
1.7.1. Sequence Exclusions for Valid Characters
The JET Guidelines are based on processing only single characters.
Pairs or longer sequences of characters can, at the option of the
registry, be handled through what the Guidelines describe as
"additional processing". These registry-specific string processing
procedures are specifically permitted by the guidelines to supplement
the per-character processing that generates the variants.
A different zone with different needs could use a modified version of
the table structure, or different types of additional processing, to
prohibit particular sequences of characters by marking them as
invalid, and to accept characters by marking them as valid. Other
modifications or extensions might be designed to prevent certain
letters from appearing at the beginning or end of labels. The use of
regular expressions in the "valid characters" column might be one way
to implement these types of restrictions, but there has been no
experience so far with that approach.
In particular, in some scripts derived from Roman characters,
sequences that have historically been typographically represented by
single "ligature" or "digraph" characters may also be represented by
the separate characters (e.g., "ae" for U+00E6 or "ij" for U+0133).
If it is desired to either prohibit these, or to treat them as
variants, some extensions to the single-character JET model may be
needed. Some careful thinking about IDNA (especially nameprep) may
also be needed, since some of these combinations are excluded there).
1.7.2. Character Pairing Issues
Some character pairings -- the use of a character form (glyph) in one
language and a different form with the same properties in a related
one -- closely approximate the issues with mapping between
Traditional and Simplified Chinese, although the history is
different. For example, it might be useful to have "o" with a stroke
(U+00F8) as a variant for "o" with diaeresis above it (U+00F6) (and
the equivalent upper-case pair) in a Swedish table, and vice versa in
a Norwegian one, or to prohibit one of these characters entirely in
each table. In a German table, U+00F8 would presumably be
prohibited, while U+00F6 might have "oe" as a variant. Obviously, if
the relevant language of registration is unknown, this type of
variant matching cannot be applied in any sensible way.
1.8. The Registration Bundle
1.8.1. Definitions and Structure
As one of its critical innovations, the JET model defines an "IDN
package", known in this document as a "registration bundle", which
consists of the primary registered string (which is used as the name
of the bundle), the information about the language table(s) used, the
variant labels for that string, and indications of which of those
labels are registered in the relevant zone file ("activated" in the
JET terminology). Registration bundles are also atomic -- one can
not add or remove variant labels from one without unregistering the
entire package. A label exists in only one registration bundle at a
time; if a new label is registered that would generate a variant that
matches one that appears in an existing package, that variant simply
is not included in the second package. A subsequent de-registration
of the first package does not cause the variant to be added to the
second. While it might be possible to change this in other models,
the JET conclusion was that other options would be far too complex to
implement and operate and would cause many new types of name
conflicts.
1.8.2. Application of the Registration Bundle
A registry has three options for handling the case where the
registration bundle contains more than one label. The policy options
are:
o Register and resolve all labels in the zone, making the zone
information identical to that of the registered labels. This
option will allow end users to find names with variants more
easily, but will result in larger zone files. For some language
tables, the zone file could become so large that it could
negatively affect the ability of the registry to perform name
resolution. If the base registration contains several characters
that have equivalents, the owner could end up having to take care
of large numbers of zones. For instance, if DIGIT ONE is a
variant of LATIN SMALL LETTER L, the owner of the domain name all-
lollypops.example.com will have to manage 32 zones. If the intent
is to keep the contents of those zones identical, the owner may
then face a significant administrative problem. If other concerns
dictate short times to live and absolute consistency of DNS
responses, the challenges may be nearly impossible.
o Block all labels other than the registered label so they cannot be
registered in the future. This option does not increase the size
of the zone file and provides maximum safety against false
positives, but it may cause end users to not be able to find names
with variants that they would expect. If the base registration
contains characters that have equivalents, Internet users who do
not know what base characters were used in the registration will
not know what character to type in to get a DNS response. For
instance, if DIGIT ONE is a variant of LATIN SMALL LETTER L, and
LATIN SMALL LETTER L is a variant of DIGIT ONE, the user who sees
"pale.example.com" will not know whether to type a "1" or a "l"
after the "pa" in the first label.
o Resolve some labels and block some other labels. This option is
likely to cause the most confusion with users because including
some variants will cause a name to be found, but using other
variants will cause the name to be not found. For example, even
if people understood that DIGIT ONE and LATIN SMALL LETTER L were
variants, a typical DNS user wouldn’t know which character to type
because they wouldn’t know whether this pair were used to register
or block the labels. However, this option can be used to balance
the desires of the name owner (that every possible attempt to
enter their name will work) with the desires of the zone
administrator (to make the zone more manageable and possibly to be
compensated for greater amounts of work needed for a single
registration). For many circumstances, it may be the most
attractive option.
In all cases, at least the registered label should appear in the
zone. It would be almost impossible to describe to name owners why
the name that they asked for is not in the zone, but some other name
that they now control is. By implication, if the requested label is
already registered, the entire registration request must be rejected.
2. Some Implications of This Approach
Historically, DNS labels were considered to be arbitrary identifier
strings, without any inherent meaning. Even in ASCII, there was no
requirement that labels form words. Labels that could not possibly
represent words in any Romance or Germanic language (the languages
that have been written in "Latin" scripts since medieval times or
earlier) have actually been quite common. In general, in those
languages, words contain at least one vowel and do not have embedded
numbers. As a result, a string such as "bc345df" cannot possibly be
a "word" in these languages. More generally, the more one moves
toward "language"-based registry restrictions, the less it is going
to be possible to construct labels out of fanciful strings. While
fanciful strings are terrible candidates for "words", they may make
very good identifiers. To take a trivial example using only ASCII
characters, "rtr32w", "rtr32x", and "rtr32z" might be very good DNS
labels for a particular zone and application. However, given the
embedded digits and lack of vowels, they, like the "bc345df" example
given above, would fail even the most superficial of tests for valid
English (or German or French (etc.)) word forms.
It is worth noting that several DNS experts have suggested that a
number of problems could be solved by prohibiting meaningful names in
labels, requiring instead that the labels be random or nonsense
strings. If methods similar to those discussed in this document were
used to force identifiers to be closer to meaningful words in real
languages, the result would be directly contradictory to those
"random name" approaches.
Interestingly, if one were trying to develop an "only words" system,
a rather different -- but very restrictive -- model could be
developed using lookups in a dictionary for the relevant language and
a listing of valid business names for the relevant area. If a string
did not appear in either, it would not be permitted to be registered.
Models that require a prior national business listing (or
registration) that is identical to the proposed domain name label
have historically been used to restrict registrations in some
country-code top level domains, so this is not a new idea. On the
other hand, if look-alike characters are a concern, even that type of
rule (or restriction) would still not avoid the need to consider
character variants.
Consequently, registries applying the principles outlined in this
document should be careful not to apply more severe restrictions than
are reasonable and appropriate while, at the same time, being aware
of how difficult it usually is to add restrictions at a later time.
3. Possible Modifications of the JET Model
The JET model was designed for CJK characters. The discussion above
implies that some extensions to it may be needed to handle the
characteristics of various alphabetic scripts and the decisions that
might be made about them in different zones. Those extensions might
include facilities to process:
o Two-character (or more) sequences, such as ligatures and
typographic spelling conventions, as variants.
o Regular expressions or some other mechanism for dealing with
string positions of characters (e.g., characters that must, or
must not, appear at the beginning or end of strings).
o Delimiter breaks to permit multiple languages to be used,
separately, within the same label. E.g., is it possible to define
a label as consisting of two or more components, each in a
different language, with some particular delimiter to define the
boundaries of the components?
4. Conclusions and Recommendations About the General Approach
After examining the implications of the potential use of the full
range of characters permitted by IDNA in DNS labels, multiple groups,
including IESG [IESG-IDN] and ICANN [ICANN-IDN] [ICANN-IDN2], have
concluded that some restrictions are needed to prevent many forms of
user confusion about the actual structure of a name or the word,
phrase, or term that it appears to spell out. The best way to
approach such restrictions appears to draw from the language and
culture of the community of registrants and users in the relevant
zone: if particular characters are likely to be surprising or
unintelligible to both of those groups, it is probably wise to not
permit them to be used in registrations. Registration restrictions
can be carried much further than restricting permitted characters to
a selected Unicode subset. The idea of a reserved "bundle" of
related labels permits probably-confusing combinations or sets of
characters to be bound together, under the control of a single
registrant. While that registrant might still use the package in a
way that confused his or her own users (the approach outlined here
will not prevent either ill-though-out ideas or stupidity), the
possibility of turning potential confusion into a hostile attack
would be considerably reduced.
At the same time, excessive restrictions may make DNS identifiers
less useful for their original purpose: identifying particular hosts
and similar resources on the network in an orderly way. Registries
creating rules and policies about what can be registered in
particular zones -- whether those are based on the JET Guidelines or
the suggestions in this document -- should balance the need for
restrictions against the need for flexibility in constructing
identifiers.
The discussion above provides many options that could be selected,
defined, and applied in different ways in different registries
(zones). Registrars and registrants would almost certainly prefer
systems in which they can predict, at least to a first order
approximation, the implications of a particular potential
registration. Predictability of that sort probably requires more
standards, and less flexibility, than the model itself might suggest.
5. A Model Table Format
The format of the table is meant to be machine-readable but not
human-readable. It is fairly trivial to convert the table into one
that can be read by people.
Each character in the table is given in the "U+" notation for Unicode
characters. The lines of the table are terminated with either a
carriage return character (ASCII 0x0D), a linefeed character (ASCII
0x0A), or a sequence of carriage return followed by linefeed (ASCII
0x0D 0x0A). The order of the lines in the table may or may not
matter, depending on how the table is constructed.
Comment lines in the table are preceded with a "#" character (ASCII
0x2C).
Each non-comment line in the table starts with the character that is
allowed in the registry and expected to be used in registrations,
which is also called the "base character". If the base character has
any variants, the base character is followed by a vertical bar
character ("|", ASCII 0x7C) and the variant string. If the base
character has more than one variant, the variants are separated by a
colon (":", ASCII 0x3A). Strings are given with a hyphen ("-", ASCII
0x2D) between each character. Comments beginning with a "#" (ASCII
0x2C), and may be preceded by spaces (" ", ASCII 0x20).
The following is an example of how a table might look. The entries
in this table are purposely silly and should not be used by any
registry as the basis for choosing variants. For the example, assume
that the registry:
o allows the FOR ALL character (U+2200) with no variants
o allows the COMPLEMENT character (U+2201) which has a single
variant of LATIN CAPITAL LETTER C (U+0043)
o allows the PROPORTION character (U+2237) which has one variant
which is the string COLON (U+003A) COLON (U+003A)
o allows the PARTIAL DIFFERENTIAL character (U+2202) which has two
variants: LATIN SMALL LETTER D (U+0064) and GREEK SMALL LETTER
DELTA (U+03B4)
The table contents (after any required header information, see
[IANA-language-registry] and the discussion in Section 7 below) would
look like:
# An example of a table
U+2200
U+2201|U+0043
U+2237|U+003A-U+003A # Note that the variant is a string
U+2202|U+0064:U+03B4 # Two variants for the same character
Implementers of table processors should remember that there are tens
of thousands of characters whose codepoints are greater than 0xFFFF.
Thus, any program that assumes that each character in the table is
represented in exactly six octets ("U", "+", and four octets
representing the character value) will fail with tables that use
characters whose value is greater than 0xFFFF.
6. A Model Label Registration Procedure: "CreateBundle"
This procedure has three inputs:
1. the proposed base registration,