various faulty implementations of both User and Server
RFC1123 REMOTE LOGIN -- TELNET October 1989
Telnets.
Although the Telnet protocol is based on a perfectly
symmetric model, in a remote login session the role of the
user at a terminal differs from the role of the server
host. For example, RFC-854 defines the meaning of CR, LF,
and CR LF as output from the server, but does not specify
what the User Telnet should send when the user presses the
"end-of-line" key on the terminal; this turns out to be
the point at issue.
When a user presses the "end-of-line" key, some User
Telnet implementations send CR LF, while others send CR
NUL (based on a different interpretation of the same
sentence in RFC-854). These will be equivalent for a
correctly-implemented ASCII server host, as discussed
above. For other servers, a mode in the User Telnet is
needed.
The existence of User Telnets that send only CR NUL when
CR is pressed creates a dilemma for non-ASCII hosts: they
can either treat CR NUL as equivalent to CR LF in input,
thus precluding the possibility of entering a "bare" CR,
or else lose complete interworking.
Suppose a user on host A uses Telnet to log into a server
host B, and then execute B's User Telnet program to log
into server host C. It is desirable for the Server/User
Telnet combination on B to be as transparent as possible,
i.e., to appear as if A were connected directly to C. In
particular, correct implementation will make B transparent
to Telnet end-of-line sequences, except that CR LF may be
translated to CR NUL or vice versa.
IMPLEMENTATION:
To understand Telnet end-of-line issues, one must have at
least a general model of the relationship of Telnet to the
local operating system. The Server Telnet process is
typically coupled into the terminal driver software of the
operating system as a pseudo-terminal. A Telnet end-of-
line sequence received by the Server Telnet must have the
same effect as pressing the end-of-line key on a real
locally-connected terminal.
Operating systems that support interactive character-at-
a-time applications (e.g., editors) typically have two
internal modes for their terminal I/O: a formatted mode,
in which local conventions for end-of-line and other
RFC1123 REMOTE LOGIN -- TELNET October 1989
formatting rules have been applied to the data stream, and
a "raw" mode, in which the application has direct access
to every character as it was entered. A Server Telnet
must be implemented in such a way that these modes have
the same effect for remote as for local terminals. For
example, suppose a CR LF or CR NUL is received by the
Server Telnet on an ASCII host. In raw mode, a CR
character is passed to the application; in formatted mode,
the local system's end-of-line convention is used.
3.3.2 Data Entry Terminals
DISCUSSION:
In addition to the line-oriented and character-oriented
ASCII terminals for which Telnet was designed, there are
several families of video display terminals that are
sometimes known as "data entry terminals" or DETs. The
IBM 3270 family is a well-known example.
Two Internet protocols have been designed to support
generic DETs: SUPDUP [TELNET:16, TELNET:17], and the DET
option [TELNET:18, TELNET:19]. The DET option drives a
data entry terminal over a Telnet connection using (sub-)
negotiation. SUPDUP is a completely separate terminal
protocol, which can be entered from Telnet by negotiation.
Although both SUPDUP and the DET option have been used
successfully in particular environments, neither has
gained general acceptance or wide implementation.
A different approach to DET interaction has been developed
for supporting the IBM 3270 family through Telnet,
although the same approach would be applicable to any DET.
The idea is to enter a "native DET" mode, in which the
native DET input/output stream is sent as binary data.
The Telnet EOR command is used to delimit logical records
(e.g., "screens") within this binary stream.
IMPLEMENTATION:
The rules for entering and leaving native DET mode are as
follows:
o The Server uses the Terminal-Type option [TELNET:10]
to learn that the client is a DET.
o It is conventional, but not required, that both ends
negotiate the EOR option [TELNET:9].
o Both ends negotiate the Binary option [TELNET:3] to
RFC1123 REMOTE LOGIN -- TELNET October 1989
enter native DET mode.
o When either end negotiates out of binary mode, the
other end does too, and the mode then reverts to
normal NVT.
3.3.3 Option Requirements
Every Telnet implementation MUST support the Binary option
[TELNET:3] and the Suppress Go Ahead option [TELNET:5], and
SHOULD support the Echo [TELNET:4], Status [TELNET:6], End-of-
Record [TELNET:9], and Extended Options List [TELNET:8]
options.
A User or Server Telnet SHOULD support the Window Size Option
[TELNET:12] if the local operating system provides the
corresponding capability.
DISCUSSION:
Note that the End-of-Record option only signifies that a
Telnet can receive a Telnet EOR without crashing;
therefore, every Telnet ought to be willing to accept
negotiation of the End-of-Record option. See also the
discussion in Section 3.2.3.
3.3.4 Option Initiation
When the Telnet protocol is used in a client/server situation,
the server SHOULD initiate negotiation of the terminal
interaction mode it expects.
DISCUSSION:
The Telnet protocol was defined to be perfectly
symmetrical, but its application is generally asymmetric.
Remote login has been known to fail because NEITHER side
initiated negotiation of the required non-default terminal
modes. It is generally the server that determines the
preferred mode, so the server needs to initiate the
negotiation; since the negotiation is symmetric, the user
can also initiate it.
A client (User Telnet) SHOULD provide a means for users to
enable and disable the initiation of option negotiation.
DISCUSSION:
A user sometimes needs to connect to an application
service (e.g., FTP or SMTP) that uses Telnet for its
RFC1123 REMOTE LOGIN -- TELNET October 1989
control stream but does not support Telnet options. User
Telnet may be used for this purpose if initiation of
option negotiation is disabled.
3.3.5 Telnet Linemode Option
DISCUSSION:
An important new Telnet option, LINEMODE [TELNET:12], has
been proposed. The LINEMODE option provides a standard
way for a User Telnet and a Server Telnet to agree that
the client rather than the server will perform terminal
character processing. When the client has prepared a
complete line of text, it will send it to the server in
(usually) one TCP packet. This option will greatly
decrease the packet cost of Telnet sessions and will also
give much better user response over congested or long-
delay networks.
The LINEMODE option allows dynamic switching between local
and remote character processing. For example, the Telnet
connection will automatically negotiate into single-
character mode while a full screen editor is running, and
then return to linemode when the editor is finished.
We expect that when this RFCis released, hosts should
implement the client side of this option, and may
implement the server side of this option. To properly
implement the server side, the server needs to be able to
tell the local system not to do any input character
processing, but to remember its current terminal state and
notify the Server Telnet process whenever the state
changes. This will allow password echoing and full screen
editors to be handled properly, for example.
3.4 TELNET/USER INTERFACE
3.4.1 Character Set Transparency
User Telnet implementations SHOULD be able to send or receive
any 7-bit ASCII character. Where possible, any special
character interpretations by the user host's operating system
SHOULD be bypassed so that these characters can conveniently be
sent and received on the connection.
Some character value MUST be reserved as "escape to command
mode"; conventionally, doubling this character allows it to be
entered as data. The specific character used SHOULD be user
selectable.
RFC1123 REMOTE LOGIN -- TELNET October 1989
On binary-mode connections, a User Telnet program MAY provide
an escape mechanism for entering arbitrary 8-bit values, if the
host operating system doesn't allow them to be entered directly
from the keyboard.
IMPLEMENTATION:
The transparency issues are less pressing on servers, but
implementors should take care in dealing with issues like:
masking off parity bits (sent by an older, non-conforming
client) before they reach programs that expect only NVT
ASCII, and properly handling programs that request 8-bit
data streams.
3.4.2 Telnet Commands
A User Telnet program MUST provide a user the capability of
entering any of the Telnet control functions IP, AO, or AYT,
and SHOULD provide the capability of entering EC, EL, and
Break.
3.4.3 TCP Connection Errors
A User Telnet program SHOULD report to the user any TCP errors
that are reported by the transport layer (see "TCP/Application
Layer Interface" section in [INTRO:1]).
3.4.4 Non-Default Telnet Contact Port
A User Telnet program SHOULD allow the user to optionally
specify a non-standard contact port number at the Server Telnet
host.
3.4.5 Flushing Output
A User Telnet program SHOULD provide the user the ability to
specify whether or not output should be flushed when an IP is
sent; see Section 3.2.4.
For any output flushing scheme that causes the User Telnet to
flush output locally until a Telnet signal is received from the
Server, there SHOULD be a way for the user to manually restore
normal output, in case the Server fails to send the expected
signal.
RFC1123 REMOTE LOGIN -- TELNET October 1989
3.5. TELNET REQUIREMENTS SUMMARY
| | | | |S| |
| | | | |H| |F
| | | | |O|M|o
| | |S| |U|U|o
| | |H| |L|S|t
| |M|O| |D|T|n
| |U|U|M| | |o
| |S|L|A|N|N|t
| |T|D|Y|O|O|t
FEATURE |SECTION | | | |T|T|e
-------------------------------------------------|--------|-|-|-|-|-|--
| | | | | | |
Option Negotiation |3.2.1 |x| | | | |
Avoid negotiation loops |3.2.1 |x| | | | |
Refuse unsupported options |3.2.1 |x| | | | |
Negotiation OK anytime on connection |3.2.1 | |x| | | |
Default to NVT |3.2.1 |x| | | | |
Send official name in Term-Type option |3.2.8 |x| | | | |
Accept any name in Term-Type option |3.2.8 |x| | | | |
Implement Binary, Suppress-GA options |3.3.3 |x| | | | |
Echo, Status, EOL, Ext-Opt-List options |3.3.3 | |x| | | |
Implement Window-Size option if appropriate |3.3.3 | |x| | | |
Server initiate mode negotiations |3.3.4 | |x| | | |
User can enable/disable init negotiations |3.3.4 | |x| | | |
| | | | | | |
Go-Aheads | | | | | | |
Non-GA server negotiate SUPPRESS-GA option |3.2.2 |x| | | | |
User or Server accept SUPPRESS-GA option |3.2.2 |x| | | | |
User Telnet ignore GA's |3.2.2 | | |x| | |
| | | | | | |
Control Functions | | | | | | |
Support SE NOP DM IP AO AYT SB |3.2.3 |x| | | | |
Support EOR EC EL Break |3.2.3 | | |x| | |
Ignore unsupported control functions |3.2.3 |x| | | | |
User, Server discard urgent data up to DM |3.2.4 |x| | | | |
User Telnet send "Synch" after IP, AO, AYT |3.2.4 | |x| | | |
Server Telnet reply Synch to IP |3.2.4 | | |x| | |
Server Telnet reply Synch to AO |3.2.4 |x| | | | |
User Telnet can flush output when send IP |3.2.4 | |x| | | |
| | | | | | |
Encoding | | | | | | |
Send high-order bit in NVT mode |3.2.5 | | | |x| |
Send high-order bit as parity bit |3.2.5 | | | | |x|
Negot. BINARY if pass high-ord. bit to applic |3.2.5 | |x| | | |
Always double IAC data byte |3.2.6 |x| | | | |
RFC1123 REMOTE LOGIN -- TELNET October 1989
Double IAC data byte in binary mode |3.2.7 |x| | | | |
Obey Telnet cmds in binary mode |3.2.7 |x| | | | |
End-of-line, CR NUL in binary mode |3.2.7 | | | | |x|
| | | | | | |
End-of-Line | | | | | | |
EOL at Server same as local end-of-line |3.3.1 |x| | | | |
ASCII Server accept CR LF or CR NUL for EOL |3.3.1 |x| | | | |
User Telnet able to send CR LF, CR NUL, or LF |3.3.1 |x| | | | |
ASCII user able to select CR LF/CR NUL |3.3.1 | |x| | | |
User Telnet default mode is CR LF |3.3.1 | |x| | | |
Non-interactive uses CR LF for EOL |3.3.1 |x| | | | |
| | | | | | |
User Telnet interface | | | | | | |
Input & output all 7-bit characters |3.4.1 | |x| | | |
Bypass local op sys interpretation |3.4.1 | |x| | | |
Escape character |3.4.1 |x| | | | |
User-settable escape character |3.4.1 | |x| | | |
Escape to enter 8-bit values |3.4.1 | | |x| | |
Can input IP, AO, AYT |3.4.2 |x| | | | |
Can input EC, EL, Break |3.4.2 | |x| | | |
Report TCP connection errors to user |3.4.3 | |x| | | |
Optional non-default contact port |3.4.4 | |x| | | |
Can spec: output flushed when IP sent |3.4.5 | |x| | | |
Can manually restore output mode |3.4.5 | |x| | | |
| | | | | | |
RFC1123 FILE TRANSFER -- FTP October 1989
4. FILE TRANSFER
4.1 FILE TRANSFER PROTOCOL -- FTP
4.1.1 INTRODUCTION
The File Transfer Protocol FTP is the primary Internet standard
for file transfer. The current specification is contained in
RFC-959 [FTP:1].
FTP uses separate simultaneous TCP connections for control and
for data transfer. The FTP protocol includes many features,
some of which are not commonly implemented. However, for every
feature in FTP, there exists at least one implementation. The
minimum implementation defined in RFC-959 was too small, so a
somewhat larger minimum implementation is defined here.
Internet users have been unnecessarily burdened for years by
deficient FTP implementations. Protocol implementors have
suffered from the erroneous opinion that implementing FTP ought
to be a small and trivial task. This is wrong, because FTP has
a user interface, because it has to deal (correctly) with the
whole variety of communication and operating system errors that
may occur, and because it has to handle the great diversity of
real file systems in the world.
4.1.2. PROTOCOL WALK-THROUGH
4.1.2.1 LOCAL Type: RFC-959 Section 3.1.1.4
An FTP program MUST support TYPE I ("IMAGE" or binary type)
as well as TYPE L 8 ("LOCAL" type with logical byte size 8).
A machine whose memory is organized into m-bit words, where
m is not a multiple of 8, MAY also support TYPE L m.
DISCUSSION:
The command "TYPE L 8" is often required to transfer
binary data between a machine whose memory is organized
into (e.g.) 36-bit words and a machine with an 8-bit
byte organization. For an 8-bit byte machine, TYPE L 8
is equivalent to IMAGE.
"TYPE L m" is sometimes specified to the FTP programs
on two m-bit word machines to ensure the correct
transfer of a native-mode binary file from one machine
to the other. However, this command should have the
same effect on these machines as "TYPE I".
RFC1123 FILE TRANSFER -- FTP October 1989
4.1.2.2 Telnet Format Control: RFC-959 Section 3.1.1.5.2
A host that makes no distinction between TYPE N and TYPE T
SHOULD implement TYPE T to be identical to TYPE N.
DISCUSSION:
This provision should ease interoperation with hosts
that do make this distinction.
Many hosts represent text files internally as strings
of ASCII characters, using the embedded ASCII format
effector characters (LF, BS, FF, ...) to control the
format when a file is printed. For such hosts, there
is no distinction between "print" files and other
files. However, systems that use record structured
files typically need a special format for printable
files (e.g., ASA carriage control). For the latter
hosts, FTP allows a choice of TYPE N or TYPE T.
4.1.2.3 Page Structure: RFC-959 Section 3.1.2.3 and Appendix I
Implementation of page structure is NOT RECOMMENDED in
general. However, if a host system does need to implement
FTP for "random access" or "holey" files, it MUST use the
defined page structure format rather than define a new
private FTP format.
4.1.2.4 Data Structure Transformations: RFC-959 Section 3.1.2
An FTP transformation between record-structure and file-
structure SHOULD be invertible, to the extent possible while
making the result useful on the target host.
DISCUSSION:
RFC-959 required strict invertibility between record-
structure and file-structure, but in practice,
efficiency and convenience often preclude it.
Therefore, the requirement is being relaxed. There are
two different objectives for transferring a file:
processing it on the target host, or just storage. For
storage, strict invertibility is important. For
processing, the file created on the target host needs
to be in the format expected by application programs on
that host.
As an example of the conflict, imagine a record-
oriented operating system that requires some data files
to have exactly 80 bytes in each record. While STORing
RFC1123 FILE TRANSFER -- FTP October 1989
a file on such a host, an FTP Server must be able to
pad each line or record to 80 bytes; a later retrieval
of such a file cannot be strictly invertible.
4.1.2.5 Data Connection Management: RFC-959 Section 3.3
A User-FTP that uses STREAM mode SHOULD send a PORT command
to assign a non-default data port before each transfer
command is issued.
DISCUSSION:
This is required because of the long delay after a TCP
connection is closed until its socket pair can be
reused, to allow multiple transfers during a single FTP
session. Sending a port command can avoided if a
transfer mode other than stream is used, by leaving the
data transfer connection open between transfers.
4.1.2.6 PASV Command: RFC-959 Section 4.1.2
A server-FTP MUST implement the PASV command.
If multiple third-party transfers are to be executed during
the same session, a new PASV command MUST be issued before
each transfer command, to obtain a unique port pair.
IMPLEMENTATION:
The format of the 227 reply to a PASV command is not
well standardized. In particular, an FTP client cannot
assume that the parentheses shown on page 40 of RFC-959
will be present (and in fact, Figure 3 on page 43 omits
them). Therefore, a User-FTP program that interprets
the PASV reply must scan the reply for the first digit
of the host and port numbers.
Note that the host number h1,h2,h3,h4 is the IP address
of the server host that is sending the reply, and that
p1,p2 is a non-default data transfer port that PASV has
assigned.
4.1.2.7 LIST and NLST Commands: RFC-959 Section 4.1.3
The data returned by an NLST command MUST contain only a
simple list of legal pathnames, such that the server can use
them directly as the arguments of subsequent data transfer
commands for the individual files.
The data returned by a LIST or NLST command SHOULD use an
RFC1123 FILE TRANSFER -- FTP October 1989
implied TYPE AN, unless the current type is EBCDIC, in which
case an implied TYPE EN SHOULD be used.
DISCUSSION:
Many FTP clients support macro-commands that will get
or put files matching a wildcard specification, using
NLST to obtain a list of pathnames. The expansion of
"multiple-put" is local to the client, but "multiple-
get" requires cooperation by the server.
The implied type for LIST and NLST is designed to
provide compatibility with existing User-FTPs, and in
particular with multiple-get commands.
4.1.2.8 SITE Command: RFC-959 Section 4.1.3
A Server-FTP SHOULD use the SITE command for non-standard
features, rather than invent new private commands or
unstandardized extensions to existing commands.
4.1.2.9 STOU Command: RFC-959 Section 4.1.3
The STOU command stores into a uniquely named file. When it
receives an STOU command, a Server-FTP MUST return the
actual file name in the "125 Transfer Starting" or the "150
Opening Data Connection" message that precedes the transfer
(the 250 reply code mentioned in RFC-959 is incorrect). The
exact format of these messages is hereby defined to be as
follows:
125 FILE: pppp
150 FILE: pppp
where pppp represents the unique pathname of the file that
will be written.
4.1.2.10 Telnet End-of-line Code: RFC-959, Page 34
Implementors MUST NOT assume any correspondence between READ
boundaries on the control connection and the Telnet EOL
sequences (CR LF).
DISCUSSION:
Thus, a server-FTP (or User-FTP) must continue reading
characters from the control connection until a complete
Telnet EOL sequence is encountered, before processing
the command (or response, respectively). Conversely, a
single READ from the control connection may include
RFC1123 FILE TRANSFER -- FTP October 1989
more than one FTP command.
4.1.2.11 FTP Replies: RFC-959 Section 4.2, Page 35
A Server-FTP MUST send only correctly formatted replies on
the control connection. Note that RFC-959 (unlike earlier
versions of the FTP spec) contains no provision for a
"spontaneous" reply message.
A Server-FTP SHOULD use the reply codes defined in RFC-959
whenever they apply. However, a server-FTP MAY use a
different reply code when needed, as long as the general
rules of Section 4.2 are followed. When the implementor has
a choice between a 4xx and 5xx reply code, a Server-FTP
SHOULD send a 4xx (temporary failure) code when there is any
reasonable possibility that a failed FTP will succeed a few
hours later.
A User-FTP SHOULD generally use only the highest-order digit
of a 3-digit reply code for making a procedural decision, to
prevent difficulties when a Server-FTP uses non-standard
reply codes.
A User-FTP MUST be able to handle multi-line replies. If
the implementation imposes a limit on the number of lines
and if this limit is exceeded, the User-FTP MUST recover,
e.g., by ignoring the excess lines until the end of the
multi-line reply is reached.
A User-FTP SHOULD NOT interpret a 421 reply code ("Service
not available, closing control connection") specially, but
SHOULD detect closing of the control connection by the
server.
DISCUSSION:
Server implementations that fail to strictly follow the
reply rules often cause FTP user programs to hang.
Note that RFC-959 resolved ambiguities in the reply
rules found in earlier FTP specifications and must be
followed.
It is important to choose FTP reply codes that properly
distinguish between temporary and permanent failures,
to allow the successful use of file transfer client
daemons. These programs depend on the reply codes to
decide whether or not to retry a failed transfer; using
a permanent failure code (5xx) for a temporary error
will cause these programs to give up unnecessarily.
RFC1123 FILE TRANSFER -- FTP October 1989
When the meaning of a reply matches exactly the text
shown in RFC-959, uniformity will be enhanced by using
the RFC-959 text verbatim. However, a Server-FTP
implementor is encouraged to choose reply text that
conveys specific system-dependent information, when
appropriate.
4.1.2.12 Connections: RFC-959 Section 5.2
The words "and the port used" in the second paragraph of
this section of RFC-959 are erroneous (historical), and they
should be ignored.
On a multihomed server host, the default data transfer port
(L-1) MUST be associated with the same local IP address as
the corresponding control connection to port L.
A user-FTP MUST NOT send any Telnet controls other than
SYNCH and IP on an FTP control connection. In particular, it
MUST NOT attempt to negotiate Telnet options on the control
connection. However, a server-FTP MUST be capable of
accepting and refusing Telnet negotiations (i.e., sending
DONT/WONT).
DISCUSSION:
Although the RFCsays: "Server- and User- processes
should follow the conventions for the Telnet
protocol...[on the control connection]", it is not the
intent that Telnet option negotiation is to be
employed.
4.1.2.13 Minimum Implementation; RFC-959 Section 5.1
The following commands and options MUST be supported by
every server-FTP and user-FTP, except in cases where the
underlying file system or operating system does not allow or
support a particular command.
Type: ASCII Non-print, IMAGE, LOCAL 8
Mode: Stream
Structure: File, Record*
Commands:
USER, PASS, ACCT,
PORT, PASV,
TYPE, MODE, STRU,
RETR, STOR, APPE,
RNFR, RNTO, DELE,
CWD, CDUP, RMD, MKD, PWD,
RFC1123 FILE TRANSFER -- FTP October 1989
LIST, NLST,
SYST, STAT,
HELP, NOOP, QUIT.
*Record structure is REQUIRED only for hosts whose file
systems support record structure.
DISCUSSION:
Vendors are encouraged to implement a larger subset of
the protocol. For example, there are important
robustness features in the protocol (e.g., Restart,
ABOR, block mode) that would be an aid to some Internet
users but are not widely implemented.
A host that does not have record structures in its file
system may still accept files with STRU R, recording
the byte stream literally.
4.1.3 SPECIFIC ISSUES
4.1.3.1 Non-standard Command Verbs
FTP allows "experimental" commands, whose names begin with
"X". If these commands are subsequently adopted as
standards, there may still be existing implementations using
the "X" form. At present, this is true for the directory
commands:
RFC-959 "Experimental"
MKD XMKD
RMD XRMD
PWD XPWD
CDUP XCUP
CWD XCWD
All FTP implementations SHOULD recognize both forms of these
commands, by simply equating them with extra entries in the
command lookup table.
IMPLEMENTATION:
A User-FTP can access a server that supports only the
"X" forms by implementing a mode switch, or
automatically using the following procedure: if the
RFC-959 form of one of the above commands is rejected
with a 500 or 502 response code, then try the
experimental form; any other response would be passed
to the user.
RFC1123 FILE TRANSFER -- FTP October 1989
4.1.3.2 Idle Timeout
A Server-FTP process SHOULD have an idle timeout, which will
terminate the process and close the control connection if
the server is inactive (i.e., no command or data transfer in
progress) for a long period of time. The idle timeout time
SHOULD be configurable, and the default should be at least 5
minutes.
A client FTP process ("User-PI" in RFC-959) will need
timeouts on responses only if it is invoked from a program.
DISCUSSION:
Without a timeout, a Server-FTP process may be left
pending indefinitely if the corresponding client
crashes without closing the control connection.
4.1.3.3 Concurrency of Data and Control
DISCUSSION:
The intent of the designers of FTP was that a user
should be able to send a STAT command at any time while
data transfer was in progress and that the server-FTP
would reply immediately with status -- e.g., the number
of bytes transferred so far. Similarly, an ABOR
command should be possible at any time during a data
transfer.
Unfortunately, some small-machine operating systems
make such concurrent programming difficult, and some
other implementers seek minimal solutions, so some FTP
implementations do not allow concurrent use of the data
and control connections. Even such a minimal server
must be prepared to accept and defer a STAT or ABOR
command that arrives during data transfer.
4.1.3.4 FTP Restart Mechanism
The description of the 110 reply on pp. 40-41 of RFC-959 is
incorrect; the correct description is as follows. A restart
reply message, sent over the control connection from the
receiving FTP to the User-FTP, has the format:
110 MARK ssss = rrrr
Here:
* ssss is a text string that appeared in a Restart Marker
RFC1123 FILE TRANSFER -- FTP October 1989
in the data stream and encodes a position in the
sender's file system;
* rrrr encodes the corresponding position in the
receiver's file system.
The encoding, which is specific to a particular file system
and network implementation, is always generated and
interpreted by the same system, either sender or receiver.
When an FTP that implements restart receives a Restart
Marker in the data stream, it SHOULD force the data to that
point to be written to stable storage before encoding the
corresponding position rrrr. An FTP sending Restart Markers
MUST NOT assume that 110 replies will be returned
synchronously with the data, i.e., it must not await a 110
reply before sending more data.
Two new reply codes are hereby defined for errors
encountered in restarting a transfer:
554 Requested action not taken: invalid REST parameter.
A 554 reply may result from a FTP service command that
follows a REST command. The reply indicates that the
existing file at the Server-FTP cannot be repositioned
as specified in the REST.
555 Requested action not taken: type or stru mismatch.
A 555 reply may result from an APPE command or from any
FTP service command following a REST command. The
reply indicates that there is some mismatch between the
current transfer parameters (type and stru) and the
attributes of the existing file.
DISCUSSION:
Note that the FTP Restart mechanism requires that Block
or Compressed mode be used for data transfer, to allow
the Restart Markers to be included within the data
stream. The frequency of Restart Markers can be low.
Restart Markers mark a place in the data stream, but
the receiver may be performing some transformation on
the data as it is stored into stable storage. In
general, the receiver's encoding must include any state
information necessary to restart this transformation at
any point of the FTP data stream. For example, in TYPE
RFC1123 FILE TRANSFER -- FTP October 1989
A transfers, some receiver hosts transform CR LF
sequences into a single LF character on disk. If a
Restart Marker happens to fall between CR and LF, the
receiver must encode in rrrr that the transfer must be
restarted in a "CR has been seen and discarded" state.
Note that the Restart Marker is required to be encoded
as a string of printable ASCII characters, regardless
of the type of the data.
RFC-959 says that restart information is to be returned
"to the user". This should not be taken literally. In
general, the User-FTP should save the restart
information (ssss,rrrr) in stable storage, e.g., append
it to a restart control file. An empty restart control
file should be created when the transfer first starts
and deleted automatically when the transfer completes
successfully. It is suggested that this file have a
name derived in an easily-identifiable manner from the
name of the file being transferred and the remote host
name; this is analogous to the means used by many text
editors for naming "backup" files.
There are three cases for FTP restart.
(1) User-to-Server Transfer
The User-FTP puts Restart Markers <ssss> at
convenient places in the data stream. When the
Server-FTP receives a Marker, it writes all prior
data to disk, encodes its file system position and
transformation state as rrrr, and returns a "110
MARK ssss = rrrr" reply over the control
connection. The User-FTP appends the pair
(ssss,rrrr) to its restart control file.
To restart the transfer, the User-FTP fetches the
last (ssss,rrrr) pair from the restart control
file, repositions its local file system and
transformation state using ssss, and sends the
command "REST rrrr" to the Server-FTP.
(2) Server-to-User Transfer
The Server-FTP puts Restart Markers <ssss> at
convenient places in the data stream. When the
User-FTP receives a Marker, it writes all prior
data to disk, encodes its file system position and
RFC1123 FILE TRANSFER -- FTP October 1989
transformation state as rrrr, and appends the pair
(rrrr,ssss) to its restart control file.
To restart the transfer, the User-FTP fetches the
last (rrrr,ssss) pair from the restart control
file, repositions its local file system and
transformation state using rrrr, and sends the
command "REST ssss" to the Server-FTP.
(3) Server-to-Server ("Third-Party") Transfer
The sending Server-FTP puts Restart Markers <ssss>
at convenient places in the data stream. When it
receives a Marker, the receiving Server-FTP writes
all prior data to disk, encodes its file system
position and transformation state as rrrr, and
sends a "110 MARK ssss = rrrr" reply over the
control connection to the User. The User-FTP
appends the pair (ssss,rrrr) to its restart
control file.
To restart the transfer, the User-FTP fetches the
last (ssss,rrrr) pair from the restart control
file, sends "REST ssss" to the sending Server-FTP,
and sends "REST rrrr" to the receiving Server-FTP.
4.1.4 FTP/USER INTERFACE
This section discusses the user interface for a User-FTP
program.
4.1.4.1 Pathname Specification
Since FTP is intended for use in a heterogeneous
environment, User-FTP implementations MUST support remote
pathnames as arbitrary character strings, so that their form
and content are not limited by the conventions of the local
operating system.
DISCUSSION:
In particular, remote pathnames can be of arbitrary
length, and all the printing ASCII characters as well
as space (0x20) must be allowed. RFC-959 allows a
pathname to contain any 7-bit ASCII character except CR
or LF.
RFC1123 FILE TRANSFER -- FTP October 1989
4.1.4.2 "QUOTE" Command
A User-FTP program MUST implement a "QUOTE" command that
will pass an arbitrary character string to the server and
display all resulting response messages to the user.
To make the "QUOTE" command useful, a User-FTP SHOULD send
transfer control commands to the server as the user enters
them, rather than saving all the commands and sending them
to the server only when a data transfer is started.
DISCUSSION:
The "QUOTE" command is essential to allow the user to
access servers that require system-specific commands
(e.g., SITE or ALLO), or to invoke new or optional
features that are not implemented by the User-FTP. For
example, "QUOTE" may be used to specify "TYPE A T" to
send a print file to hosts that require the
distinction, even if the User-FTP does not recognize
that TYPE.
4.1.4.3 Displaying Replies to User
A User-FTP SHOULD display to the user the full text of all
error reply messages it receives. It SHOULD have a
"verbose" mode in which all commands it sends and the full
text and reply codes it receives are displayed, for
diagnosis of problems.
4.1.4.4 Maintaining Synchronization
The state machine in a User-FTP SHOULD be forgiving of
missing and unexpected reply messages, in order to maintain
command synchronization with the server.
RFC1123 FILE TRANSFER -- FTP October 1989
4.1.5 FTP REQUIREMENTS SUMMARY
| | | | |S| |
| | | | |H| |F
| | | | |O|M|o
| | |S| |U|U|o
| | |H| |L|S|t
| |M|O| |D|T|n
| |U|U|M| | |o
| |S|L|A|N|N|t
| |T|D|Y|O|O|t
FEATURE |SECTION | | | |T|T|e
-------------------------------------------|---------------|-|-|-|-|-|--
Implement TYPE T if same as TYPE N |4.1.2.2 | |x| | | |
File/Record transform invertible if poss. |4.1.2.4 | |x| | | |
User-FTP send PORT cmd for stream mode |4.1.2.5 | |x| | | |
Server-FTP implement PASV |4.1.2.6 |x| | | | |
PASV is per-transfer |4.1.2.6 |x| | | | |
NLST reply usable in RETR cmds |4.1.2.7 |x| | | | |
Implied type for LIST and NLST |4.1.2.7 | |x| | | |
SITE cmd for non-standard features |4.1.2.8 | |x| | | |
STOU cmd return pathname as specified |4.1.2.9 |x| | | | |
Use TCP READ boundaries on control conn. |4.1.2.10 | | | | |x|
| | | | | | |
Server-FTP send only correct reply format |4.1.2.11 |x| | | | |
Server-FTP use defined reply code if poss. |4.1.2.11 | |x| | | |
New reply code following Section 4.2 |4.1.2.11 | | |x| | |
User-FTP use only high digit of reply |4.1.2.11 | |x| | | |
User-FTP handle multi-line reply lines |4.1.2.11 |x| | | | |
User-FTP handle 421 reply specially |4.1.2.11 | | | |x| |
| | | | | | |
Default data port same IP addr as ctl conn |4.1.2.12 |x| | | | |
User-FTP send Telnet cmds exc. SYNCH, IP |4.1.2.12 | | | | |x|
User-FTP negotiate Telnet options |4.1.2.12 | | | | |x|
Server-FTP handle Telnet options |4.1.2.12 |x| | | | |
Handle "Experimental" directory cmds |4.1.3.1 | |x| | | |
Idle timeout in server-FTP |4.1.3.2 | |x| | | |
Configurable idle timeout |4.1.3.2 | |x| | | |
Receiver checkpoint data at Restart Marker |4.1.3.4 | |x| | | |
Sender assume 110 replies are synchronous |4.1.3.4 | | | | |x|
| | | | | | |
Support TYPE: | | | | | | |
ASCII - Non-Print (AN) |4.1.2.13 |x| | | | |
ASCII - Telnet (AT) -- if same as AN |4.1.2.2 | |x| | | |
ASCII - Carriage Control (AC) |959 3.1.1.5.2 | | |x| | |
EBCDIC - (any form) |959 3.1.1.2 | | |x| | |
IMAGE |4.1.2.1 |x| | | | |
LOCAL 8 |4.1.2.1 |x| | | | |
RFC1123 FILE TRANSFER -- FTP October 1989
LOCAL m |4.1.2.1 | | |x| | |2
| | | | | | |
Support MODE: | | | | | | |
Stream |4.1.2.13 |x| | | | |
Block |959 3.4.2 | | |x| | |
| | | | | | |
Support STRUCTURE: | | | | | | |
File |4.1.2.13 |x| | | | |
Record |4.1.2.13 |x| | | | |3
Page |4.1.2.3 | | | |x| |
| | | | | | |
Support commands: | | | | | | |
USER |4.1.2.13 |x| | | | |
PASS |4.1.2.13 |x| | | | |
ACCT |4.1.2.13 |x| | | | |
CWD |4.1.2.13 |x| | | | |
CDUP |4.1.2.13 |x| | | | |
SMNT |959 5.3.1 | | |x| | |
REIN |959 5.3.1 | | |x| | |
QUIT |4.1.2.13 |x| | | | |
| | | | | | |
PORT |4.1.2.13 |x| | | | |
PASV |4.1.2.6 |x| | | | |
TYPE |4.1.2.13 |x| | | | |1
STRU |4.1.2.13 |x| | | | |1
MODE |4.1.2.13 |x| | | | |1
| | | | | | |
RETR |4.1.2.13 |x| | | | |
STOR |4.1.2.13 |x| | | | |
STOU |959 5.3.1 | | |x| | |
APPE |4.1.2.13 |x| | | | |
ALLO |959 5.3.1 | | |x| | |
REST |959 5.3.1 | | |x| | |
RNFR |4.1.2.13 |x| | | | |
RNTO |4.1.2.13 |x| | | | |
ABOR |959 5.3.1 | | |x| | |
DELE |4.1.2.13 |x| | | | |
RMD |4.1.2.13 |x| | | | |
MKD |4.1.2.13 |x| | | | |
PWD |4.1.2.13 |x| | | | |
LIST |4.1.2.13 |x| | | | |
NLST |4.1.2.13 |x| | | | |
SITE |4.1.2.8 | | |x| | |
STAT |4.1.2.13 |x| | | | |
SYST |4.1.2.13 |x| | | | |
HELP |4.1.2.13 |x| | | | |
NOOP |4.1.2.13 |x| | | | |
| | | | | | |
RFC1123 FILE TRANSFER -- FTP October 1989
User Interface: | | | | | | |
Arbitrary pathnames |4.1.4.1 |x| | | | |
Implement "QUOTE" command |4.1.4.2 |x| | | | |
Transfer control commands immediately |4.1.4.2 | |x| | | |
Display error messages to user |4.1.4.3 | |x| | | |
Verbose mode |4.1.4.3 | |x| | | |
Maintain synchronization with server |4.1.4.4 | |x| | | |
Footnotes:
(1) For the values shown earlier.
(2) Here m is number of bits in a memory word.
(3) Required for host with record-structured file system, optional
otherwise.
RFC1123 FILE TRANSFER -- TFTP October 1989
4.2 TRIVIAL FILE TRANSFER PROTOCOL -- TFTP
4.2.1 INTRODUCTION
The Trivial File Transfer Protocol TFTP is defined in RFC-783
[TFTP:1].
TFTP provides its own reliable delivery with UDP as its
transport protocol, using a simple stop-and-wait acknowledgment
system. Since TFTP has an effective window of only one 512
octet segment, it can provide good performance only over paths
that have a small delay*bandwidth product. The TFTP file
interface is very simple, providing no access control or
security.
TFTP's most important application is bootstrapping a host over
a local network, since it is simple and small enough to be
easily implemented in EPROM [BOOT:1, BOOT:2]. Vendors are
urged to support TFTP for booting.
4.2.2 PROTOCOL WALK-THROUGH
The TFTP specification [TFTP:1] is written in an open style,
and does not fully specify many parts of the protocol.
4.2.2.1 Transfer Modes: RFC-783, Page 3
The transfer mode "mail" SHOULD NOT be supported.
4.2.2.2 UDP Header: RFC-783, Page 17
The Length field of a UDP header is incorrectly defined; it
includes the UDP header length (8).
4.2.3 SPECIFIC ISSUES
4.2.3.1 Sorcerer's Apprentice Syndrome
There is a serious bug, known as the "Sorcerer's Apprentice
Syndrome," in the protocol specification. While it does not
cause incorrect operation of the transfer (the file will
always be transferred correctly if the transfer completes),
this bug may cause excessive retransmission, which may cause
the transfer to time out.
Implementations MUST contain the fix for this problem: the
sender (i.e., the side originating the DATA packets) must
never resend the current DATA packet on receipt of a
RFC1123 FILE TRANSFER -- TFTP October 1989
duplicate ACK.
DISCUSSION:
The bug is caused by the protocol rule that either
side, on receiving an old duplicate datagram, may
resend the current datagram. If a packet is delayed in
the network but later successfully delivered after
either side has timed out and retransmitted a packet, a
duplicate copy of the response may be generated. If
the other side responds to this duplicate with a
duplicate of its own, then every datagram will be sent
in duplicate for the remainder of the transfer (unless
a datagram is lost, breaking the repetition). Worse
yet, since the delay is often caused by congestion,
this duplicate transmission will usually causes more
congestion, leading to more delayed packets, etc.
The following example may help to clarify this problem.
TFTP A TFTP B
(1) Receive ACK X-1
Send DATA X
(2) Receive DATA X
Send ACK X
(ACK X is delayed in network,
and A times out):
(3) Retransmit DATA X
(4) Receive DATA X again
Send ACK X again
(5) Receive (delayed) ACK X
Send DATA X+1
(6) Receive DATA X+1
Send ACK X+1
(7) Receive ACK X again
Send DATA X+1 again
(8) Receive DATA X+1 again
Send ACK X+1 again
(9) Receive ACK X+1
Send DATA X+2
(10) Receive DATA X+2
Send ACK X+3
(11) Receive ACK X+1 again
Send DATA X+2 again
(12) Receive DATA X+2 again
Send ACK X+3 again
RFC1123 FILE TRANSFER -- TFTP October 1989
Notice that once the delayed ACK arrives, the protocol
settles down to duplicate all further packets
(sequences 5-8 and 9-12). The problem is caused not by
either side timing out, but by both sides
retransmitting the current packet when they receive a
duplicate.
The fix is to break the retransmission loop, as
indicated above. This is analogous to the behavior of
TCP. It is then possible to remove the retransmission
timer on the receiver, since the resent ACK will never
cause any action; this is a useful simplification where
TFTP is used in a bootstrap program. It is OK to allow
the timer to remain, and it may be helpful if the
retransmitted ACK replaces one that was genuinely lost
in the network. The sender still requires a retransmit
timer, of course.
4.2.3.2 Timeout Algorithms
A TFTP implementation MUST use an adaptive timeout.
IMPLEMENTATION:
TCP retransmission algorithms provide a useful base to
work from. At least an exponential backoff of
retransmission timeout is necessary.
4.2.3.3 Extensions
A variety of non-standard extensions have been made to TFTP,
including additional transfer modes and a secure operation
mode (with passwords). None of these have been
standardized.
4.2.3.4 Access Control