4.2 Server/client relationship
The NFS version 3 protocol is designed to allow servers to be
as simple and general as possible. Sometimes the simplicity
of the server can be a problem, if the client implements
complicated file system semantics.
For example, some operating systems allow removal of open
files. A process can open a file and, while it is open,
remove it from the directory. The file can be read and
written as long as the process keeps it open, even though the
file has no name in the file system. It is impossible for a
stateless server to implement these semantics. The client
can do some tricks such as renaming the file on remove (to a
hidden name), and only physically deleting it on close. The
NFS version 3 protocol provides sufficient functionality to
implement most file system semantics on a client.
Every NFS version 3 protocol client can also potentially be a
server, and remote and local mounted file systems can be
freely mixed. This leads to some problems when a client
travels down the directory tree of a remote file system and
reaches the mount point on the server for another remote file
system. Allowing the server to follow the second remote mount
would require loop detection, server lookup, and user
revalidation. Instead, both NFS version 2 protocol and NFS
version 3 protocol implementations do not typically let
clients cross a server's mount point. When a client does a
LOOKUP on a directory on which the server has mounted a file
system, the client sees the underlying directory instead of
the mounted directory.
For example, if a server has a file system called /usr and
mounts another file system on /usr/src, if a client mounts
/usr, it does not see the mounted version of /usr/src. A
client could do remote mounts that match the server's mount
points to maintain the server's view. In this example, the
client would also have to mount /usr/src in addition to /usr,
even if they are from the same server.
4.3 Path name interpretation
There are a few complications to the rule that path names are
always parsed on the client. For example, symbolic links
could have different interpretations on different clients.
There is no answer to this problem in this specification.
Another common problem for non-UNIX implementations is the
special interpretation of the pathname, "..", to mean the
parent of a given directory. A future revision of the
protocol may use an explicit flag to indicate the parent
instead - however it is not a problem as many working
non-UNIX implementations exist.
4.4 Permission issues
The NFS version 3 protocol, strictly speaking, does not
define the permission checking used by servers. However, it
is expected that a server will do normal operating system
permission checking using AUTH_UNIX style authentication as
the basis of its protection mechanism, or another stronger
form of authentication such as AUTH_DES or AUTH_KERB. With
AUTH_UNIX authentication, the server gets the client's
effective uid, effective gid, and groups on each call and
uses them to check permission. These are the so-called UNIX
credentials. AUTH_DES and AUTH_KERB use a network name, or
netname, as the basis for identification (from which a UNIX
server derives the necessary standard UNIX credentials).
There are problems with this method that have been solved.
Using uid and gid implies that the client and server share
the same uid list. Every server and client pair must have the
same mapping from user to uid and from group to gid. Since
every client can also be a server, this tends to imply that
the whole network shares the same uid/gid space. If this is
not the case, then it usually falls upon the server to
perform some custom mapping of credentials from one
authentication domain into another. A discussion of
techniques for managing a shared user space or for providing
mechanisms for user ID mapping is beyond the scope of this
specification.
Another problem arises due to the usually stateful open
operation. Most operating systems check permission at open
time, and then check that the file is open on each read and
write request. With stateless servers, the server cannot
detect that the file is open and must do permission checking
on each read and write call. UNIX client semantics of access
permission checking on open can be provided with the ACCESS
procedure call in this revision, which allows a client to
explicitly check access permissions without resorting to
trying the operation. On a local file system, a user can open
a file and then change the permissions so that no one is
allowed to touch it, but will still be able to write to the
file because it is open. On a remote file system, by
contrast, the write would fail. To get around this problem,
the server's permission checking algorithm should allow the
owner of a file to access it regardless of the permission
setting. This is needed in a practical NFS version 3 protocol
server implementation, but it does depart from correct local
file system semantics. This should not affect the return
result of access permissions as returned by the ACCESS
procedure, however.
A similar problem has to do with paging in an executable
program over the network. The operating system usually checks
for execute permission before opening a file for demand
paging, and then reads blocks from the open file. In a local
UNIX file system, an executable file does not need read
permission to execute (pagein). An NFS version 3 protocol
server can not tell the difference between a normal file read
(where the read permission bit is meaningful) and a demand
pagein read (where the server should allow access to the
executable file if the execute bit is set for that user or
group or public). To make this work, the server allows
reading of files if the uid given in the call has either
execute or read permission on the file, through ownership,
group membership or public access. Again, this departs from
correct local file system semantics.
In most operating systems, a particular user (on UNIX, the
uid 0) has access to all files, no matter what permission and
ownership they have. This superuser permission may not be
allowed on the server, since anyone who can become superuser
on their client could gain access to all remote files. A UNIX
server by default maps uid 0 to a distinguished value
(UID_NOBODY), as well as mapping the groups list, before
doing its access checking. A server implementation may
provide a mechanism to change this mapping. This works except
for NFS version 3 protocol root file systems (required for
diskless NFS version 3 protocol client support), where
superuser access cannot be avoided. Export options are used,
on the server, to restrict the set of clients allowed
superuser access.
4.5 Duplicate request cache
The typical NFS version 3 protocol failure recovery model
uses client time-out and retry to handle server crashes,
network partitions, and lost server replies. A retried
request is called a duplicate of the original.
When used in a file server context, the term idempotent can
be used to distinguish between operation types. An idempotent
request is one that a server can perform more than once with
equivalent results (though it may in fact change, as a side
effect, the access time on a file, say for READ). Some NFS
operations are obviously non-idempotent. They cannot be
reprocessed without special attention simply because they may
fail if tried a second time. The CREATE request, for example,
can be used to create a file for which the owner does not
have write permission. A duplicate of this request cannot
succeed if the original succeeded. Likewise, a file can be
removed only once.
The side effects caused by performing a duplicate
non-idempotent request can be destructive (for example, a
truncate operation causing lost writes). The combination of a
stateless design with the common choice of an unreliable
network transport (UDP) implies the possibility of
destructive replays of non-idempotent requests. Though to be
more accurate, it is the inherent stateless design of the NFS
version 3 protocol on top of an unreliable RPC mechanism that
yields the possibility of destructive replays of
non-idempotent requests, since even in an implementation of
the NFS version 3 protocol over a reliable
connection-oriented transport, a connection break with
automatic reestablishment requires duplicate request
processing (the client will retransmit the request, and the
server needs to deal with a potential duplicate
non-idempotent request).
Most NFS version 3 protocol server implementations use a
cache of recent requests (called the duplicate request cache)
for the processing of duplicate non-idempotent requests. The
duplicate request cache provides a short-term memory
mechanism in which the original completion status of a
request is remembered and the operation attempted only once.
If a duplicate copy of this request is received, then the
original completion status is returned.
The duplicate-request cache mechanism has been useful in
reducing destructive side effects caused by duplicate NFS
version 3 protocol requests. This mechanism, however, does
not guarantee against these destructive side effects in all
failure modes. Most servers store the duplicate request cache
in RAM, so the contents are lost if the server crashes. The
exception to this may possibly occur in a redundant server
approach to high availability, where the file system itself
may be used to share the duplicate request cache state. Even
if the cache survives server reboots (or failovers in the
high availability case), its effectiveness is a function of
its size. A network partition can cause a cache entry to be
reused before a client receives a reply for the corresponding
request. If this happens, the duplicate request will be
processed as a new one, possibly with destructive side
effects.
A good description of the implementation and use of a
duplicate request cache can be found in [Juszczak].
4.6 File name component handling
Server implementations of NFS version 3 protocol will
frequently impose restrictions on the names which can be
created. Many servers will also forbid the use of names that
contain certain characters, such as the path component
separator used by the server operating system. For example,
the UFS file system will reject a name which contains "/",
while "." and ".." are distinguished in UFS, and may not be
specified as the name when creating a file system object.
The exact error status values return for these errors is
specified in the description of each procedure argument. The
values (which conform to NFS version 2 protocol server
practice) are not necessarily obvious, nor are they
consistent from one procedure to the next.
4.7 Synchronous modifying operations
Data-modifying operations in the NFS version 3 protocol are
synchronous. When a procedure returns to the client, the
client can assume that the operation has completed and any
data associated with the request is now on stable storage.
4.8 Stable storage
NFS version 3 protocol servers must be able to recover
without data loss from multiple power failures (including
cascading power failures, that is, several power failures in
quick succession), operating system failures, and hardware
failure of components other than the storage medium itself
(for example, disk, nonvolatile RAM).
Some examples of stable storage that are allowable for an NFS
server include:
1. Media commit of data, that is, the modified data has
been successfully written to the disk media, for example,
the disk platter.
2. An immediate reply disk drive with battery-backed
on-drive intermediate storage or uninterruptible power
system (UPS).
3. Server commit of data with battery-backed intermediate
storage and recovery software.
4. Cache commit with uninterruptible power system (UPS) and
recovery software.
Conversely, the following are not examples of stable
storage:
1. An immediate reply disk drive without battery-backed
on-drive intermediate storage or uninterruptible power
system (UPS).
2. Cache commit without both uninterruptible power system
(UPS) and recovery software.
The only exception to this (introduced in this protocol
revision) is as described under the WRITE procedure on the
handling of the stable bit, and the use of the COMMIT
procedure. It is the use of the synchronous COMMIT procedure
that provides the necessary semantic support in the NFS
version 3 protocol.
4.9 Lookups and name resolution
A common objection to the NFS version 3 protocol is the
philosophy of component-by-component LOOKUP by the client in
resolving a name. The objection is that this is inefficient,
as latencies for component-by-component LOOKUP would be
unbearable.
Implementation practice solves this issue. A name cache,
providing component to file-handle mapping, is kept on the
client to short circuit actual LOOKUP invocations over the
wire. The cache is subject to cache timeout parameters that
bound attributes.
4.10 Adaptive retransmission
Most client implementations use either an exponential
back-off strategy to some maximum retransmission value, or a
more adaptive strategy that attempts congestion avoidance.
Congestion avoidance schemes in NFS request retransmission
are modelled on the work presented in [Jacobson]. [Nowicki]
and [Macklem] describe congestion avoidance schemes to be
applied to the NFS protocol over UDP.
4.11 Caching policies
The NFS version 3 protocol does not define a policy for
caching on the client or server. In particular, there is no
support for strict cache consistency between a client and
server, nor between different clients. See [Kazar] for a
discussion of the issues of cache synchronization and
mechanisms in several distributed file systems.
4.12 Stable versus unstable writes
The setting of the stable field in the WRITE arguments, that
is whether or not to do asynchronous WRITE requests, is
straightforward on a UNIX client. If the NFS version 3
protocol client receives a write request that is not marked
as being asynchronous, it should generate the RPC with stable
set to TRUE. If the request is marked as being asynchronous,
the RPC should be generated with stable set to FALSE. If the
response comes back with the committed field set to TRUE, the
client should just mark the write request as done and no
further action is required. If committed is set to FALSE,
indicating that the buffer was not synchronized with the
server's disk, the client will need to mark the buffer in
some way which indicates that a copy of the buffer lives on
the server and that a new copy does not need to be sent to
the server, but that a commit is required.
Note that this algorithm introduces a new state for buffers,
thus there are now three states for buffers. The three states
are dirty, done but needs to be committed, and done. This
extra state on the client will likely require modifications
to the system outside of the NFS version 3 protocol client.
One proposal that was rejected was the addition of a boolean
commit argument to the WRITE operation. It would be used to
indicate whether the server should do a full file commit
after doing the write. This seems as if it could be useful if
the client knew that it was doing the last write on the file.
It is difficult to see how this could be used, given existing
client architectures though.
The asynchronous write opens up the window of problems
associated with write sharing. For example: client A writes
some data asynchronously. Client A is still holding the
buffers cached, waiting to commit them later. Client B reads
the modified data and writes it back to the server. The
server then crashes. When it comes back up, client A issues a
COMMIT operation which returns with a different cookie as
well as changed attributes. In this case, the correct action
may or may not be to retransmit the cached buffers.
Unfortunately, client A can't tell for sure, so it will need
to retransmit the buffers, thus overwriting the changes from
client B. Fortunately, write sharing is rare and the
solution matches the current write sharing situation. Without
using locking for synchronization, the behaviour will be
indeterminate.
In a high availability (redundant system) server
implementation, two cases exist which relate to the verf
changing. If the high availability server implementation
does not use a shared-memory scheme, then the verf should
change on failover, since the unsynchronized data is not
available to the second processor and there is no guarantee
that the system which had the data cached was able to flush
it to stable storage before going down. The client will need
to retransmit the data to be safe. In a shared-memory high
availability server implementation, the verf would not need
to change because the server would still have the cached data
available to it to be flushed. The exact policy regarding the
verf in a shared memory high availability implementation,
however, is up to the server implementor.
4.13 32 bit clients/servers and 64 bit clients/servers
The 64 bit nature of the NFS version 3 protocol introduces
several compatibility problems. The most notable two are
mismatched clients and servers, that is, a 32 bit client and
a 64 bit server or a 64 bit client and a 32 bit server.
The problems of a 64 bit client and a 32 bit server are easy
to handle. The client will never encounter a file that it can
not handle. If it sends a request to the server that the
server can not handle, the server should reject the request
with an appropriate error.
The problems of a 32 bit client and a 64 bit server are much
harder to handle. In this situation, the server does not have
a problem because it can handle anything that the client can
generate. However, the client may encounter a file that it
can not handle. The client will not be able to handle a file
whose size can not be expressed in 32 bits. Thus, the client
will not be able to properly decode the size of the file into
its local attributes structure. Also, a file can grow beyond
the limit of the client while the client is accessing the
file.
The solutions to these problems are left up to the individual
implementor. However, there are two common approaches used to
resolve this situation. The implementor can choose between
them or even can invent a new solution altogether.
The most common solution is for the client to deny access to
any file whose size can not be expressed in 32 bits. This is
probably the safest, but does introduce some strange
semantics when the file grows beyond the limit of the client
while it is being access by that client. The file becomes
inaccessible even while it is being accessed.
The second solution is for the client to map any size greater
than it can handle to the maximum size that it can handle.
Effectively, it is lying to the application program. This
allows the application access as much of the file as possible
given the 32 bit offset restriction. This eliminates the
strange semantic of the file effectively disappearing after
it has been accessed, but does introduce other problems. The
client will not be able to access the entire file.
Currently, the first solution is the recommended solution.
However, client implementors are encouraged to do the best
that they can to reduce the effects of this situation.
5.0 Appendix I: Mount protocol
The changes from the NFS version 2 protocol to the NFS version 3
protocol have required some changes to be made in the MOUNT
protocol. To meet the needs of the NFS version 3 protocol, a
new version of the MOUNT protocol has been defined. This new
protocol satisfies the requirements of the NFS version 3
protocol and addresses several other current market
requirements.
5.1 RPC Information
5.1.1 Authentication
The MOUNT service uses AUTH_NONE in the NULL procedure.
AUTH_UNIX, AUTH_SHORT, AUTH_DES, or AUTH_KERB are used for all
other procedures. Other authentication types may be supported
in the future.
5.1.2 Constants
These are the RPC constants needed to call the MOUNT service.
They are given in decimal.
PROGRAM 100005
VERSION 3
5.1.3 Transport address
The MOUNT service is normally supported over the TCP and UDP
protocols. The rpcbind daemon should be queried for the correct
transport address.
5.1.4 Sizes
const MNTPATHLEN = 1024; /* Maximum bytes in a path name */
const MNTNAMLEN = 255; /* Maximum bytes in a name */
const FHSIZE3 = 64; /* Maximum bytes in a V3 file handle */
5.1.5 Basic Data Types
typedef opaque fhandle3<FHSIZE3>;
typedef string dirpath<MNTPATHLEN>;
typedef string name<MNTNAMLEN>;
enum mountstat3 {
MNT3_OK = 0, /* no error */
MNT3ERR_PERM = 1, /* Not owner */
MNT3ERR_NOENT = 2, /* No such file or directory */
MNT3ERR_IO = 5, /* I/O error */
MNT3ERR_ACCES = 13, /* Permission denied */
MNT3ERR_NOTDIR = 20, /* Not a directory */
MNT3ERR_INVAL = 22, /* Invalid argument */
MNT3ERR_NAMETOOLONG = 63, /* Filename too long */
MNT3ERR_NOTSUPP = 10004, /* Operation not supported */
MNT3ERR_SERVERFAULT = 10006 /* A failure on the server */
};
5.2 Server Procedures
The following sections define the RPC procedures supplied by a
MOUNT version 3 protocol server. The RPC procedure number is
given at the top of the page with the name and version. The
SYNOPSIS provides the name of the procedure, the list of the
names of the arguments, the list of the names of the results,
followed by the XDR argument declarations and results
declarations. The information in the SYNOPSIS is specified in
RPC Data Description Language as defined in [RFC1014]. The
DESCRIPTION section tells what the procedure is expected to do
and how its arguments and results are used. The ERRORS section
lists the errors returned for specific types of failures. The
IMPLEMENTATION field describes how the procedure is expected to
work and how it should be used by clients.
program MOUNT_PROGRAM {
version MOUNT_V3 {
void MOUNTPROC3_NULL(void) = 0;
mountres3 MOUNTPROC3_MNT(dirpath) = 1;
mountlist MOUNTPROC3_DUMP(void) = 2;
void MOUNTPROC3_UMNT(dirpath) = 3;
void MOUNTPROC3_UMNTALL(void) = 4;
exports MOUNTPROC3_EXPORT(void) = 5;
} = 3;
} = 100005;
5.2.0 Procedure 0: Null - Do nothing
SYNOPSIS
void MOUNTPROC3_NULL(void) = 0;
DESCRIPTION
Procedure NULL does not do any work. It is made available
to allow server response testing and timing.
IMPLEMENTATION
It is important that this procedure do no work at all so
that it can be used to measure the overhead of processing
a service request. By convention, the NULL procedure
should never require any authentication. A server may
choose to ignore this convention, in a more secure
implementation, where responding to the NULL procedure
call acknowledges the existence of a resource to an
unauthenticated client.
ERRORS
Since the NULL procedure takes no MOUNT protocol arguments
and returns no MOUNT protocol response, it can not return
a MOUNT protocol error. However, it is possible that some
server implementations may return RPC errors based on
security and authentication requirements.
5.2.1 Procedure 1: MNT - Add mount entry
SYNOPSIS
mountres3 MOUNTPROC3_MNT(dirpath) = 1;
struct mountres3_ok {
fhandle3 fhandle;
int auth_flavors<>;
};
union mountres3 switch (mountstat3 fhs_status) {
case MNT_OK:
mountres3_ok mountinfo;
default:
void;
};
DESCRIPTION
Procedure MNT maps a pathname on the server to a file
handle. The pathname is an ASCII string that describes a
directory on the server. If the call is successful
(MNT3_OK), the server returns an NFS version 3 protocol
file handle and a vector of RPC authentication flavors
that are supported with the client's use of the file
handle (or any file handles derived from it). The
authentication flavors are defined in Section 7.2 and
section 9 of [RFC1057].
IMPLEMENTATION
If mountres3.fhs_status is MNT3_OK, then
mountres3.mountinfo contains the file handle for the
directory and a list of acceptable authentication
flavors. This file handle may only be used in the NFS
version 3 protocol. This procedure also results in the
server adding a new entry to its mount list recording that
this client has mounted the directory. AUTH_UNIX
authentication or better is required.
ERRORS
MNT3ERR_NOENT
MNT3ERR_IO
MNT3ERR_ACCES
MNT3ERR_NOTDIR
MNT3ERR_NAMETOOLONG
5.2.2 Procedure 2: DUMP - Return mount entries
SYNOPSIS
mountlist MOUNTPROC3_DUMP(void) = 2;
typedef struct mountbody *mountlist;
struct mountbody {
name ml_hostname;
dirpath ml_directory;
mountlist ml_next;
};
DESCRIPTION
Procedure DUMP returns the list of remotely mounted file
systems. The mountlist contains one entry for each client
host name and directory pair.
IMPLEMENTATION
This list is derived from a list maintained on the server
of clients that have requested file handles with the MNT
procedure. Entries are removed from this list only when a
client calls the UMNT or UMNTALL procedure. Entries may
become stale if a client crashes and does not issue either
UMNT calls for all of the file systems that it had
previously mounted or a UMNTALL to remove all entries that
existed for it on the server.
ERRORS
There are no MOUNT protocol errors which can be returned
from this procedure. However, RPC errors may be returned
for authentication or other RPC failures.
5.2.3 Procedure 3: UMNT - Remove mount entry
SYNOPSIS
void MOUNTPROC3_UMNT(dirpath) = 3;
DESCRIPTION
Procedure UMNT removes the mount list entry for the
directory that was previously the subject of a MNT call
from this client. AUTH_UNIX authentication or better is
required.
IMPLEMENTATION
Typically, server implementations have maintained a list
of clients which have file systems mounted. In the past,
this list has been used to inform clients that the server
was going to be shutdown.
ERRORS
There are no MOUNT protocol errors which can be returned
from this procedure. However, RPC errors may be returned
for authentication or other RPC failures.
5.2.4 Procedure 4: UMNTALL - Remove all mount entries
SYNOPSIS
void MOUNTPROC3_UMNTALL(void) = 4;
DESCRIPTION
Procedure UMNTALL removes all of the mount entries for
this client previously recorded by calls to MNT. AUTH_UNIX
authentication or better is required.
IMPLEMENTATION
This procedure should be used by clients when they are
recovering after a system shutdown. If the client could
not successfully unmount all of its file systems before
being shutdown or the client crashed because of a software
or hardware problem, there may be servers which still have
mount entries for this client. This is an easy way for the
client to inform all servers at once that it does not have
any mounted file systems. However, since this procedure
is generally implemented using broadcast RPC, it is only
of limited usefullness.
ERRORS
There are no MOUNT protocol errors which can be returned
from this procedure. However, RPC errors may be returned
for authentication or other RPC failures.
5.2.5 Procedure 5: EXPORT - Return export list
SYNOPSIS
exports MOUNTPROC3_EXPORT(void) = 5;
typedef struct groupnode *groups;
struct groupnode {
name gr_name;
groups gr_next;
};
typedef struct exportnode *exports;
struct exportnode {
dirpath ex_dir;
groups ex_groups;
exports ex_next;
};
DESCRIPTION
Procedure EXPORT returns a list of all the exported file
systems and which clients are allowed to mount each one.
The names in the group list are implementation-specific
and cannot be directly interpreted by clients. These names
can represent hosts or groups of hosts.
IMPLEMENTATION
This procedure generally returns the contents of a list of
shared or exported file systems. These are the file
systems which are made available to NFS version 3 protocol
clients.
ERRORS
There are no MOUNT protocol errors which can be returned
from this procedure. However, RPC errors may be returned
for authentication or other RPC failures.
6.0 Appendix II: Lock manager protocol
Because the NFS version 2 protocol as well as the NFS version 3
protocol is stateless, an additional Network Lock Manager (NLM)
protocol is required to support locking of NFS-mounted files.
The NLM version 3 protocol, which is used with the NFS version 2
protocol, is documented in [X/OpenNFS].
Some of the changes in the NFS version 3 protocol require a
new version of the NLM protocol. This new protocol is the NLM
version 4 protocol. The following table summarizes the
correspondence between versions of the NFS protocol and NLM
protocol.
NFS and NLM protocol compatibility
+---------+---------+
| NFS | NLM |
| Version | Version |
+===================+
| 2 | 1,3 |
+---------+---------+
| 3 | 4 |
+---------+---------+
This appendix only discusses the differences between the NLM
version 3 protocol and the NLM version 4 protocol. As in the
NFS version 3 protocol, almost all the names in the NLM version
4 protocol have been changed to include a version number. This
appendix does not discuss changes that consist solely of a name
change.
6.1 RPC Information
6.1.1 Authentication
The NLM service uses AUTH_NONE in the NULL procedure.
AUTH_UNIX, AUTH_SHORT, AUTH_DES, and AUTH_KERB are used for
all other procedures. Other authentication types may be
supported in the future.
6.1.2 Constants
These are the RPC constants needed to call the NLM service.
They are given in decimal.
PROGRAM 100021
VERSION 4
6.1.3 Transport Address
The NLM service is normally supported over the TCP and UDP
protocols. The rpcbind daemon should be queried for the
correct transport address.
6.1.4 Basic Data Types
uint64
typedef unsigned hyper uint64;
int64
typedef hyper int64;
uint32
typedef unsigned long uint32;
int32
typedef long int32;
These types are new for the NLM version 4 protocol. They are
the same as in the NFS version 3 protocol.
nlm4_stats
enum nlm4_stats {
NLM4_GRANTED = 0,
NLM4_DENIED = 1,
NLM4_DENIED_NOLOCKS = 2,
NLM4_BLOCKED = 3,
NLM4_DENIED_GRACE_PERIOD = 4,
NLM4_DEADLCK = 5,
NLM4_ROFS = 6,
NLM4_STALE_FH = 7,
NLM4_FBIG = 8,
NLM4_FAILED = 9
};
Nlm4_stats indicates the success or failure of a call. This
version contains several new error codes, so that clients can
provide more precise failure information to applications.
NLM4_GRANTED
The call completed successfully.
NLM4_DENIED
The call failed. For attempts to set a lock, this status
implies that if the client retries the call later, it may
succeed.
NLM4_DENIED_NOLOCKS
The call failed because the server could not allocate the
necessary resources.
NLM4_BLOCKED
Indicates that a blocking request cannot be granted
immediately. The server will issue an NLMPROC4_GRANTED
callback to the client when the lock is granted.
NLM4_DENIED_GRACE_PERIOD
The call failed because the server is reestablishing old
locks after a reboot and is not yet ready to resume normal
service.
NLM4_DEADLCK
The request could not be granted and blocking would cause
a deadlock.
NLM4_ROFS
The call failed because the remote file system is
read-only. For example, some server implementations might
not support exclusive locks on read-only file systems.
NLM4_STALE_FH
The call failed because it uses an invalid file handle.
This can happen if the file has been removed or if access
to the file has been revoked on the server.
NLM4_FBIG
The call failed because it specified a length or offset
that exceeds the range supported by the server.
NLM4_FAILED
The call failed for some reason not already listed. The
client should take this status as a strong hint not to
retry the request.
nlm4_holder
struct nlm4_holder {
bool exclusive;
int32 svid;
netobj oh;
uint64 l_offset;
uint64 l_len;
};
This structure indicates the holder of a lock. The exclusive
field tells whether the holder has an exclusive lock or a
shared lock. The svid field identifies the process that is
holding the lock. The oh field is an opaque object that
identifies the host or process that is holding the lock. The
l_len and l_offset fields identify the region that is locked.
The only difference between the NLM version 3 protocol and
the NLM version 4 protocol is that in the NLM version 3
protocol, the l_len and l_offset fields are 32 bits wide,
while they are 64 bits wide in the NLM version 4 protocol.
nlm4_lock
struct nlm4_lock {
string caller_name<LM_MAXSTRLEN>;
netobj fh;
netobj oh;
int32 svid;
uint64 l_offset;
uint64 l_len;
};
This structure describes a lock request. The caller_name
field identifies the host that is making the request. The fh
field identifies the file to lock. The oh field is an opaque
object that identifies the host or process that is making the
request, and the svid field identifies the process that is
making the request. The l_offset and l_len fields identify
the region of the file that the lock controls. A l_len of 0
means "to end of file".
There are two differences between the NLM version 3 protocol
and the NLM version 4 protocol versions of this structure.
First, in the NLM version 3 protocol, the length and offset
are 32 bits wide, while they are 64 bits wide in the NLM
version 4 protocol. Second, in the NLM version 3 protocol,
the file handle is a fixed-length NFS version 2 protocol file
handle, which is encoded as a byte count followed by a byte
array. In the NFS version 3 protocol, the file handle is
already variable-length, so it is copied directly into the fh
field. That is, the first four bytes of the fh field are the
same as the byte count in an NFS version 3 protocol nfs_fh3.
The rest of the fh field contains the byte array from the NFS
version 3 protocol nfs_fh3.
nlm4_share
struct nlm4_share {
string caller_name<LM_MAXSTRLEN>;
netobj fh;
netobj oh;
fsh4_mode mode;
fsh4_access access;
};
This structure is used to support DOS file sharing. The
caller_name field identifies the host making the request.
The fh field identifies the file to be operated on. The oh
field is an opaque object that identifies the host or process
that is making the request. The mode and access fields
specify the file-sharing and access modes. The encoding of fh
is a byte count, followed by the file handle byte array. See
the description of nlm4_lock for more details.
6.2 NLM Procedures
The procedures in the NLM version 4 protocol are semantically
the same as those in the NLM version 3 protocol. The only
semantic difference is the addition of a NULL procedure that
can be used to test for server responsiveness. The procedure
names with _MSG and _RES suffixes denote asynchronous
messages; for these the void response implies no reply. A
syntactic change is that the procedures were renamed to avoid
name conflicts with the values of nlm4_stats. Thus the
procedure definition is as follows.
version NLM4_VERS {
void
NLMPROC4_NULL(void) = 0;
nlm4_testres
NLMPROC4_TEST(nlm4_testargs) = 1;
nlm4_res
NLMPROC4_LOCK(nlm4_lockargs) = 2;
nlm4_res
NLMPROC4_CANCEL(nlm4_cancargs) = 3;
nlm4_res
NLMPROC4_UNLOCK(nlm4_unlockargs) = 4;
nlm4_res
NLMPROC4_GRANTED(nlm4_testargs) = 5;
void
NLMPROC4_TEST_MSG(nlm4_testargs) = 6;
void
NLMPROC4_LOCK_MSG(nlm4_lockargs) = 7;
void
NLMPROC4_CANCEL_MSG(nlm4_cancargs) = 8;
void
NLMPROC4_UNLOCK_MSG(nlm4_unlockargs) = 9;
void
NLMPROC4_GRANTED_MSG(nlm4_testargs) = 10;
void
NLMPROC4_TEST_RES(nlm4_testres) = 11;
void
NLMPROC4_LOCK_RES(nlm4_res) = 12;
void
NLMPROC4_CANCEL_RES(nlm4_res) = 13;
void
NLMPROC4_UNLOCK_RES(nlm4_res) = 14;
void
NLMPROC4_GRANTED_RES(nlm4_res) = 15;
nlm4_shareres
NLMPROC4_SHARE(nlm4_shareargs) = 20;
nlm4_shareres
NLMPROC4_UNSHARE(nlm4_shareargs) = 21;
nlm4_res
NLMPROC4_NM_LOCK(nlm4_lockargs) = 22;
void
NLMPROC4_FREE_ALL(nlm4_notify) = 23;
} = 4;
6.2.0 Procedure 0: NULL - Do nothing
SYNOPSIS
void NLMPROC4_NULL(void) = 0;
DESCRIPTION
The NULL procedure does no work. It is made available in
all RPC services to allow server response testing and
timing.
IMPLEMENTATION
It is important that this procedure do no work at all so
that it can be used to measure the overhead of processing
a service request. By convention, the NULL procedure
should never require any authentication.
ERRORS
It is possible that some server implementations may return
RPC errors based on security and authentication
requirements.
6.3 Implementation issues
6.3.1 64-bit offsets and lengths
Some NFS version 3 protocol servers can only support
requests where the file offset or length fits in 32 or
fewer bits. For these servers, the lock manager will have
the same restriction. If such a lock manager receives a
request that it cannot handle (because the offset or
length uses more than 32 bits), it should return the
error, NLM4_FBIG.
6.3.2 File handles
The change in the file handle format from the NFS version
2 protocol to the NFS version 3 protocol complicates the
lock manager. First, the lock manager needs some way to
tell when an NFS version 2 protocol file handle refers to
the same file as an NFS version 3 protocol file handle.
(This is assuming that the lock manager supports both NLM
version 3 protocol clients and NLM version 4 protocol
clients.) Second, if the lock manager runs the file handle
through a hashing function, the hashing function may need
to be retuned to work with NFS version 3 protocol file
handles as well as NFS version 2 protocol file handles.
7.0 Appendix III: Bibliography
[Corbin] Corbin, John, "The Art of Distributed
Programming-Programming Techniques for Remote
Procedure Calls." Springer-Verlag, New York, New
York. 1991. Basic description of RPC and XDR
and how to program distributed applications
using them.
[Glover] Glover, Fred, "TNFS Protocol Specification,"
Trusted System Interest Group, Work in
Progress.
[Israel] Israel, Robert K., Sandra Jett, James Pownell,
George M. Ericson, "Eliminating Data Copies in
UNIX-based NFS Servers," Uniforum Conference
Proceedings, San Francisco, CA,
February 27 - March 2, 1989. Describes two
methods for reducing data copies in NFS server
code.
[Jacobson] Jacobson, V., "Congestion Control and
Avoidance," Proc. ACM SIGCOMM `88, Stanford, CA,
August 1988. The paper describing improvements
to TCP to allow use over Wide Area Networks and
through gateways connecting networks of varying
capacity. This work was a starting point for the
NFS Dynamic Retransmission work.
[Juszczak] Juszczak, Chet, "Improving the Performance and
Correctness of an NFS Server," USENIX Conference
Proceedings, USENIX Association, Berkeley, CA,
June 1990, pages 53-63. Describes reply cache
implementation that avoids work in the server by
handling duplicate requests. More important,
though listed as a side-effect, the reply cache
aids in the avoidance of destructive
non-idempotent operation re-application --
improving correctness.
[Kazar] Kazar, Michael Leon, "Synchronization and Caching
Issues in the Andrew File System," USENIX Conference
Proceedings, USENIX Association, Berkeley, CA,
Dallas Winter 1988, pages 27-36. A description
of the cache consistency scheme in AFS.
Contrasted with other distributed file systems.
[Macklem] Macklem, Rick, "Lessons Learned Tuning the
4.3BSD Reno Implementation of the NFS Protocol,"
Winter USENIX Conference Proceedings, USENIX
Association, Berkeley, CA, January 1991.
Describes performance work in tuning the 4.3BSD
Reno NFS implementation. Describes performance
improvement (reduced CPU loading) through
elimination of data copies.
[Mogul] Mogul, Jeffrey C., "A Recovery Protocol for Spritely
NFS," USENIX File System Workshop Proceedings,
Ann Arbor, MI, USENIX Association, Berkeley, CA,
May 1992. Second paper on Spritely NFS proposes
a lease-based scheme for recovering state of
consistency protocol.
[Nowicki] Nowicki, Bill, "Transport Issues in the Network
File System," ACM SIGCOMM newsletter Computer
Communication Review, April 1989. A brief
description of the basis for the dynamic
retransmission work.
[Pawlowski] Pawlowski, Brian, Ron Hixon, Mark Stein, Joseph
Tumminaro, "Network Computing in the UNIX and
IBM Mainframe Environment," Uniforum `89 Conf.
Proc., (1989) Description of an NFS server
implementation for IBM's MVS operating system.
[RFC1014] Sun Microsystems, Inc., "XDR: External Data
Representation Standard", RFC1014,
Sun Microsystems, Inc., June 1987.
Specification for canonical format for data
exchange, used with RPC.
[RFC1057] Sun Microsystems, Inc., "RPC: Remote Procedure
Call Protocol Specification", RFC1057,
Sun Microsystems, Inc., June 1988.
Remote procedure protocol specification.
[RFC1094] Sun Microsystems, Inc., "Network Filesystem
Specification", RFC1094, Sun Microsystems, Inc.,
March 1989. NFS version 2 protocol
specification.
[Sandberg] Sandberg, R., D. Goldberg, S. Kleiman, D. Walsh,
B. Lyon, "Design and Implementation of the Sun
Network Filesystem," USENIX Conference
Proceedings, USENIX Association, Berkeley, CA,
Summer 1985. The basic paper describing the
SunOS implementation of the NFS version 2
protocol, and discusses the goals, protocol
specification and trade-offs.
[Srinivasan] Srinivasan, V., Jeffrey C. Mogul, "Spritely
NFS: Implementation and Performance of Cache
Consistency Protocols", WRL Research Report
89/5, Digital Equipment Corporation Western
Research Laboratory, 100 Hamilton Ave., Palo
Alto, CA, 94301, May 1989. This paper analyzes
the effect of applying a Sprite-like consistency
protocol applied to standard NFS. The issues of
recovery in a stateful environment are covered
in [Mogul].
[X/OpenNFS] X/Open Company, Ltd., X/Open CAE Specification:
Protocols for X/Open Internetworking: XNFS,
X/Open Company, Ltd., Apex Plaza, Forbury Road,
Reading Berkshire, RG1 1AX, United Kingdom,
1991. This is an indispensable reference for
NFS version 2 protocol and accompanying
protocols, including the Lock Manager and the
Portmapper.
[X/OpenPCNFS] X/Open Company, Ltd., X/Open CAE Specification:
Protocols for X/Open Internetworking: (PC)NFS,
Developer's Specification, X/Open Company, Ltd.,
Apex Plaza, Forbury Road, Reading Berkshire, RG1
1AX, United Kingdom, 1991. This is an
indispensable reference for NFS version 2
protocol and accompanying protocols, including
the Lock Manager and the Portmapper.
8. Security Considerations
Since sensitive file data may be transmitted or received
from a server by the NFS protocol, authentication, privacy,
and data integrity issues should be addressed by implementations
of this protocol.
As with the previous protocol revision (version 2), NFS
version 3 defers to the authentication provisions of the
supporting RPC protocol [RFC1057], and assumes that data
privacy and integrity are provided by underlying transport
layers as available in each implementation of the protocol.
See section 4.4 for a discussion relating to file access
permissions.
9. Acknowledgements
This description of the protocol is derived from an original
document written by Brian Pawlowski and revised by Peter
Staubach. This protocol is the result of a co-operative
effort that comprises the contributions of Geoff Arnold,
Brent Callaghan, John Corbin, Fred Glover, Chet Juszczak,
Mike Eisler, John Gillono, Dave Hitz, Mike Kupfer, Rick
Macklem, Ron Minnich, Brian Pawlowski, David Robinson, Rusty
Sandberg, Craig Schamp, Spencer Shepler, Carl Smith, Mark
Stein, Peter Staubach, Tom Talpey, Rob Thurlow, and Mark
Wittle.
10. Authors' Addresses
Address comments related to this protocol to:
nfs3@eng.sun.com
Brent Callaghan
Sun Microsystems, Inc.
2550 Garcia Avenue
Mailstop UMTV05-44
Mountain View, CA 94043-1100
Phone: 1-415-336-1051
Fax: 1-415-336-6015
EMail: brent.callaghan@eng.sun.com
Brian Pawlowski
Network Appliance Corp.
319 North Bernardo Ave.
Mountain View, CA 94043
Phone: 1-415-428-5136
Fax: 1-415-428-5151
EMail: beepy@netapp.com
Peter Staubach
Sun Microsystems, Inc.
2550 Garcia Avenue
Mailstop UMTV05-44
Mountain View, CA 94043-1100
Phone: 1-415-336-5615
Fax: 1-415-336-6015