NRT No Retransmission - do not ask for retransmissions of
this packet group if not all received within timeout
Cheriton [page 39]
RFC1045 VMTP February 1988
period, just deliver or discard.
MDG Member of Destination Group - this packet is sent to a
group and the client is a member of this group.
CMG Continued Message - the message (Request or Response) is
continued in the next packet group. The next packet
group has to be part of the same run of packet groups.
STI Skip Transaction Identifiers - the next transaction
identifier that the Client plans to use is the current
transaction plus 256, if part of the same run and at
least this big if not. In a Request, this authorizes
the Server to send back up to 256 packet groups
containing the Response.
DRT Delay Response Transmission - set by request sender if
multiple responses are expected (as indicated by the MRD
flag in the RequestCode) and it may be overrun by
multiple responses. The responder(s) should then
introduce a short random delay in sending the Response
to minimize the danger of overrunning the Client. This
is normally only used for responding to multicast
Requests where the Client may be receiving a large
number of Responses, as indicated by the MRD flag in the
Request flags. Otherwise, the Response is sent
immediately.
RetransmitCount:
3 bits - the ordinal number of transmissions of this
packet group prior to this one, modulo 8. This field is
used in estimation of roundtrip times. This count may
wrap around during a message transaction. However, it
should be sufficient to match acknowledgments and
responses with a particular transmission.
ForwardCount: 4 bits indicating the number of times this Request has
been forwarded. The original Request is always sent
with a ForwardCount of 0.
Interpacket Gap: 8 bits.
Indicates the recommended time to use between subsequent
packet transmissions within a multi-packet packet group
transmission. The Interpacket Gap time is in 1/32nd of
a network packet transmission time for a packet of size
MTU for the node. (Thus, the maximum gap time is 8
packet times.)
Cheriton [page 40]
RFC1045 VMTP February 1988
PGcount: 8 bits
The number of packet groups that this packet group
represents in addition to that specified by the
Transaction field. This is used in acknowledging
multiple packet groups in streamed communication.
Priority 4-bit identifier for priority for the processing of this
request both on transmission and reception. The
interpretation is:
1100 urgent/emergency
1000 important
0000 normal
0100 background
Viewing the higher-order bit as a sign bit (with 1
meaning negative), low values are high priority and high
values are low priority. The low-order 2 bits indicate
additional (lower) gradations for each level.
Function Code: 1 bit - types of VMTP packets. If the low-order bit of
the function code is 0, the packet is sent to the
Server, else it is sent to the Client.
0 Request
1 Response
Transaction: 32 bits:
Identifier for this message transaction.
PacketDelivery: 32 bits:
Delivery indicates the segment blocks contained in this
packet. Each bit corresponds to one 512-octet block of
segment data. A 1 bit in the i-th bit position
(counting the LSB as 0) indicates the presence of the
i-th segment block.
Server: 64 bits
Entity identifier for the server or server group
associated with this transaction. This is the receiver
when a Request packet and the sender when a Response
packet.
Cheriton [page 41]
RFC1045 VMTP February 1988
Code: 32 bits The Request Code and Response Code, set either at the
user level or VMTP level depending on use and packet
type. Both the Request and Response codes include 8
high-order bits from the following set of control bits:
CMD Conditional Message Delivery - only deliver the request
or response if the receiving entity is waiting for it at
the time of delivery, otherwise drop the message.
DGM DataGram Message - indicates that the message is being
sent as a datagram. If a Request message, do not wait
for reply, or retransmit. If a Response message, treat
this message transaction as idempotent.
MDM Message Delivery Mask - indicates that the MsgDelivery
field is being used. Otherwise, the MsgDelivery field
is available for general use.
SDA Segment Data Appended - segment data is appended to the
message control block, with the total size of the
segment specified by the SegmentSize field. Otherwise,
the segment data is null and the SegmentSize field is
not used by VMTP and available for user- or RPC-level
uses.
CRE CoResident Entity - indicates that the CoResidentEntity
field in the message should be interpreted by VMTP.
Otherwise, this field is available for additional user
data.
MRD Multiple Responses Desired - multiple Responses are
desired to to this Request if it is multicast.
Otherwise, the VMTP module can discard subsequent
Responses after the first Response.
PIC Public Interface Code - Values for Code with this bit
set are reserved for definition by the VMTP
specification and other standard protocols defined on
top of VMTP.
RES Reserved for future use. Must be 0.
CoResidentEntity
64-bit Identifier for an entity or group of entities
with which the Server entity or entities must be
co-resident, i.e. route only to entities (identified by
Server) on the same host(s) as that specified by
Cheriton [page 42]
RFC1045 VMTP February 1988
CoResidentEntity, Only meaningful if CRE is set in the
Code field.
User Data 12 octets Space in the header for the VMTP user to
specify user-specific control and data.
MsgDelivery: 32 bits
The segment blocks being transmitted (in total) in this
packet group following the conventions for the
PacketDelivery field. This field is ignored by the
protocol and treated as an additional user data field if
MDM is 0. On transmission, the user level sets the
MsgDelivery to indicate those portions of the segment to
be transmitted. On receipt, the MsgDelivery field is
modified by the VMTP module to indicate the segment data
blocks that were actually received before the message
control block is passed to the user or RPC level. In
particular, the kernel does not discard the packet group
if segment data blocks are missing. A Server or Client
entity receiving a message with a MsgDelivery in use
must check the field to ensure adequate delivery and
retry the operation if necessary.
SegmentSize: 32 bits
Size of segment in octets, up to a maximum of 16
kilooctets without streaming and 4 megaoctets with
streaming, if SDA is set. Otherwise, this field is
ignored by the protocol and treated as an additional
user data field.
Segment Data: 0-16 kilooctets
0 octets if SDA is 0, else the portion of the segment
corresponding to the Delivery Mask, limited by the
SegmentSize and the MTU, padded out to a multiple of 64
bits.
Checksum: 32 bits.
The 32-bit checksum for the header and segment data.
The VMTP checksum algorithm <9> develops a 32-bit checksum by computing
_______________
<9> This algorithm and description are largely due to Steve Deering of
Stanford University.
Cheriton [page 43]
RFC1045 VMTP February 1988
two 16-bit, ones-complement sums (like IP), each covering different
parts of the packet. The packet is divided into clusters of 16 16-bit
words. The first, third, fifth,... clusters are added to the first sum,
and the second, fourth, sixth,... clusters are added to the second sum.
Addition stops at the end of the packet; there is no need to pad out to
a cluster boundary (although it is necessary that the packet be an
integral multiple of 64 bits; padding octets may have any value and are
included in the checksum and in the transmitted packet). If either of
the resulting sums is zero, it is changed to 0xFFFF. The two sums are
appended to the transmitted packet, with the first sum being transmitted
first. Four bytes of zero in place of the checksum may be used to
indicate that no checksum was computed.
The 16-bit, ones-complement addition in this algorithm is the same as
used in IP and, therefore, subject to the same optimizations. In
particular, the words may be added up 32-bits at a time as long as the
carry-out of each addition is added to the sum on the following
addition, using an "add-with-carry" type of instruction. (64-bit or
128-bit additions would also work on machines that have registers that
big.)
A particular weakness of this algorithm (shared by IP) is that it does
not detect the erroneous swapping of 16-bit words, which may easily
occur due to software errors. A future version of VMTP is expected to
include a more secure algorithm, but such an algorithm appears to
require hardware support for efficient execution.
Not all of these fields are used in every packet. The specific packet
formats are described below. If a field is not mentioned in the
description of a packet type, its use is assumed to be clear from the
above description.
Cheriton [page 44]
RFC1045 VMTP February 1988
3.3. Request Packet
The Request packet (or packet group) is sent from the client to the
server or group of servers to solicit processing plus the return of zero
or more responses. A Request packet is identified by a 0 in the LSB of
the fourth 32-bit word in the packet.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ Client (8 octets) +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Ver | |H|E|M| |
|sion | Domain |C|P|P| Length |
| | |O|G|G| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|N|A|N|N|N|M|C|S|D|Retra|Forward| Inter- | |R|R|R| |
|R|P|S|E|R|D|M|T|R|nsmit| Count | Packet | Prior |E|E|E|0|
|S|G|R|R|T|G|G|I|T|Count| | Gap | -ity |S|S|S| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Transaction |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PacketDelivery |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ Server (8 octets) +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|C|D|M|S|R|C|M|P| |
|M|G|D|D|E|R|R|I| RequestCode |
|D|M|M|A|S|E|D|C| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ CoResidentEntity (8 octets) +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> User Data (12 octets) <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MsgDelivery |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SegmentSize |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> segment data, if any <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3-1: Request Packet Format
The fields of the Request packet are set according to the semantics
described in Section 3.2 with the following qualifications.
Cheriton [page 45]
RFC1045 VMTP February 1988
InterPacketGap The estimated interpacket gap time the client would like
for the Response packet group to be sent by the Server
in responding to this Request.
Transaction Identifier for transaction, at least one greater than
the previously issued Request from this Client.
Server Server to which this Request is destined.
RequestCode Request code for this request, indicating the operation
to perform.
Cheriton [page 46]
RFC1045 VMTP February 1988
3.4. Response Packet
The Response packet is sent from the Server to the Client in response to
a Request, identified by a 1 in the LSB of the fourth 32-bit word in the
packet.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ Client (8 octets) +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Ver | |H|E|M| |
|sion | Domain |C|P|P| Length |
| | |O|G|G| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|N|A|N|N|N|R|C|S|R|Retra|Forward| | |R|R|R| |
|R|P|S|E|R|E|M|T|E|nsmit| Count | PGcount | Prior |E|E|E|1|
|S|G|R|R|T|S|G|I|S|Count| | | -ity |S|S|S| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Transaction |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PacketDelivery |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ Server (8 octets) +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|C|D|M|S|R|R|R|R| |
|M|G|D|D|E|E|E|E| ResponseCode |
|D|M|M|A|S|S|S|S| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> UserData (20 octets) <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MsgDelivery |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Segment Size |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> segment data, if any <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3-2: Response Packet Format
The fields of the Response packet are set according to the semantics
described in Section 3.2 with the following qualifications.
Client, Version, Domain, Transaction
Match those in the Request packet group to which this is
Cheriton [page 47]
RFC1045 VMTP February 1988
a response.
STI 1 if this Response is using one or more of the
transaction identifiers skipped by the Client after the
Request to which this is a Response. STI in the Request
essentially allocates up to 256 transaction identifiers
for the Server to use in a run of Response packet
groups.
RetransmitCount The retransmit count from the last Request packet
received to which this is a response.
ForwardCount The number of times the corresponding Request was
forwarded before this Response was generated.
PGcount The number of consecutively previous packet groups that
this response is acknowledging in addition to the one
identified by the Transaction identifier.
Server Server sending this response. This may differ from that
originally specified in the Request packet if the
original Server was a server group, or the request was
forwarded.
The next two chapters describes the protocol operation using these
packet formats, with the the Client and the Server portions described
separately.
Cheriton [page 48]
RFC1045 VMTP February 1988
4. Client Protocol Operation
This chapter describes the operation of the client portion of VMTP in
terms of the procedures for handling VMTP user events, packet reception
events, management operations and timeout events. Note that the client
portion of VMTP is separable from the server portion. It is feasible to
have a node that only implements the client end of VMTP.
To simplify the description, we define a client state record (CSR) plus
some standard utility routines.
4.1. Client State Record Fields
In the following protocol description, there is one client state record
(CSR) per (client,transaction) outstanding message transaction. Here is
a suggested set of fields.
Link Link to next CSR when queued in one of the transmission,
timeout or message queues.
QueuePtr Pointer to queue head in which this CSR is contained or
NULL if none. Queue could be one of transmission queue,
timeout queue, server queue or response queue.
ProcessIdentification
The process identification and address space.
Priority Priority for processing, network service, etc.
State One of the client states described below.
FinishupFunc Procedure to be executed on the CSR when it is completes
its processing in transmission or timeout queues.
TimeoutCount Time to remain in timeout queue.
TimeoutLimit User-specified time after which the message transaction
is aborted. The timeout is infinite if set to zero.
RetransCount Number of retransmissions since last hearing from the
Server.
LastTransmitTime
The time at which the last packet was sent. This field
is used to calculate roundtrip times, using the
RetransmitCount to match the responding packet to a
Cheriton [page 49]
RFC1045 VMTP February 1988
particular transmission. I.e. Response or management
NotifyVmtpClient operation to Request and a management
NotifyVmtpServer operation to a Response.
TimetoLive Time to live to be used on transmission of IP packets.
TransmissionMask
Bit mask indicating the portions of the segment to
transmit. Set before entering the transmission queue
and cleared incrementally as the 512-byte segment blocks
of the segment are transmitted.
LocalClientLink Link to next CSR hashing to same hash index in the
ClientMap.
LocalClient Entity identifier for client when this CSR is used to
send a Request packet.
LocalTransaction
Transaction identifier for current message transaction
the local client has outstanding.
LocalPrincipal Account identification, possibly including key and key
timeout.
LocalDelivery Bit mask of segment blocks that have not been
acknowledged in the Request or have been received in the
Response, depending on the state.
ResponseQueue Queue of CSR's representing the queued Responses for
this entity.
VMTP Header Prototype VMTP header, used to generate and store the
header portion of a Request for transmission and
retransmission on timeout.
SegmentDesc Description of the segment data associated with the CSR,
either the area storing the original Request data, the
area for receiving Request data, or the area storing the
Response data that is returned.
HostAddr The network or internetwork host address to which the
Client last transmitted. This field also indicates the
type of the address, e.g. IP, Ethernet, etc.
Note: the CSR can be combined with a light-weight process descriptor
with considerable benefit if the process is designed to block when it
Cheriton [page 50]
RFC1045 VMTP February 1988
issues a message transaction. In particular, by combining the two
descriptors, the implementation saves time because it only needs to
locate and queue one descriptor with various operations (rather than
having to locate two descriptors). It also saves space, given that the
VMTP header prototype provides space such as the user data field which
may serve to store processor state for when the process is preempted.
Non-preemptive blocking can use the process stack to store the processor
state so only a program counter and stack pointer may be required in the
process descriptor beyond what we have described. (This is the approach
used in the V kernel.)
4.2. Client Protocol States
A Client State Record records the state of message transaction generated
by this host, identified by the (Client, Transaction) values in the CSR.
As a client originating a transaction, it is in one of the following
states.
AwaitingResponse
Waiting for a Response packet group to arrive with the
same (Client,Transaction) identification.
ReceivingResponse
Waiting for additional packets in the Response packet
group it is currently receiving.
"Other" Not waiting for a response, which can be Processing or
some other operating system state, or one of the Server
states if it also acts as a server.
This covers all the states for a client.
4.3. State Transition Diagrams
The client state transitions are illustrated in Figure 4-1. The client
goes into the state AwaitingResponse on sending a request unless it is a
datagram request. In the AwaitingResponse state, it can timeout and
retry and eventually give up and return to the processing state unless
it receives a Response. (A NotifyVmtpClient operation resets the
timeout but does not change the state.) On receipt of a single packet
response, it returns to the processing state. Otherwise, it goes to
ReceivingResponse state. After timeout or final response packet is
received, the client returns to the processing state. The processing
state also includes any other state besides those associated with
issuing a message transaction.
Cheriton [page 51]
RFC1045 VMTP February 1988
+------------+
| Processing |<--------------------|
| |<-------------| |
| |<---| | |
+|------^--^-+ Single Last |
Transmit | | Packet Response |
| | | Response Packet |
| | | | | |
+-DGM->+ Timeout | | Final timeout
| | | | |
+V-----------+ | +-----------+
| Awaiting |----+ | Receiving |->Response-+
| Response |->Response->| Response | |
| | (multi- | |<----------+
+-|--------^-+ packet) +----------^+
V | | |
+-Timeout+ +>Timeout+
Figure 4-1: Client State Transitions
4.4. User Interface
The RPC or user interface to VMTP is implementation-dependent and may
use systems calls, functions or some other mechanism. The list of
requests that follow is intended to suggest the basic functionality that
should be available.
Send( mcb, timeout, segptr, segsize )
Initiate a message transaction to the server and request
message specified by mcb and return a response in mcb,
if it is received within the specified timeout period
(or else return USER_TIMEOUT in the Code field). The
segptr parameter specifies the location from which the
segment data is sent and the location into which the
response data is to be delivered. The segsize field
indicates the maximum length of this area.
GetResponse( responsemcb, timeout, segptr, segsize )
Get the next response sent to this client as part of the
current message transaction, returning the segment data,
if any, into the memory specified by segptr and segsize.
This interface assumes that there is a client entity associated with the
invoking process that is to be used with these operations. Otherwise,
the client entity must be specified as an additional parameter.
Cheriton [page 52]
RFC1045 VMTP February 1988
4.5. Event Processing
The following events may occur in the VMTP client:
- User Requests
* Send
* GetResponse
- Packet Arrival
* Response Packet
* Request
The minimal Client implementation handles Request packets for
its VMTP management (server) module and sends NotifyVmtpClient
requests in response to others, indicating the specified
server does not exist.
- Management Operation - NotifyVmtpClient
- Timeouts
* Client Retransmission Timeout
The handling of these events is described in detail in the following
subsections.
We first describe some conventions and procedures used in the
description. A field of the received packet is indicated as (for
example) p.Transaction, for the Transaction field. Optional portions of
the code, such as the streaming handling code are prefixed with a "|" in
the first column.
MapClient( client )
Return pointer to CSR for client with the specified
clientId, else NULL.
SendPacketGroup( csr )
Send the packet group (Request, Response) according to
that specified by the CSR.
NotifyClient( csr, p, code )
Invoke the NotifyVmtpClient operation with the
parameters csr.RemoteClient, p.control,
Cheriton [page 53]
RFC1045 VMTP February 1988
csr.ReceiveSeqNumber, csr.RemoteTransaction and
csr.RemoteDelivery, and code. If csr is NULL, use
p.Client, p.Transaction and p.PacketDelivery instead and
the global ReceiveSequenceNumber, if supported. This
function simplifies the description over calling
NotifyVmtpClient directly in the procedural
specification below. (See Appendix III.)
NotifyServer( csr, p, code )
Invoke the NotifyVmtpServer operation with the
parameters p.Server, csr.LocalClient,
csr.LocalTransaction, csr.LocalDelivery and code. Use
p.Client, P.Transaction and 0 for the clientId, transact
and delivery parameters if csr is NULL. This function
simplifies the description over calling NotifyVmtpServer
directly in the procedural specification below. (See
Appendix III.)
DGMset(p) True if DGM bit set in packet (or csr) else False.
(Similar functions are used for other bits.)
Timeout( csr, timeperiod, func )
Set or reset timer on csr record for timeperiod later
and invoke func if the timeout expires.
4.6. Client User-invoked Events
A user event occurs when a VMTP user application invokes one of the VMTP
interface procedures.
4.6.1. Send
Send( mcb, timeout, segptr, segsize )
map to main CSR for this client.
increment csr.LocalTransaction
Init csr and check parameters and segment if any.
Set SDA if sending appended data.
Flush queued replies from previous transaction, if any.
if local non-group server then
deliver locally
await response
return
if GroupId(server) then
Check for and deliver to local members.
if CRE request and non-group local CR entity then
Cheriton [page 54]
RFC1045 VMTP February 1988
await response
return
endif
set MDG if member of this group.
endif
clear csr.RetransCount
set csr.TransmissionMask
set csr.TimeLimit to timeout
set csr.HostAddr for csr.Server
SendPacketGroup( csr )
if DGMset(csr) then
return
endif
set csr.State to AwaitingResponse
Timeout( rootcsr, TC1(csr.Server), LocalClientTimeout )
return
end Send
Notes:
1. Normally, the HostAddr is extracted from the ServerHost
cache, which maps server entity identifiers to host
addresses. However, on cache miss, the client first queries
the network using the ProbeEntity operation, as specified in
Appendix III, determining the host address from the Response.
The ProbeEntity operation is handled as a separate message
transaction by the Client.
The stream interface incorporates a parameter to pass a responseHandler
procedure that is invoked when the message transaction completes.
StreamSend( mcb, timeout, segptr, segsize, responseHandler )
map to main CSR for this client.
| Allocate a new csr if root in use.
| lastcsr := First csr for last request.
| if STIset(lastcsr)
| csr.LocalTransaction := lastcsr.LocalTransaction + 256
| else
| csr.LocalTransaction := lastcsr.LocalTransaction + 1
Init csr and check parameters and segment if any.
. . . ( rest is the same as for the normal Send)
Notes:
1. Each outstanding message transaction is represented by a CSR
queued on the root CSR for this client entity. The root CSR
is used to handle timeouts, etc. On timeout, the last packet
Cheriton [page 55]
RFC1045 VMTP February 1988
from the last packet group is retransmitted (with or without
the segment data).
4.6.2. GetResponse
GetResponse( req, timeout, segptr, segsize )
csr := CurrentCSR;
if responses queued then return next response
(in req, segptr to max of segsize )
if timeout is zero then return KERNEL_TIMEOUT error
set state to AWAITING_RESPONSE
Timeout( csr, timeout, ReturnKernelTimeout );
end GetResponse
Notes:
1. GetResponse is only used with multicast Requests, which is
the only case in which multiple (different) Responses should
be received.
2. A response must remain queued until the next message
transaction is invoked to filter out duplicates of this
response.
3. If the response is incomplete (only relevant if a
multi-packet response), then the client may wait for the
response to be fully received, including issuing requests for
retransmission (using NotifyVmtpServer operations) before
returning the response.
4. As an optimization, a response may be stored in the CSR of
the client. In this case, the response must be transferred
to a separate buffer (for duplicate suppression) before
waiting for another response. Using this optimization, a
response buffer is not allocated in the common case of the
client receiving only one response.
4.7. Packet Arrival
In general, on packet reception, a packet is mapped to the client state
record, decrypted if necessary using the key in the CSR. It then has
its checksum verified and then is transformed to the right byte order.
The packet is then processed fully relative to its packet function code.
It is discarded immediately if it is addressed to a different domain
than the domain(s) in which the receiving host participates.
Cheriton [page 56]
RFC1045 VMTP February 1988
For each of the 2 packet types, we assume a procedure called with a
pointer p to the VMTP packet and psize, the size of the packet in
octets. Thus, generic packet reception is:
if not LocalDomain(p.Domain) then return;
csr := MapClient( p.Client )
if csr is NULL then
HandleNoCsr( p, psize )
return
if Secure(p) then
if SecureVMTP not supported then
{ Assume a Request. }
if not Multicast(p) then
NotifyClient(NULL, p, SECURITY_NOT_SUPPORTED )
return
endif
| Decrypt( csr.Key, p, psize )
if p.Checksum not null then
if not VerifyChecksum(p, psize) then return;
if OppositeByteOrder(p) then ByteSwap( p, psize )
if psize not equal sizeof(VmtpHeader) + 4*p.Length then
NotifyClient(NULL, p, VMTP_ERROR )
return
Invoke Procedure[p.FuncCode]( csr, p, psize )
Discard packet and return
Notes:
1. The Procedure[p.FuncCode] refers to one of the 2 procedures
corresponding to the two different packet types of VMTP,
Requests and Responses.
2. In all the following descriptions, a packet is discarded on
"return" unless otherwise stated.
3. The procedure HandleNoCSR is a management routine that
allocates and initializes a CSR and processes the packet or
else sends an error indication to the sender of the packet.
This procedure is described in greater detail in Section
4.8.1.
Cheriton [page 57]
RFC1045 VMTP February 1988
4.7.1. Response
This procedure handles incoming Response packets.
HandleResponse( csr, p, psize )
if not LocalClient( csr ) then
if Multicast then return
| if Migrated( p.Client ) then
| NotifyServer(csr, p ENTITY_MIGRATED )
| else
NotifyServer(csr, p, ENTITY_NOT_HERE )
return
endif
if NSRset(p) then
if Streaming not supported then
NotifyServer(csr, p, STREAMING_NOT_SUPPORTED )
return STREAMED_RESPONSE
| Find csr corresponding to p.Transaction
| if none found then
| NotifyServer(csr, p, BAD_TRANSACTION_ID )
| return
else
if csr.LocalTransaction not equal p.Transaction then
NotifyServer(csr, p, BAD_TRANSACTION_ID )
return
endif
Locate reply buffer rb for this p.Server
if found then
if rb.State is not ReceivingResponse then
{ Duplicate }
if APGset(p) or NERset(p) then
{ Send Response to stop response packets. }
NotifyServer(csr, p, RESPONSE_DISCARDED )
endif
return
endif
{ rb.State is ReceivingRequest}
if new segment data then retain in CSR segment area.
if packetgroup not complete then
Timeout( rb, TC3(p.Server), LocalClientTimeout )
return;
endif
goto EndPacketGroup
endif
{ Otherwise, a new response message. }
Cheriton [page 58]
RFC1045 VMTP February 1988
if (NSRset(p) or NERset(p)) and NoStreaming then
NotifyServer(csr, p, VMTP_ERROR )
return
| if NSRset(p) then
| { Check consecutive with previous packet group }
| Find last packet group CSR from p.Server.
| if p.Transaction not
| lastcsr.RemoteTransaction+1 mod 2**32 then
| { Out of order packet group }
| NotifyServer(csr, p, BAD_TRANSACTION_ID)
| return
| endif
| if lastcsr not completed then
| NotifyServer(lastcsr, p, RETRY )
| endif
| if CMG(lastcsr) then
| Add segment data to lastcsr Response
| Notify lastcsr with new packet group.
| Clear lastcsr.VerifyInterval
| else
| if lastcsr available then
| use it for this packet group
| else allocate and initialize new CSR
| Save message and segment data in new CSR area.
| endif
| else { First packet group }
Allocate and init reply buffer rb for this response.
if allocation fails then
NotifyServer(csr, p, BUSY )
return
Set rb.State to ReceivingResponse
Copy message and segment data to rb's segment area
and set rb.PacketDelivery to that delivered.
Save p.Server host address in ServerHost cache.
endif
if packetgroup not complete then
Timeout( rb, TS1(p.Client), LocalClientTimeout )
return;
endif
endPacketGroup:
{ We have received last packet in packet group. }
if APGset(p) then NotifyServer(csr, p, OK )
| if NERset(p) and CMGset(p) then
| Queue waiting for continuation packet group.
| Timeout( rb, TC2(rb.Server), LocalClientTimeout )
| return
| endif
Cheriton [page 59]
RFC1045 VMTP February 1988
{ Deliver response message. }
Deliver response to Client, or queue as appropriate.
end HandleResponse
Notes:
1. The mechanism for handling streaming is optional and can be
replaced with the tests for use of streaming. Note that the
server should never stream at the Client unless the Client
has streamed at the Server or has used the STI control bit.
Otherwise, streamed Responses are a protocol error.
2. As an optimization, a Response can be stored into the CSR for
the Client rather than allocating a separate CSR for a
response buffer. However, if multiple responses are handled,
the code must be careful to perform duplicate detection on
the Response stored there as well as those queued. In
addition, GetResponse must create a queued version of this
Response before allowing it to be overwritten.
3. The handling of Group Responses has been omitted for brevity.
Basically, a Response is accepted if there has been a Request
received locally from the same Client and same Transaction
that has not been responded to. In this case, the Response
is delivered to the Server or queued.
Cheriton [page 60]
RFC1045 VMTP February 1988
4.8. Management Operations
VMTP uses management operations (invoked as remote procedure calls) to
effectively acknowledge packet groups and request retransmissions. The
following routine is invoked by the Client's management module on
request from the Server.
NotifyVmtpClient( clientId,ctrl,receiveSeqNumber,transact,delivery,code)
Get csr for clientId
if none then return
if RemoteClient( csr ) and not NotifyVmtpRemoteClient then
return
| else (for streaming)
| Find csr with same LocalTransaction as transact
| if csr is NULL then return
if csr.State not AwaitingResponse then return
if ctrl.PGcount then ack previous packet groups.
select on code
case OK:
Notify ack'ed segment blocks from delivery
Clear csr.RetransCount;
Timeout( csr, TC1(csr.Server), LocalClientTimeout )
return
case RETRY:
Set csr.TransmissionMask to missing segment blocks,
as specified by delivery
SendPacketGroup( csr )
Timeout( csr, TC1(csr.Server), LocalClientTimeout )
case RETRY_ALL
Set csr.TransmissionMask to retransmit all blocks.
SendPacketGroup( csr )
Timeout( csr, TC1(csr.Server), LocalClientTimeout )
| if streaming then
| Restart transmission of packet groups,
| starting from transact+1
return
case BUSY:
if csr.TimeLimit exceeded then
Set csr.Code to USER_TIMEOUT
return Response to application
return;
Set csr.TransmissionMask for full retransmission
Clear csr.RetransCount
Timeout( csr, TC1(csr.Server), LocalClientTimeout )
return
case ENTITY_MIGRATED:
Get new host address for entity
Cheriton [page 61]
RFC1045 VMTP February 1988
Set csr.TransmissionMask for full retransmission
Clear csr.RetransCount
SendPacketGroup( csr )
Timeout( csr, TC1(csr.Server), LocalClientTimeout )
return
case STREAMING_NOT_SUPPORTED:
Record that server does not support streaming
if CMG(csr) then forget this packet group
else resend Request as separate packet group.
return
default:
Set csr.Code to code
return Response to application
return;
endselect
end NotifyVmtpClient
Notes:
1. The delivery parameter indicates the segment blocks received
by the Server. That is, a 1 bit in the i-th position
indicates that the i-th segment block in the segment data of
the Request was received. All subsequent NotifyVmtpClient
operations for this transaction should be set to acknowledge
a superset of the segment blocks in this packet. In
particular, the Client need not be prepared to retransmit the
segment data once it has been acknowledged by a Notify
operation.
4.8.1. HandleNoCSR
HandleNoCSR is called when a packet arrives for which there is no CSR
matching the client field of the packet.
HandleNoCSR( p, psize )
if Secure(p) then
if SecureVMTP not supported then
{ Assume a Request }
if not Multicast(p) then
NotifyClient(NULL,p,SECURITY_NOT_SUPPORTED)
return
endif
HandleRequestNoCSR( p, psize )
return
endif
Cheriton [page 62]
RFC1045 VMTP February 1988
if p.Checksum not null then
if not VerifyChecksum(p, psize) then return;
if OppositeByteOrder(p) then ByteSwap( p, psize )
if psize not equal sizeof(VmtpHeader) + 4*p.Length then
NotifyClient(NULL, p, VMTP_ERROR )
return
if p.FuncCode is Response then
| if Migrated( p.Client ) then
| NotifyServer(csr, p ENTITY_MIGRATED )
| else
NotifyServer(csr, p, NONEXISTENT_ENTITY )
return
endif
if p.FuncCode is Request then
HandleRequestNoCSR( p, psize )
return
end HandleNoCSR
Notes:
1. The node need only check to see if the client entity has
migrated if in fact it supports migration of entities.
2. The procedure HandleRequestNoCSR is specified in Section
5.8.1. In the minimal client version, it need only handle
Probe requests and can do so directly without allocating a
new CSR.
Cheriton [page 63]
RFC1045 VMTP February 1988
4.9. Timeouts
A client with a message transaction in progress has a single timer
corresponding to the first unacknowledged request message. (In the
absence of streaming, this request is also the last request sent.) This
timeout is handled as follows:
LocalClientTimeout( csr )
select on csr.State
case AwaitingResponse:
if csr.RetransCount > MaxRetrans(csr.Server) then
terminate Client's message transactions up to
and including the current message transaction.
set return code to KERNEL_TIMEOUT
return
increment csr.RetransCount
Resend current packet group with APG set.
Timeout( csr, TC2(csr.Server), LocalClientTimeout )
return
case ReceivingResponse:
if DGMset(csr) or csr.RetransCount > Max then
if MDMset(csr) then
Set MCB.MsgDeliveryMask to blocks received.
else
Set csr.Code to BAD_REPLY_SEGMENT
return to user Client
endif
increment csr.RetransCount
NotifyServer with RETRY
Timeout( csr, TC3(csr.Server), LocalClientTimeout )
return
end select
end LocalClientTimeout
Notes:
1. A Client can only request retransmission of a Response if the
Response is not idempotent. If idempotent, it must
retransmit the Request. The Server should generally support
the MsgDeliveryMask for Requests that it treats as idempotent
and that require multi-packet Responses. Otherwise, there is
no selective retransmission for idempotent message
transactions.
2. The current packet group is the last one transmitted. Thus,
with streaming, there may be several packet groups
outstanding that precede the current packet group.
Cheriton [page 64]
RFC1045 VMTP February 1988
3. The Request packet group should be retransmitted without the
segment data, resulting in a single short packet in the
retransmission. The Server must then send a
NotifyVmtpClient with a RETRY or RETRY_ALL code to get the
segment data transmitted as needed. This strategy minimizes
the overhead on the network and the server(s) for
retransmissions.
Cheriton [page 65]
RFC1045 VMTP February 1988
5. Server Protocol Operation
This section describes the operation of the server portion of the
protocol in terms of the procedures for handling VMTP user events,
packet reception events and timeout events. Each server is assumed to
implement the client procedures described in the previous chapter.
(This is not strictly necessary but it simplifies the exposition.)
5.1. Remote Client State Record Fields
The CSR for a server is extended with the following fields, in addition
to the ones listed for the client version.
RemoteClient Identifier for remote client that sent the Request that
this CSR is handling.
RemoteClientLink
Link to next CSR hashing to same hash index in the
ClientMap.
RemoteTransaction
Transaction identifier for Request from remote client.
RemoteDelivery The segment blocks received so far as part of a Request
or yet to be acknowledged as part of a Response.
VerifyInterval Time interval since there was confirmation that the
remote Client was still valid.
RemotePrincipal Account identification, possibly including key and key
timeout for secure communication.
5.2. Remote Client Protocol States
A CSR in the server end is in one of the following states.
AwaitingRequest Waiting for a Request packet group. It may be marked as
waiting on a specific Client, or on any Client.
ReceivingRequest
Waiting to receive additional Request packets in a
multi-packet group Request.
Responded The Response has been sent and the CSR is timing out,
providing duplicate suppression and retransmission (if
Cheriton [page 66]
RFC1045 VMTP February 1988
the Response was not idempotent).
ResponseDiscarded
Response has been acknowledged or has timed out so
cannot be retransmitted. However, duplicates are still
filtered and CSR can be reused for new message
transaction.
Processing Executing on behalf of the Client.
Forwarded The message transaction has been forwarded to another
Server that is to respond directly to the Client.
5.3. State Transition Diagrams
The CSR state transitions in the server are illustrated in Figure 5-1.
The CSR generally starts in the AwaitingRequest state. On receipt of a
Request, the Server either has an up-to-date CSR for the Client or else
it sends a Probe request (as a separate VMTP message transaction) to the
VMTP management module associated with the Client. In the latter case,
the processing of the Request is delayed until a Response to the Probe
request is received. At that time, the CSR information is brought up to
date and the Request is processed. If the Request is a single-packet
request, the CSR is then set in the Processing state to handle the
request. Otherwise (a multi-packet Request), the CSR is put into the
ReceivingResponse state, waiting to receive subsequent Request packets
that constitute the Request message. It exits the ReceivingRequest
state on timeout or on receiving the last Request packet. In the former
case, the request is delivered with an indication of the portion
received, using the MsgDelivery field if MDM is set. After request
processing is complete, either the Response is sent and the CSR enters
the Responded state or the message transaction is forwarded and the CSR
enters the Forwarded state.
In the Responded state, if the Response is not marked as idempotent, the
Response is retransmitted on receipt of a retransmission of the
corresponding Request, on receipt of a NotifyVmtpServer operation
requesting retransmission or on timeout at which time APG is set,
requesting an acknowledgment from the Client. The Response is
retransmitted some maximum number of times at which time the Response is
discarded and the CSR is marked accordingly. If a Request or a
NotifyVmtpServer operation is received expecting retransmission of the
Response after the CSR has entered the ResponseDiscarded state, a
NotifyVmtpClient operation is sent back (or invoked in the Client
management module) indicating that the response was discarded unless the
Request was multicast, in which case no action is taken. After a
Cheriton [page 67]
RFC1045 VMTP February 1988
(Retransmit Forwarded Request and NotifyVmtpClient)
Request/
Ack/
+Timeout+
V |
+-|-------^-+
| |
+-Time-| Forwarded |<-------------+
| out +-----------+ |
| |
| (Retransmit Response) |
| Request |
V Ack |
| +-Timeout-+ |
| V | |
+---------+ Ack/ +|---------^+ |
+-Time-|Response |<-Timeout--| Responded | |
| out |Discarded| +----^------+ |
| +---------+ | |
| +------------+ | |
| | |->-Send Response-+ |
| | |->-forward Request--------+
+->| Processing |<----------------------+
| | |<----------------+ |
| | |<---| | |
| +-|--------^-+ | Last |
| Receive | | Request |
| | Timeout Single Packet |
| | | Packet | Timeout
| | | Request ^ ^
| | | ^ +|-----|--+
| +-V--------|-+ | |Receiving|<-+Time
+->| Awaiting |->--+->Request->| Request |--+ out
| Request | | (multi- +---------+
+------|-----+ ^ packet)
Request |
| Response
Send Probe to
| Probe
+---V----+ |
|Awaiting| ^
|Response|-->--+
|to Probe|
+--------+
Figure 5-1: Remote Client State Transitions
timeout corresponding to the time required to filter out duplicates, the
Cheriton [page 68]
RFC1045 VMTP February 1988
CSR returns either to the AwaitingRequest state or to the Processing
state. Note that "Ack" refers to acknowledgment by a Notify operation.
A Request that is forwarded leaves the CSR in the Forwarded state. In
the Forwarded state, the forwarded Request is retransmitted
periodically, expecting NotifyRemoteClient operations back from the
Server to which the Request was forwarded, analogous to the Client
behavior in the AwaitingResponse state. In this state, a