determine if a record in one file occurred before or after a record
in a different file. This may be somewhat alleviated by placing
marking messages with a timestamp into all destination files. If
these have coordinated timestamps, then there will be some indication
of the time of receipt of the individual messages.
6.3.4 Replaying
Without any sequence indication or timestamp, messages may be
recorded and replayed at a later time. An attacker may record a set
of messages that indicate normal activity of a machine. At a later
time, that attacker may remove that machine from the network and
replay the syslog messages to the collector. Even with a TIMESTAMP
field in the HEADER part, an attacker may record the packets and
could simply modify them to reflect the current time before
retransmitting them. The administrators may find nothing unusual in
the received messages and their receipt would falsely indicate normal
activity of the machine.
6.4 Reliable Delivery
As there is no mechanism within either the syslog process or the
protocol to ensure delivery, and since the underlying transport is
UDP, some messages may be lost. They may either be dropped through
network congestion, or they may be maliciously intercepted and
discarded. The consequences of the drop of one or more syslog
messages cannot be determined. If the messages are simple status
updates, then their non-receipt may either not be noticed, or it may
cause an annoyance for the system operators. On the other hand, if
the messages are more critical, then the administrators may not
become aware of a developing and potentially serious problem.
Messages may also be intercepted and discarded by an attacker as a
way to hide unauthorized activities.
6.5 Message Integrity
Besides being discarded, syslog messages may be damaged in transit,
or an attacker may maliciously modify them. In the case of a packet
containing a syslog message being damaged, there are various
mechanisms built into the link layer as well as into the IP [9] and
UDP protocols which may detect the damage. An intermediary router
may discard a damaged IP packet [10]. Damage to a UDP packet may be
detected by the receiving UDP module, which may silently discard it.
In any case, the original contents of the message will not be
delivered to the collector. Additionally, if an attacker is
positioned between the sender and collector of syslog messages, they
may be able to intercept and modify those messages while in-transit
to hide unauthorized activities.
6.6 Message Observation
While there are no strict guidelines pertaining to the event message
format, most syslog messages are generated in human readable form
with the assumption that capable administrators should be able to
read them and understand their meaning. Neither the syslog protocol
nor the syslog application have mechanisms to provide confidentiality
of the messages in transit. In most cases passing clear-text
messages is a benefit to the operations staff if they are sniffing
the packets off of the wire. The operations staff may be able to
read the messages and associate them with other events seen from
other packets crossing the wire to track down and correct problems.
Unfortunately, an attacker may also be able to observe the human-
readable contents of syslog messages. The attacker may then use the
knowledge gained from those messages to compromise a machine or do
other damage.
6.7 Message Prioritization and Differentiation
While the processes that create the messages may signify the
importance of the events through the use of the message Priority
value, there is no distinct association between this value and the
importance of delivery of the packet. As an example of this,
consider an application that generates two event messages. The first
is a normal status message but the second could be an important
message denoting a problem with the process. This second message
would have an appropriately higher Severity value associated with the
importance of that event. If the operators had configured that both
of these messages be transported to a syslog collector then they
would, in turn, be given to UDP for transmission. Under normal
conditions, no distinction would be made between them and they would
be transmitted in their order.
Again, under normal circumstances, the receiver would accept syslog
messages as they are received. If many devices are transmitting
normal status messages, but one is transmitting an important event
message, there is no inherent mechanism within the syslog protocol to
prioritize the important message over the other messages.
On a case-by-case basis, device operators may find some way to
associate the different levels with the quality of service
identifiers. As an example, the operators may elect to define some
linkage between syslog messages that have a specific Priority value
with a specific value to be used in the IPv4 Precedence field [9],
the IPv6 Traffic Class octet [11], or the Differentiated Services
field [12]. In the above example, the operators may have the ability
to associate the status message with normal delivery while
associating the message indicating a problem with a high reliability,
low latency queue as it goes through the network. This would have
the affect of prioritizing the essential messages before the normal
status messages. Even with this hop-by-hop prioritization, this
queuing mechanism could still lead to head of line blocking on the
transmitting device as well as buffer starvation on the receiving
device if there are many near-simultaneous messages being sent or
received. This behavior is not unique to syslog but is endemic to
all operations that transmit messages serially.
There are security concerns for this behavior. Head of line blocking
of the transmission of important event messages may relegate the
conveyance of important messages behind less important messages. If
the queue is cleared appropriately, this may only add seconds to the
transmission of the important message. On the other hand, if the
queue is not cleared, then important messages may not be transmitted.
Also at the receiving side, if the syslog receiver is suffering from
buffer starvation due to large numbers of messages being received
near-simultaneously, important messages may be dropped
indiscriminately along with other messages. While these are problems
with the devices and their capacities, the protocol security concern
is that there is no prioritization of the relatively more important
messages over the less important messages.
6.8 Misconfiguration
Since there is no control information distributed about any messages
or configurations, it is wholly the responsibility of the network
administrator to ensure that the messages are actually going to the
intended recipient. Cases have been noted where devices were
inadvertently configured to send syslog messages to the wrong
receiver. In many cases, the inadvertent receiver may not be
configured to receive syslog messages and it will probably discard
them. In certain other cases, the receipt of syslog messages has
been known to cause problems for the unintended recipient [13]. If
messages are not going to the intended recipient, then they cannot be
reviewed or processed.
6.9 Forwarding Loop
As it is shown in Figure 1, machines may be configured to relay
syslog messages to subsequent relays before reaching a collector. In
one particular case, an administrator found that he had mistakenly
configured two relays to forward messages with certain Priority
values to each other. When either of these machines either received
or generated that type of message, it would forward it to the other
relay. That relay would, in turn, forward it back. This cycle did
cause degradation to the intervening network as well as to the
processing availability on the two devices. Network administrators
must take care to not cause such a death spiral.
6.10 Load Considerations
Network administrators must take the time to estimate the appropriate
size of the syslog receivers. An attacker may perform a Denial of
Service attack by filling the disk of the collector with false
messages. Placing the records in a circular file may alleviate this
but that has the consequence of not ensuring that an administrator
will be able to review the records in the future. Along this line, a
receiver or collector must have a network interface capable of
receiving all messages sent to it.
Administrators and network planners must also critically review the
network paths between the devices, the relays, and the collectors.
Generated syslog messages should not overwhelm any of the network
links.
7. IANA Considerations
The syslog protocol has been assigned UDP port 514. This port
assignment will be maintained by IANA exclusively for this protocol.
The syslog protocol provides for the definition of named attributes
to indicate the Severity of each message and the Facility that
generated the message as described in Section 4. The name space
identifiers for these attributes are defined as numbers. The
protocol does not define the specific assignment of the name space
for these numbers; the application developer or system vendor is
allowed to define the attribute, its semantics, and the associated
numbers. This name space will not be controlled to prevent
collisions as systems are expected to use the same attributes,
semantics and associated numbers to describe events that are deemed
similar even between heterogeneous devices.
8. Conclusion and Other Efforts
The syslog protocol may be effectively used to transport event
notification messages across a network. In all cases, it is
important that the syslog message receiver embody the principle of
"be liberal in what you accept". It is highly recommended that the
network operators who choose to use this understand the
characteristics of the protocol and its security implications.
There have been attempts in the past to standardize the format of the
syslog message. The most notable attempt culminated in a BOF at the
Fortieth Internet Engineering Task Force meeting in 1997. This was
the Universal Logging Protocol (ulp) BOF and the minutes of their
meeting are on-line at the IETF Proceedings web site [14].
Many good thoughts came from that effort and interested implementers
may want to find some of the notes or papers produced from that
effort.
At the time of this writing, efforts are underway to allow the usage
of international character sets in applications that have been
traditionally thought of as being text-only. The HOSTNAME and
TIMESTAMP fields described above are representative of this. Also,
the entire CONTENT field has traditionally been printing characters
and spaces in the code set known as US-ASCII. It is hoped that the
proponents of these internationalization efforts will find a suitable
way to allow the use of international character sets within syslog
messages without being disruptive. It should also be hoped that
implementers will allow for the future acceptance of additional code
sets and that they may make appropriate plans. Again, it must be
cautioned that the simplicity of the existing system has been a
tremendous value to its acceptance. Anything that lessens that
simplicity may diminish that value.
Acknowledgements
The following people provided content feedback during the writing of
this document:
Jon Knight <J.P.Knight@lboro.ac.uk>
Magosanyi Arpad <mag@bunuel.tii.matav.hu>
Balazs Scheidler <bazsi@balabit.hu>
Jon Callas <jon@counterpane.com>
Eliot Lear <lear@cisco.com>
Petter Reinholdtsen <pere@hungry.com>
Darren Reed <darrenr@reed.wattle.id.au>
Alfonso De Gregorio <dira@speedcom.it>
Eric Allman <eric@sendmail.com>
Andrew Ross <andrew@kiwi-enterprises.com>
George Maslyar <george.maslyar@primark.com>
Albert Mietus <albert@ons-huis.net>
Russ Allbery <rra@stanford.edu>
Titus D. Winters <titus@cs.hmc.edu>
Edwin P. Boon <Edwin.Boon@consul.com>
Jeroen M. Mostert <Jeroen.Mostert@consul.com>
Eric Allman is the original inventor and author of the syslog daemon
and protocol. The author of this memo and the community at large
would like to express their appreciation for this work and for the
usefulness that it has provided over the years.
A large amount of additional information about this de-facto standard
operating system feature may usually be found in the syslog.conf file
as well as in the man pages for syslog.conf, syslog, syslogd, and
logger, of many Unix and Unix-like devices.
References
1 Postel, J., "User Datagram Protocol", STD 6, RFC768, August 1980.
2 Crocker, D. and P. Overell, "Augmented BNF for Syntax
Specifications: ABNF", RFC2234, November 1997.
3 USA Standard Code for Information Interchange, USASI X3.4-1968
4 Mockapetris, P., "Domain Names - Concepts and Facilities", STD 13,
RFC1034, November 1987.
5 Mockapetris, P., "Domain names - Implementation and
Specification", STD 13, RFC1035, November 1987.
6 Hinden, R. and S. Deering, "IP Version 6 Addressing Architecture",
RFC2373, July 1998.
7 Data elements and interchange formats - Information exchange -
Representation of dates and times, International Organization for
Standardization, Reference number ISO 8601 : 1988 (E), 1988
8 Stowe, M., et al, "Chemical Mimicry: Bolas Spiders Emit Components
of Moth Prey Species Sex Pheromones", Science, 1987
9 Postel, J., "Internet Protocol", STD 5, RFC791, September 1981.
10 Baker, F., "Requirements for IP Version 4 Routers", RFC1812, June
1995.
11 Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6)
Specification", RFC2460, December 1998.
12 Nichols, K., Blake, S., Baker, F. and D. Black, "Definition of the
Differentiated Services Field (DS Field) in the IPv4 and IPv6
Headers", RFC2474, December 1998.
13 Cisco Systems Product Security Incident Response Team (PSIRT),
"Field Notice: Cisco IOS(r) Syslog Crash", January 11, 1999
http://www.cisco.com/warp/public/707/advisory.html
14 Walker, D., IETF Secretariat, "Proceedings of the Fortieth
Internet Engineering Task Force, Washington, DC, USA, December 8-
12, 1997
http://www.ietf.org/proceedings/97dec/index.html
Author's Address
Chris Lonvick
Cisco Systems
12515 Research Blvd.
Austin, TX, USA
Phone: +1.512.378.1182
EMail: clonvick@cisco.com
Full Copyright Statement
Copyright (C) The Internet Society (2001). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFCEditor function is currently provided by the
Internet Society.