| 20 kbps | 1 | 2 | 1,2 | - | 0.58 | 0.73 | 1.31 |
| 20 kbps | 1,2 | - | 1,2 | - | 0.86 | 0.84 | 1.70 |
Table 2: Unicast simulations with packet loss
4.2. Multicast
Next, we investigated the RTCP bandwidth share in multicast
scenarios; i.e., we simulated the topologies T-4, T-8, and T-16 and
measured the fraction of the session bandwidth that was used for RTCP
packets. Again we considered different situations and protocol
configurations (e.g., with or without bit errors, groups with AVP
and/or AVPF agents, etc.). For reasons of readability, we present
only selected results. For a documentation of all results, see [5].
The simulations of the different topologies in scenarios where no
losses occur (neither through bit errors nor through congestion) show
a similar behavior as in the unicast case. For all group sizes, the
maximum RTCP bit rate share used is 5.06% of the session bandwidth in
a simulation of 16 session members in a low-bit-rate scenario
(session bandwidth = 20 kbps) with several senders. In all other
scenarios without losses, the RTCP bit rate share used is below that.
Thus, the requirement that not more than 5% of the session bit rate
should be used for RTCP is fulfilled with reasonable accuracy.
Simulations where bit errors are randomly inserted in RTP and RTCP
packets and the corrupted packets are discarded give the same
results. The 5% rule is kept (at maximum 5.07% of the session
bandwidth is used for RTCP).
Finally, we conducted simulations where we reduced the link bandwidth
and thereby caused congestion-related losses. These simulations are
different from the previous bit error simulations, in that the losses
occur more in bursts and are more correlated, also between different
agents. The correlation and "burstiness" of the packet loss is due
to the queuing discipline in the routers we simulated; we used simple
FIFO queues with a drop-tail strategy to handle congestion. Random
Early Detection (RED) queues may enhance the performance, because the
burstiness of the packet loss might be reduced; however, this is not
the subject of our investigations, but is left for future study. The
delay between the agents, which also influences RTP and RTCP packets,
is much more variable because of the added queuing delay. Still the
RTCP bit rate share used does not increase beyond 5.09% of the
session bandwidth. Thus, also for these special cases the
requirement is fulfilled.
4.3. Summary of the RTCP Bit Rate Measurements
We have shown that for unicast and reasonable multicast scenarios,
feedback implosion does not happen. The requirement that at maximum
5% of the session bandwidth is used for RTCP is fulfilled for all
investigated scenarios.
5. Feedback Measurements
In this section we describe the results of feedback delay
measurements, which we conducted in the simulations. Therefore, we
use two metrics for measuring the performance of the algorithms;
these are the "mean waiting time" (MWT) and the number of feedback
packets that are sent, suppressed, or not allowed. The waiting time
is the time, measured at a certain agent, between the detection of a
packet loss event and the time when the corresponding feedback is
sent. Assuming that the value of the feedback decreases with its
delay, we think that the mean waiting time is a good metric to
measure the performance gain we could get by using AVPF instead of
AVP.
The feedback an RTP/AVPF agent wants to send can be either sent or
not sent. If it was not sent, this could be due to feedback
suppression (i.e., another receiver already sent the same feedback)
or because the feedback was not allowed (i.e., the max_feedback_delay
was exceeded). We traced for every detected loss, if the agent sent
the corresponding feedback or not and if not, why. The more feedback
was not allowed, the worse the performance of the algorithm.
Together with the waiting times, this gives us a good hint of the
overall performance of the scheme.
5.1. Unicast
In the unicast case, the maximum dithering interval T_dither_max is
fixed and set to zero. This is because it does not make sense for a
unicast receiver to wait for other receivers if they have the same
feedback to send. But still feedback can be delayed or might not be
permitted to be sent at all. The regularly scheduled packets are
spaced according to T_rr, which depends in the unicast case mainly on
the session bandwidth.
Table 3 shows the mean waiting times (MWTs) measured in seconds for
some configurations of the unicast topology T-2. The number of
feedback packets that are sent or discarded is listed also (feedback
sent (sent) or feedback discarded (disc)). We do not list suppressed
packets, because for the unicast case feedback suppression does not
apply. In the simulations, agent A1 was a sender and agent A2 was a
pure receiver.
| | | Feedback Statistics |
| Session | | AVP | AVPF |
|Bandwidth| PLR | sent |disc| MWT | sent |disc| MWT |
+---------+-------+------+----+-------+------+----+-------+
| 2 Mbps | 0.001 | 781 | 0 | 2.604 | 756 | 0 | 0.015 |
| 2 Mbps | 0.01 | 7480 | 0 | 2.591 | 7548 | 2 | 0.006 |
| 2 Mbps | cong. | 25 | 0 | 2.557 | 1741 | 0 | 0.001 |
| 20 kbps | 0.001 | 79 | 0 | 2.472 | 74 | 2 | 0.034 |
| 20 kbps | 0.01 | 780 | 0 | 2.605 | 709 | 64 | 0.163 |
| 20 kbps | cong. | 780 | 0 | 2.590 | 687 | 70 | 0.162 |
Table 3: Feedback statistics for the unicast simulations
From the table above we see that the mean waiting time can be
decreased dramatically by using AVPF instead of AVP. While the
waiting times for agents using AVP is always around 2.5 seconds (half
the minimum interval average), it can be decreased to a few ms for
most of the AVPF configurations.
In the configurations with high session bandwidth, normally all
triggered feedback is sent. This is because more RTCP bandwidth is
available. There are only very few exceptions, which are probably
due to more than one packet loss within one RTCP interval, where the
first loss was by chance sent quite early. In this case, it might be
possible that the second feedback is triggered after the early packet
was sent, but possibly too early to append it to the next regularly
scheduled report, because of the limitation of the
max_feedback_delay. This is different for the cases with a small
session bandwidth, where the RTCP bandwidth share is quite low and
T_rr thus larger. After an early packet was sent, the time to the
next regularly scheduled packet can be very high. We saw that in
some cases the time was larger than the max_feedback_delay, and in
these cases the feedback is not allowed to be sent at all.
With a different setting of max_feedback_delay, it is possible to
have either more feedback that is not allowed and a decreased mean
waiting time or more feedback that is sent but an increased waiting
time. Thus, the parameter should be set with care according to the
application’s needs.
5.2. Multicast
In this section, we describe some measurements of feedback statistics
in the multicast simulations. We picked out certain characteristic
and representative results. We considered the topology T-16.
Different scenarios and applications are simulated for this topology.
The parameters of the different links are set as follows. The agents
A2, A3, and A4 are connected to the middle node of the multicast
tree, i.e., agent A1, via high bandwidth and low-delay links. The
other agents are connected to the nodes 2, 3, and 4 via different
link characteristics. The agents connected to node 2 represent
mobile users. They suffer in certain configurations from a certain
byte error rate on their access links and the delays are high. The
agents that are connected to node 3 have low-bandwidth access links,
but do not suffer from bit errors. The last agents, which are
connected to node 4, have high bandwidth and low delay.
5.2.1. Shared Losses vs. Distributed Losses
In our first investigation, we wanted to see the effect of the loss
characteristic on the algorithm’s performance. We investigate the
cases where packet loss occurs for several users simultaneously
(shared losses) or totally independently (distributed losses). We
first define agent A1 to be the sender. In the case of shared
losses, we inserted a constant byte error rate on one of the middle
links, i.e., the link between A1 and A2. In the case of distributed
losses, we inserted the same byte error rate on all links downstream
of A2.
These scenarios are especially interesting because of the feedback
suppression algorithm. When all receivers share the same loss, it is
only necessary for one of them to send the loss report. Hence if a
member receives feedback with the same content that it has scheduled
to be sent, it suppresses the scheduled feedback. Of course, this
suppressed feedback does not contribute to the mean waiting times.
So we expect reduced waiting times for shared losses, because the
probability is high that one of the receivers can send the feedback
more or less immediately. The results are shown in the following
table.
| | Feedback Statistics |
| | Shared Losses | Distributed Losses |
|Agent|sent|fbsp|disc|sum | MWT |sent|fbsp|disc|sum | MWT |
+-----+----+----+----+----+-----+----+----+----+----+-----+
| A2 | 274| 351| 25| 650|0.267| -| -| -| -| -|
| A5 | 231| 408| 11| 650|0.243| 619| 2| 32| 653|0.663|
| A6 | 234| 407| 9| 650|0.235| 587| 2| 32| 621|0.701|
| A7 | 223| 414| 13| 650|0.253| 594| 6| 41| 641|0.658|
| A8 | 188| 443| 19| 650|0.235| 596| 1| 32| 629|0.677|
Table 4: Feedback statistics for multicast simulations
Table 4 shows the feedback statistics for the simulation of a large
group size. All 16 agents of topology T-16 joined the RTP session.
However, only agent A1 acts as an RTP sender; the other agents are
pure receivers. Only 4 or 5 agents suffer from packet loss, i.e.,
A2, A5, A6, A7, and A8 for the case of shared losses and A5, A6, A7,
and A8 in the case of distributed losses. Since the number of
session members is the same for both cases, T_rr is also the same on
the average. Still the mean waiting times are reduced by more than
50% in the case of shared losses. This proves our assumption that
shared losses enhance the performance of the algorithm, regardless of
the loss characteristic.
The feedback suppression mechanism seems to be working quite well.
Even though some feedback is sent from different receivers (i.e.,
1150 loss reports are sent in total and only 650 packets were lost,
resulting in loss reports being received on the average 1.8 times),
most of the redundant feedback was suppressed. That is, 2023 loss
reports were suppressed from 3250 individual detected losses, which
means that more than 60% of the feedback was actually suppressed.
6. Investigations on "l"
In this section, we want to investigate the effect of the parameter
"l" on the T_dither_max calculation in RTP/AVPF agents. We
investigate the feedback suppression performance as well as the
report delay for three sample scenarios.
For all receivers, the T_dither_max value is calculated as
T_dither_max = l * T_rr, with l = 0.5. The rationale for this is
that, in general, if the receiver has no round-trip time (RTT)
estimation, it does not know how long it should wait for other
receivers to send feedback. The feedback suppression algorithm would
certainly fail if the time selected is too short. However, the
waiting time is increased unnecessarily (and thus the value of the
feedback is decreased) in case the chosen value is too large.
Ideally, the optimum time value could be found for each case, but
this is not always feasible. On the other hand, it is not dangerous
if the optimum time is not used. A decreased feedback value and a
failure of the feedback suppression mechanism do not hurt the network
stability. We have shown for the cases of distributed losses that
the overall bandwidth constraints are kept in any case and thus we
could only lose some performance by choosing the wrong time value.
On the other hand, a good measure for T_dither_max is the RTCP
interval T_rr. This value increases with the number of session
members. Also, we know that we can send feedback at least every
T_rr. Thus, increasing T_dither max beyond T_rr would certainly make
no sense. So by choosing T_rr/2, we guarantee that at least
sometimes (i.e., when a loss is detected in the first half of the
interval between two regularly scheduled RTCP packets) we are allowed
to send early packets. Because of the randomness of T_dither, we
still have a good chance of sending the early packet in time.
The AVPF profile specifies that the calculation of T_dither_max, as
given above, is common to session members having an RTT estimation
and to those not having it. If this were not so, participants using
different calculations for T_dither_max might also have very
different mean waiting times before sending feedback, which
translates into different reporting priorities. For example, in a
scenario where T_rr = 1 s and the RTT = 100 ms, receivers using the
RTT estimation would, on average, send more feedback than those not
using it. This might partially cancel out the feedback suppression
mechanism and even cause feedback implosion. Also note that, in a
general case where the losses are shared, the feedback suppression
mechanism works if the feedback packets from each receiver have
enough time to reach each of the other ones before the calculated
T_dither_max seconds. Therefore, in scenarios of very high bandwidth
(small T_rr), the calculated T_dither_max could be much smaller than
the propagation delay between receivers, which would translate into a
failure of the feedback suppression mechanism. In these cases, one
solution could be to limit the bandwidth available to receivers (see
[10]) such that this does not happen. Another solution could be to
develop a mechanism for feedback suppression based on the RTT
estimation between senders. This will not be discussed here and may
be the subject of another document. Note, however, that a really
high bandwidth media stream is not that likely to rely on this kind
of error repair in the first place.
In the following, we define three representative sample scenarios.
We use the topology from the previous section, T-16. Most of the
agents contribute only little to the simulations, because we
introduced an error rate only on the link between the sender A1 and
the agent A2.
The first scenario represents those cases, where losses are shared
between two agents. One agent is located upstream on the path
between the other agent and the sender. Therefore, agent A2 and
agent A5 see the same losses that are introduced on the link between
the sender and agent A2. Agents A6, A7, and A8 do not join the RTP
session. From the other agents, only agents A3 and A9 join. All
agents are pure receivers, except A1, which is the sender.
The second scenario also represents cases where losses are shared
between two agents, but this time the agents are located on different
branches of the multicast tree. The delays to the sender are roughly
of the same magnitude. Agents A5 and A6 share the same losses.
Agents A3 and A9 join the RTP session, but are pure receivers and do
not see any losses.
Finally, in the third scenario, the losses are shared between two
agents, A5 and A6. The same agents as in the second scenario are
active. However, the delays of the links are different. The delay
of the link between agents A2 and A5 is reduced to 20 ms and between
A2 and A6 to 40 ms.
All agents beside agent A1 are pure RTP receivers. Thus, these
agents do not have an RTT estimation to the source. T_dither_max is
calculated with the above given formula, depending only on T_rr and
l, which means that all agents should calculate roughly the same
T_dither_max.
6.1. Feedback Suppression Performance
The feedback suppression rate for an agent is defined as the ratio of
the total number of feedback packets not sent out of the total number
of feedback packets the agent intended to send (i.e., the sum of sent
and not sent). The reasons for not sending a packet include: the
receiver already saw the same loss reported in a receiver report
coming from another session member or the max_feedback_delay
(application-specific) was surpassed.
The results for the feedback suppression rate of the agent Af that is
further away from the sender are depicted in Table 5. In general, it
can be seen that the feedback suppression rate increases as l
increases. However there is a threshold, depending on the
environment, from which the additional gain is not significant
anymore.
| | Feedback Suppression Rate |
| l | Scen. 1 | Scen. 2 | Scen. 3 |
+------+---------+---------+---------+
| 0.10 | 0.671 | 0.051 | 0.089 |
| 0.25 | 0.582 | 0.060 | 0.210 |
| 0.50 | 0.524 | 0.114 | 0.361 |
| 0.75 | 0.523 | 0.180 | 0.370 |
| 1.00 | 0.523 | 0.204 | 0.369 |
| 1.25 | 0.506 | 0.187 | 0.372 |
| 1.50 | 0.536 | 0.213 | 0.414 |
| 1.75 | 0.526 | 0.215 | 0.424 |
| 2.00 | 0.535 | 0.216 | 0.400 |
| 3.00 | 0.522 | 0.220 | 0.405 |
| 4.00 | 0.522 | 0.220 | 0.405 |
Table 5: Fraction of feedback that was suppressed at agent (Af) of
the total number of feedback messages the agent wanted to send
Similar results can be seen in Table 6 for the agent An that is
nearer to the sender.
| | Feedback Suppression Rate |
| l | Scen. 1 | Scen. 2 | Scen. 3 |
+------+---------+---------+---------+
| 0.10 | 0.056 | 0.056 | 0.090 |
| 0.25 | 0.063 | 0.055 | 0.166 |
| 0.50 | 0.116 | 0.099 | 0.255 |
| 0.75 | 0.141 | 0.141 | 0.312 |
| 1.00 | 0.179 | 0.175 | 0.352 |
| 1.25 | 0.206 | 0.176 | 0.361 |
| 1.50 | 0.193 | 0.193 | 0.337 |
| 1.75 | 0.197 | 0.204 | 0.341 |
| 2.00 | 0.207 | 0.207 | 0.368 |
| 3.00 | 0.196 | 0.203 | 0.359 |
| 4.00 | 0.196 | 0.203 | 0.359 |
Table 6: Fraction of feedback that was suppressed at agent (An) of
the total number of feedback messages the agent wanted to send
The rate of feedback suppression failure is depicted in Table 7. The
trend of additional performance increase is not significant beyond a
certain threshold. Dependence on the scenario is noticeable here as
well.
| |Feedback Suppr. Failure Rate |
| l | Scen. 1 | Scen. 2 | Scen. 3 |
+------+---------+---------+---------+
| 0.10 | 0.273 | 0.893 | 0.822 |
| 0.25 | 0.355 | 0.885 | 0.624 |
| 0.50 | 0.364 | 0.787 | 0.385 |
| 0.75 | 0.334 | 0.679 | 0.318 |
| 1.00 | 0.298 | 0.621 | 0.279 |
| 1.25 | 0.289 | 0.637 | 0.267 |
| 1.50 | 0.274 | 0.595 | 0.249 |
| 1.75 | 0.274 | 0.580 | 0.235 |
| 2.00 | 0.258 | 0.577 | 0.233 |
| 3.00 | 0.282 | 0.577 | 0.236 |
| 4.00 | 0.282 | 0.577 | 0.236 |
Table 7: The ratio of feedback suppression failures.
Summarizing the feedback suppression results, it can be said that in
general the feedback suppression performance increases as l
increases. However, beyond a certain threshold, depending on
environment parameters such as propagation delays or session
bandwidth, the additional increase is not significant anymore. This
threshold is not uniform across all scenarios; a value of l=0.5 seems
to produce reasonable results with acceptable (though not optimal)
overhead.
6.2. Loss Report Delay
In this section, we show the results for the measured report delay
during the simulations of the three sample scenarios. This
measurement is a metric of the performance of the algorithms, because
the value of the feedback for the sender typically decreases with the
delay of its reception. The loss report delay is measured as the
time at the sender between sending a packet and receiving the first
corresponding loss report.
| | Mean Loss Report Delay |
| l | Scen. 1 | Scen. 2 | Scen. 3 |
+------+---------+---------+---------+
| 0.10 | 0.124 | 0.282 | 0.210 |
| 0.25 | 0.168 | 0.266 | 0.234 |