Traffic Load generated by Highlight
Overview
This page explains and quantifies the small amount of management overhead that Highlight monitoring adds on a network.
Background
All management tools generate extra traffic in order to collect data from a network, but Highlight is designed to absolutely minimise this overhead through a number of optimisations:
- Not walking large portions of a device's MIB (SNMP Information set) just to discover it
- Only extracting the data we actually need from a device, with each query
- Grouping requests for multiple items into single requests (Chained OIDs)
- Grouping multiple requests into a single packet, where possible (Get Bulk operations)
- Compressing data as much as possible
For regular polling, Highlight collects data from each monitored device normally every three minutes. If you are using different polling rates, you'll need to increase or decrease these numbers in proportion.
Some data collection is done at longer intervals, particularly Application information using Cisco NBAR or Flow.
Link Health reports
A Link Health report covers a bearer, or individual virtual connections over it such as classes of service or VLANs. Each report creates one 400-byte packet every 3 minutes, or 20bits per second average, per link or class or VLAN.
Example:
Highlight traffic for a site with one bearer plus four classes of service
(1 + 4) X 20bps or 100bps, which would be 0.005% on a 2M circuit |
Highlight traffic for a site with two bearers, each with 6 VLANs
2 X (1 + 6) X 20bps or 280bps, which would be 0.013% on a 2M circuit |
Performance Tests
Results for performance tests are collected every three minutes. Each report creates one 200-byte packet every three minutes, or 10bits per second average, per test.
Example:
Highlight traffic for five performance tests running from one site
5 X 10bps or 50bps, which would be 0.0025% on a 2M circuit |
On top of this there will also be the traffic generated by the test itself (for example pings).
Application Visibility using NBAR
Application details using NBAR are collected from Cisco devices every 30 minutes, and each collection generates approximately 20 x 1000-byte packets.
Example:
Highlight traffic from a site for one NBAR report
A burst of 20kbps for a few seconds, every 30 minutes. |
Application Visibility using FLOW
Flow traffic gets from a device (router or switch, for example) into Highlight in two stages :
- From the device, to the Highlight collector: this is raw Flow data, and there is a lot of it.
- From the Highlight collector, uploaded to the Highlight platform. The collector aggregates and compresses Flow data, by a factor of typically 100x, so it's worth putting the collector(s) as close as possible to the devices generating Flow.
Raw Flow data will typically amount to an additional 2-4% load on a connection; this is a standard ‘Flow Export’ metric and would apply whether it’s Highlight or another tool collecting the Flow data.
Uploaded data is sent in a burst every 60 minutes rather than continuously and is similar in volume to NBAR data.
Example:
Highlight traffic from collector to platform for one Flow report
A burst of 20kbps for a few seconds, every 60 minutes. |
Application Visibility using AppVis™
AppVis uses the same transport mechanisms as Flow but forwards the application information from the Flow Collector every 30 minutes rather than 60 minutes.
Example:
Highlight traffic from collector to platform for one AppVis 30 minute summary
20KB sent as a burst, every 30 minutes. |
Load on Devices
How much load is put on a device being monitored by Highlight ?
Because of the low volume of SNMP traffic Highlight generates, the additional load on a monitored device is not measurable for normal polling for Link Health and performance tests. 'Normal' in this context would be perhaps up to a total of 20 tests. Even beyond this, the load on a small router might only be an additional 0.5% for the one or two seconds every three minutes that data is collected.
Application-level tests need more consideration. NBAR and Flow tests impose a non-trivial load on the router or switch being monitored. This additional load would be anything from 2% to 25%, depending on
- The volume of traffic flowing over the monitored interface(s)
- The size and available processing power of the device
As a general recommendation, monitor the CPU level on a device while enabling NBAR or Flow (also needed for AppVis) to see the effect in that particular case.
Other Considerations
The above numbers are additive, so consideration should be made for a site monitoring more than a single bearer.
Example:
A typical enterprise site with one bearer, four classes of service, two performance tests and one NBAR application view:
Link Health: (1 + 4) X 20bps = 100bps Performance: 2 x 10bps = 20bps Applications: burst of 20Kbps |
The total would be a background load of 120bps (0.006% on 2M line) and a few seconds at 20kbps every 30 minutes.
You can see that Highlight traffic load is very small. However, we have had instances where Service Providers monitor a large number of sites across a single gateway line. For example, monitoring 1000 sites down a single ADSL gateway management connection, in which case it’s good to do the above calculations and ensure the line can carry the traffic.
Highlight uses Spread Polling techniques so that large numbers of polls into a customer’s network are spread over a period of time rather than done all at once, which reduces burst loading on circuits carrying them.