System and Network Monitoring and Management With SNMP
John Sellens
GNAC Canada
jsellens@gnac.com
USENIX LISA-NT 99
July, 1999
Introduction and Goals
- An introduction to practical SNMP use
- Provide a simple overview or survey of what is possible
- ``Industry acceptable practices''
- Some basic tool coverage
- Relatively independent of operating systems, etc.
What is SNMP?
- Simple Network Management Protocol
- Defined in Internet Standard 15, RFC 1157
- A method to query and control ``devices'' connected to a network
- Transaction based, uses UDP
ports 161 and 162
- Implemented by virtually every device on an IP network now
- Used for almost all network management and monitoring now
- Less used for computer system management and monitoring, but also
very useful in that area
SNMP Basics
- Two systems involved
- Agent - the device on the network
- Manager - sends queries to and receives responses from the agent
- Three goals
- Easy and small to implement
- Extensible to accomodate changing needs
- System and architecture independent
Protocol Basics
- There are five types of SNMP messages
- Get
- GetNext
- Set
- Response
- Trap e.g. coldStart, linkUp, linkDown, etc.
- Simple, short, easy to contruct, ``unreliable'' UDP datagrams
Some Simple Examples
% snmpget host1 pswd system.sysUpTime.0
system.sysUpTime.0 = Timeticks: (28946675) 3 days, 8:24:26
% snmpget localhost pswd system.sysContact.0
system.sysContact.0 = jsellens@uunet.ca
% snmpset localhost wrpswd \
system.sysContact.0 s jsellens@gnac.com
system.sysContact.0 = jsellens@gnac.com
% snmpget localhost pswd system.sysContact.0
system.sysContact.0 = jsellens@gnac.com
Some More Simple Examples
%
snmpget host1 pswd interfaces.ifTable.ifEntry.ifDescr.5 \
interfaces.ifTable.ifEntry.ifPhysAddress.5 \
interfaces.ifTable.ifEntry.ifOperStatus.5 \
interfaces.ifTable.ifEntry.ifSpeed.5 \
interfaces.ifTable.ifEntry.ifInOctets.5
interfaces.ifTable.ifEntry.ifDescr.5 = ed0
interfaces.ifTable.ifEntry.ifPhysAddress.5 = 0:60:97:93:4e:5b
interfaces.ifTable.ifEntry.ifOperStatus.5 = up(1)
interfaces.ifTable.ifEntry.ifSpeed.5 = Gauge: 10000000
interfaces.ifTable.ifEntry.ifInOctets.5 = 38675277
Even More Simple Examples
%
env PREFIX=.iso.org.dod.internet.private.enterprises.ucdavis \
snmpget localhost pswd disk.diskPath.3 \
disk.diskUsed.3 disk.diskAvail.3 disk.diskTotal.3 \
disk.diskPercent.3
enterprises.ucdavis.disk.diskPath.3 = /tmp
enterprises.ucdavis.disk.diskUsed.3 = 33
enterprises.ucdavis.disk.diskAvail.3 = 28863
enterprises.ucdavis.disk.diskTotal.3 = 31408
enterprises.ucdavis.disk.diskPercent.3 = 8
% snmpwalk localhost public
... lots of output ...
SNMP Variables, MIBs, and OIDs
- Every object is a variable, with a hierarchical name
e.g. iso.org.dod.internet.mgmt.mib-2.system.sysUpTime.0
- A ``Management Information Base'' is a
textual description of a set of SNMP variables and the possible values
- An ``Object Identifier'' is
the numeric equivalent
e.g. 1.3.6.1.2.1.1.3.0
- You can typically query by either name (full or partial) or OID
- MIB-II (RFC 1213) is expected
on every device
- The final number is (usually) an index into a table of values
e.g. the interface or disk volume tables
SNMP Security
- In SNMP v1, interactions are governed by shared secrets knows as
``community strings'' - usually a ``read'' and a ``write''
- This is not a ``high-security'' mechanism
- Everything is clear text, and it's all or nothing
- No challenge response, no one time passwords
- But this is a result of the typical intended use of the protocol
i.e., automated, not commands initiated by humans at a keyboard
- v2 and v3 improve things, as does the
``View-Based Access Control Model'' (RFC2275), but are (currently) less used
Device and OS Specific Information
- Most vendor SNMP agents also implement private enterprise MIBs
- Often have to resort to reading the raw MIBs to understand
- e.g. Cisco has an environmental MIB, NT has domain and DHCP MIBs,
NetApp has disk-related MIBs
- Different vendors are better or worse, some information is hard
to find and/or use
- But the vendor MIBs often have the most worthwhile variables
SNMP Applied: Monitoring
- Monitoring is for trends and exceptions
- Poll or Trap
- Collect history for analysis and execption identification
- Accounting or session data, traffic or activity levels
- Ripe for automation
- Have tools ready for investigation when needed
- Can do a lot with only the standard MIBs
Monitoring: How to Notice Problems?
- Catch traps
- as long as the agent will trap what you want
- and as long as you don't lose the trap packet
- Periodic poll
- put the last polled value in a file, complain if changed
- some variables are ``last change time'' which is convenient
for noticing transient changes
- Polling is a nice backup mechanism for traps
- Perl/VB/Tcl script or large commercial application
- Some sort of notification mechanism
Monitoring: How to Watch Trends?
- Some sort of historical database, and some sort of visualization
- MRTG (and its successor Cricket) is the way to go
- Very easy to track just about anything, and low overhead
- Favorite things to graph: memory, CPU, disk, network, uptime,
users, mail waiting to be read, queue lengths, ...
Monitoring: What?
- State changes e.g. network interface up/down
- Or last change time, which is easier to notice when flapping
- Threshold limits e.g. disk more than 99.99% full
- Queue sizes, traffic levels, users, web hits, ...
- Also watch for minimum thresholds
- e.g. less than 3 users in the daytime might be a problem
- Directional changes e.g. uptime should always be increasing
- Security related stuff - # users in user database, failed
authentications, etc.
SNMP Applied: Management
- Management is for manipulation and administration
- Remote configuration and manipulation
- Device configuration backups
- Kick unruly users off dialup ports
- More often triggered manually
- Some tasks are well automated e.g. config backups
Management: How?
- Depends largely on vendor-specific MIBs and implementation
- And often the vendor's applications or OpenView addons
- But you can accomplish a lot with a few simple SNMP gets and sets
- Usually best if wrapped by a script for ease of use
- Change management and version control are problematic
- Consider doing periodic config file backup and doing revision
control on that
Management: What?
- Really depends on the agent's capabilities and the device type
- Networking devices:
- Enable/disable/configure ports on routers or switches
- Some routing tweaks e.g. weights, gateways
- Computers:
- Typically fewer things that you can change on the OS
- The NT MIBs imply a reasonable level of remote control
- Like any remote manipulation, don't shoot yourself in the foot
- Prediction: this will get better and be more effective soon
Basic SNMP Tools - 1
- A basic tool set for interactive or script use
- I'm not counting commercial applications as ``basic'' tools
- Primarily UNIX in origin, but also on NT (mostly)
- Not surprisingly, these tools happen to be freely available
- Current OS releases typically include an agent, and perhaps some tools
Basic SNMP Tools - 2
- UCD / CMU SNMP
- Command line tools in C for manager and agent
- Implements standard MIB-II, Host Resources MIB,
plus the ``ucdavis'' MIB
- Extensible - can monitor anything
- libsnmp for writing applications
- UCD agent not (yet) available for Windows
- Better than ``snmputil'' from the NT reskit
Basic SNMP Tools - 3
- Scotty - Tcl Extensions
- The Scotty package contains the Tnm Tcl extension,
and the Tkined interactive network editor application
- Tnm provides a very good library of SNMP-related functions,
written in Tcl
- scotty is a Tcl shell that includes Tnm
- Very nice, well documented, worth learning Tcl for
- version 3 beta works under Windows
Basic SNMP Tools - 4
- Perl
- Simon Leinen's SNMP_Session.pm provides the basic SNMP functions
in a native Perl module
- Works, but is currently somewhat rudimentary
- No support for MIBs, so everything is specified as an OID
- Mike Mitchell's SNMP_util.pm
provides convenient covers for some of the SNMP_Session.pm
functions, and a labelling mechanism for SNMP variables
- Windows-specific tools - I'm not familiar enough to say
Basic SNMP Applications - 1
- There are many applications, commercial and non-commercial, that
use SNMP
- These are typically for network monitoring and management, though
they usually have some flexibility
- What follows is a quick overview of some ``well known''
applications
Basic SNMP Applications - 2
- MRTG - Multi Router Traffic Grapher
- An AMAZINGLY useful tool by Tobias Oetiker - you need MRTG (or
Cricket below)
- Generates graphs of almost anything over time, using a simple
config file, and makes them available through a web server
- Typically used to monitor traffic counts on routers and other
network devices, but can graph anything e.g., temperature, uptime,
messages in mail queue, disk usage, etc.
- Very widely used, lots of examples on the web site
- Works on Windows NT and UNIX
Basic SNMP Applications - 2
- MRTG Successors
- MRTG is not as fast as one would like it to be, and if you're
monitoring lots of things, you can generate a non-trivial load
- RRD Tool is Oetiker's next generation tool - Round Robin Database
tool
- major speed improvement over the MRTG engine
- Cricket by Jeff Allen
is a high performance replacement for MRTG that uses RRD Tool
as it's engine
- easy, tree-structured configuration, flexible
- very fast
- MRTG is more mature, but Cricket is getting there
Basic SNMP Applications - 2
- Other Freely Available SNMP Applications
- NOCOL/NetConsole (Network Operation Center On-Line) is a system
and network monitoring tool that uses SNMP for some of its polling
- Big Brother has similar goals but approaches them differently
- Some Commercial SNMP Applications
- Typically for mapping, monitoring and management, with lots of
graphics
- HP OpenView - very popular, with lots of plug-ins
- Cabletron Spectrum - an OpenView alternative
- Netcool from Micromuse
- Similar to nocol, but far more sophsticated, fewer graphics
Summary
- If you're not using SNMP yet, start now
- The state of the art is advancing
- More and more SNMP tools, more and more SNMP applications
- Better security and control models
- Modest collection of SNMP pointers at:
http://www.generalconcepts.com/resources/snmp/
- Questions?
John Sellens
1999-07-20