Skip to content

Simulating Agents

Jump to bottom Edit New page
nick n edited this page May 28, 2021 · 17 revisions

1. Introduction

Sometimes we need to develop features without direct access to the relevant hardware. This page documents a method for simulating an SNMP agent from an alien device on your own computer.

2. Get snmp datadump

We need to have the snmp data from the alien device.

the snmpsim/snmprec version is preferred since it doesn’t need data mangling, it does require a few more packages to be installed & anonymizing data can be a challenge. the data quality however is a lot better.

2.1. ⚠️ privacy and sensitive data warning ⚠️

providing this data in a public fashion could be an issue for you. first of, the less data has been obfuscated the easier it is for us the help.

2.2. ‼️ we will not share your data in any way! ‼️

unless you allow us to do so explicitly.

2.3. options to get the data to the devs directly.

2.3.1. private git repository

github allows free private repos. you can upload your data there and give one of our devs access.

2.3.2. irc: #netdisco@libera

if you don’t want to create a private repo you can provide us with a download link on irc. most of us are on there using bouncers, so your link will not get lost. you can use the web client if you don’t want to install a full irc client.

2.3.3. email

as a last resort emailing the data to the devs is also an option.

3. snmp data dump options

the snmpsim/snmprec version is preferred.

3.1. snmpsim / snmprec version

the complete info is provided in step #4, so do read that. here is a short version to get you started.

install python prerequisites: (since step 4 already provides macos instructions i will use an opensuse example here)

install prerequisites on opensuse:

sudo zypper in python3-pip python3-virtualenv

install snmpsim:

mkdir python-env
cd python-env
virtualenv snmpsim
source snmpsim/bin/activate
pip install snmpsim

dump snmp data:

snmprec.py --agent-udpv4-endpoint={device} --start-object=1.0 --output-file={device}.snmprec --protocol-version=2c --community={community}

this should dump all snmp data that’s available on the device. keep in mind this might take a while.

3.2. snmpwalk version

The following command will deliver a file (probably a few megabytes in size):

snmpwalk -v 2c -c {community} -ObentU {device} .1 > snmpwalk.out

4. Install snmpsim

While you are waiting for the user to respond with their snmpwalk output, install the SNMP Simulator. This is a pure-Python, open-source agent simulator.

The easiest installation is to use pip to download from PyPI. On my MacOS system, I needed to install pip and virtualenv first:

sudo easy_install pip
sudo pip install virtualenv

Then create a virtualenv (which is Python’s equivalent of Perl’s local::lib) and install snmpsim into it:

mkdir python-env && cd python-env
virtualenv snmpsim
source snmpsim/bin/activate
pip install snmpsim

The most useful commands in this bundle are datafile.py which we will use to import the snmpwalk, and snmpsimd.py which runs the agent simulator.

Should you exit the shell and need to re-enter the virtualenv, simply run the source command as shown above.

5. Fix up the snmpwalk

Let’s assume your INBOX has now received a fresh snmpwalk.

For some reason, snmpwalk is a bit inconsistent in how it renders STRING values and also very long data values. We need all STRINGS to be on one line and enclosed in double-quotes.

The following command will stitch together multi-line STRINGs (run this first):

perl -0777 -pe 's/\n+(?!\.1\.)//g' {filename}

The following command will fix unquoted STRINGs (run this second):

perl -pe 's/= STRING: (?!")(.*)/= STRING: "\1"/g' {filename}

6. Import the snmpwalk to snmpsim

The snmpsim tool stores agent data in files with an .snmprec extension. The filename prefix (without extension) becomes the SNMP v2 community, so that you can serve multiple agents from one IP. Later I will show how to work around this for Netdisco, which needs one IP per device.

mkdir snmpsimdata && cd snmpsimdata
datafile.py --source-record-type=snmpwalk --input-file={/absolute/path/to/snmpwalk} > {devicename}.snmprec

Remember that {devicename} will become the SNMP v2 community for this agent. Also you must use an absolute path when specifying the input file to datafile.py.

7. Start the simulator

We’ll run snmpsimd.py listening on localhost on an unprivileged port. It will import all .snmprec files found in the snmpsimdata directory, as well as some sample .snmprec files it ships with. The first time snmpsimd.py starts, there is a pause while it generates some lookup tables.

snmpsimd.py --agent-udpv4-endpoint=127.0.0.1:1161 --v2c-arch --data-dir={/absolute/path/to/snmpsimdata}

You can test this with:

snmpstatus -v 2c -c {devicename} 127.0.0.1:1161

Isn’t that great? 😊

8. Supporting Netdisco

Most of the time one simulated agent is sufficient. Occasionally you may wish to simulate two or more devices. Netdisco needs them to be on different IPs.

We can simply add more --agent-udpv4-endpoint parameters to snmpsimd.py to listen on multiple IPs, and Netdisco will then treat them as different devices.

To add additional persistent loopback addresses on MacOS requires a startup script, as shown in this blog post.

Netdisco will also need telling to use the UDP port 1161 (or you can read how to start snmpsimd.py on port 161). This simple configuration will work:

# Netdisco backend deployment.yml
snmp_remoteport:
  1161: '127.0.0.0/8'

Also configure your device_auth setting to use each IP with a different community:

# Netdisco backend deployment.yml
device_auth:
  - tag: device1
    only: 127.0.0.1
    community: device1
  - tag: device2
    only: 127.0.0.2
    community: device2

9. Skipping problematic data

Sometimes Netdisco or net-snmp will choke on data coming from the simulator.

One example where I have seen this is on the ENTITY-MIB data from a Brocade router. As I didn’t need this for my development, I have two workaround options: either run only the Netdisco worker stage I require, or disable the problematic worker stage (entities, in this case).

Let’s look at each in turn:

9.1. Run a single worker stage

This will run only the Neighbor stage of discovery:

~/bin/netdisco-do discover::neighbors -d 172.0.0.2 -DI

You can see all the stages available by browsing the distribution files.

9.2. Disable a single worker stage

You can create a worker plugin to override one stage of a Netdisco action, and even target it at only a specific device. In this example I will override the discover::entities stage for device 127.0.0.2.

First enable local plugins in Netdisco, and enable your new plugin:

# Netdisco backend deployment.yml
site_local_files: true
extra_worker_plugins:
  - X::Discover::Entities

Then create the plugin directory (replacing NETDISCO_HOME with Netdisco’s home directory):

mkdir -p ${NETDISCO_HOME}/nd-site-local/lib/App/NetdiscoX/Worker/Plugin/Discover

And copy this content to Entities.pm in that directory:

package App::NetdiscoX::Worker::Plugin::Discover::Entities;

use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';

register_worker({
    phase => 'main', driver => 'snmp',
    priority => 110, only => ['127.0.0.2'],
  }, sub {
    my ($job, $workerconf) = @_;
    return Status->done(
      sprintf ' [%s] modules - skipped for this device', $job->device->ip);
});

true;

Now, when you run Netdisco discover, the entities stage will be skipped for device 127.0.0.2. Configuration in the plugin is used to target specific devices (using an access control list). See the backend worker documentation to learn more.

Add a custom footer
Add a custom sidebar
Clone this wiki locally