Week 13: Queues and Stats

New for This Week

• Improve how we store NF stats in ZooKeeper
  • Fix memory leak
  • Smooth out spikes
• Implement skeleton distributed queue (D20).
• NFs can track their "headroom" and share it with the manager. This how many cores are used for this application less how many instances of this NF are running. This will allow only the original instance to start new NFs (we set the "headroom" in all children, where the current core is not the master, to 0). (D22).
• Researched how exactly to fork NFs. See discussion on #3. In summary:
  • We can use rte_eal_remote_launch to start a thread on another core (assuming the original NF was run with more than one core as an argument. This is how we can tell onvm the NF is "forkable").
  • nflib is not thread-safe at the moment. This will need a pretty large refactor.

Week 12: ZooKeeper Stats

New for This Week

• Store some basic NF stats in ZooKeeper
  • Per NF, store the RX ring utilization percentage
  • Use ZOO_SEQUENCE + ZOO_EPHEMERAL to create stat nodes for active NFs. In the manger, we simply need to track a mapping of the NF instance ID to the path returned back from ZooKeeper.
  • See D18 and D19
  • Some ZooKeeper cleanup code, see D17

Research

I did some reading into how to fork NFs. If we just call fork from inside an NF, we won't be able to runt the new copy on a different core (since only the master process can run rte_eal_remote_launch). Or, we could have the manager start the new NF: could we load the code into shared memory somehow so the manager can access a function pointer somehow, or could we store the command line arguments and have the manager do a fork/exec kind of thing.

Also, I'm thinking I'm going to have to make a distributed queue using ZooKeeper to track who should start these new NF copies. If an instance is running a service that "needs help", it should be able to atomically "claim" it and fork the existing copy somehow. This shouldn't be all that hard to write, the forking problem is the more difficult one.

Week 11: Distributed Routing Complete

New This Week

• Manager can route incoming packets from other instances to the proper NF. This involves encapsulating and decapsulating the onvm_pkt_meta structure to be sent on the wire as well as the packet data. We have to read/set the udata64 field in struct mbuf with the metadata. Diff.
• Submitted manager patch to allow displaying stats on a webpage instead of in the console. This is towards a distributed stats app, as well as improved manager logging to a file. Diff.

For Next Week

• Put stats in zookeeper
• Test out how dpdk environment works when forked

My Demo

• Basic distributed packet routing demo (speed tester -> basic monitor)

Week 10: Beginning Distributed Routing

New for this week:

• Basics of vxlan encapsulation/decapsulation
• onvm manager can encapsulate + route to a remote instance of a service when one is not found locally

Not working yet:

• Can't quite figure out how to get the manager on the decapsulating side to see incoming packets. Not sure where the issue is

Additional Commentary (needs nimbus VPN):

• Adding basic vxlan code
• Connecting vxlan code to onvm
• Add ethernet header support to packet helper