Running an application with an MFS

Step 1: Building the FBMM enabled kernel

If you want to manually build and install the kernel yourself, follow these steps

1. Clone the FBMM kernel repo onto your target machine and install the required kernel build dependencies.
2. Build and install the kernel with the following config options below. The Makefiles for the MFS modules expect the compile output to be placed in <kernel dir>/kbuild/
   • CONFIG_TRANSPARENT_HUGEPAGE=y
   • CONFIG_PAGE_TABLE_ISOLATION=n
   • CONFIG_RETPOLINE=n
   • CONFIG_GDB_SCRIPTS=y
   • CONFIG_FRAME_POINTERS=y
   • CONFIG_IKHEADERS=y
   • CONFIG_SLAB_FREELIST_RANDOM=y
   • CONFIG_SHUFFLE_PAGE_ALLOCATOR=y
   • CONFIG_FS_DAX=y
   • CONFIG_DAX=y
   • CONFIG_BLK_DEV_RAM=y
   • CONFIG_FILE_BASED_MM=y
   • CONFIG_BLK_DEV_PMEM=y
   • CONFIG_ND_BLK=y
   • CONFIG_BTT=y
   • CONFIG_NVDIMM_PFN=y
   • CONFIG_NVDIMM_DAX=y
   • CONFIG_X86_PMEM_LEGACY=y
   • CONFIG_INIT_ON_ALLOC_DEFAULT_ON=n
3. Build the MFS kernel modules by going into their directory and running make. If you don't know how to build and install a kernel, follow these helpful instructions from Mark Mansi.

If you instead want to use a script to build the kernel, you can use the setup instructions from the USENIX ATC artifact README. This requires running some software on your workstation, but also goes through the effort of installing dependencies on the test machine.

Mounting an MFS

BasicMMFS

1. sudo insmod <kernel dir>/BasicMMFS/basicmmfs.ko
2. sudo mount -t BasicMMFS BasicMMFS -o numpages=<size> <mntdir>

Where <size> is the number of pages to reserve for BasicMMFS

BandwidthMMFS

1. sudo insmod <kernel dir>/BandwidthMMFS/bandwidth.ko
2. sudo mount -t BandwidthMMFS BandwidthMMFS <mntdir>
3. To set the interleave weight for a node: echo <weight> | sudo tee /sys/fs/bwmmfs*/node<nid>/weight

ContigMMFS

1. sudo insmod <kernel dir>/ContigMMFS/contigmmfs.ko
2. sudo mount -t ContigMMFS ContigMMFS <mntdir>

TieredMMFS

1. Follow the following instructions to reserve both local and remote memory using the memmap boot option.
2. sudo insmod <kernel dir>/TieredMMFS/tieredmmfs.ko
3. sudo mount -t TieredMMFS -o slowmem=/dev/pmem1 -o basepage=<use basepages> /dev/pmem0 <mntdir>

Where <use basepages> is true if the MFS should only allocate base pages, and false if the MFS should allocate 2MB pages.

Enabling FBMM

1. Make sure the MFS mount directory is accessable to user programs: sudo chown -R $USER <mntdir>
2. Enable FBMM: echo 1 | sudo tee /sys/kernel/mm/fbmm/state

Running an application with an MFS

A user sets the MFS to use for an application by writing the mount directory of the MFS to /proc/<pid>/fbmm_mnt_dir, where <pid> is the PID of the process. To have an application allocate its memory from an MFS on startup, use the FBMM wrapper program provided in this repo at ./bmks/fbmm_wrapper.c. Its usage is

./fbmm_wrapper <mntdir> <app> <args>