dp_auditorium

DP-Auditorium

This folder contains the DP-Auditorium library for auditing differential privacy guarantees as described in the introductory paper DP-Auditorium: a Large Scale Library for Auditing Differential Privacy.

The root directory is structured as follows. It contains four main folders and a runner module described below:

• Testers: Contains modules introducing PropertyTesters that check if there is evidence to reject the hypothesis that a privacy guarantee holds on a fixed pair of datasets.
• Generators: Contains modules introducing DatasetGenerators that output pairs of neighboring datasets, for example under the add/remove neighboring relation.
• Mechanisms: Contains examples of private and non-private mechanisms used to exemplify testers.
• privacy_test_runner.py: Module with runner that specializes to a tester and generator and instantiates the PropertyTester on several datasets' trials generated by the DatasetGenerator.
• Examples: Folder containing examples on how to combine the above tools.

Details on the signature of the above objects, can be found in the interfaces.py module. For further details please refer to the paper DP-Auditorium: a Large Scale Library for Auditing Differential Privacy.

Examples

An illustrative example is available in examples/run_mean_mechanism_example.py. This binary defines a Hockey-Stick divergence test as the property tester and a dataset generator that employs a random search (using Vizier) for suggesting datasets. Subsequently, the runner is instantiated with these two objects to conduct a test on a non-private mean mechanism.

There are two ways to run this, either via Bazel or after installing the library using pip install. Before install, please ensure that your machine has the sqlite3 library installed, e.g., sudo apt-get install libsqlite3-dev on Ubuntu machines.

Run with Bazelisk

For the first option, you need to have Bazelisk installed. Once that is done, run:

bazelisk build dp_auditorium:all
bazelisk run dp_auditorium/examples:run_mean_mechanism_example

Run via pip

For the second option, you will need the setuptools package installed. To ensure this, you may run pip install --upgrade setuptools. Then, to demonstrate our example, run:

python -m pip install .
python dp_auditorium/examples/run_mean_mechanism_example.py

Some MacOS users may run into issues linking Sqlite3 with Python during the installation of DP-Auditorium. In these cases, users may have to pass the relevant C++ options to pip directly. For example,

CFLAGS=-Wno-error=implicit-function-declaration pip install .