Skip to content

Commit

Permalink
Link to arXiv version
Browse files Browse the repository at this point in the history
  • Loading branch information
tholop committed Jun 30, 2021
1 parent b1aa6ca commit 27d7532
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 5 deletions.
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@

# PrivateKube

PrivateKube is an extension to the popular Kubernetes datacenter orchestrator that adds privacy as a new type of resource to be managed alongside other traditional compute resources, such as CPU, GPU, and memory. A description of the project can be found on our [webpage](https://systems.cs.columbia.edu/PrivateKube/) and in our OSDI'21 paper, titled [Privacy Budget Scheduling](https://www.usenix.org/conference/osdi21/presentation/luo) (PDF locally available [here](https://columbia.github.io/PrivateKube/papers/osdi2021privatekube.pdf)).
PrivateKube is an extension to the popular Kubernetes datacenter orchestrator that adds privacy as a new type of resource to be managed alongside other traditional compute resources, such as CPU, GPU, and memory. A description of the project can be found on our [webpage](https://systems.cs.columbia.edu/PrivateKube/) and in our OSDI'21 paper, titled [Privacy Budget Scheduling](https://www.usenix.org/conference/osdi21/presentation/luo) (PDF locally available [here](https://columbia.github.io/PrivateKube/papers/osdi2021privatekube.pdf) and extended version available on [arXiv](https://arxiv.org/abs/2106.15335).


## Repo structure

This repository contains the artifact release for the OSDI paper:
- [system](system/): The PrivateKube system, which implements the privacy resource and a new scheduling algorithm for it, called *Dominant Privacy Fairness (DPF)*.
- [privatekube](privatekube/): A python client for interaction with the PrivateKube system and performing macrobenchmark evaluation.
- [privatekube](privatekube/): A Python client for interaction with the PrivateKube system and performing macrobenchmark evaluation.
- [simulator](simulator/): A simulator for microbenchmarking privacy scheduling algorithms in tightly controlled settings.
- [examples](examples/): Usage examples for various components, please refer its [README](./examples/README.md) for details.
- [evaluation](evaluation/): Scripts to reproduce the macrobenchmark and microbenchmark evaluation results from our paper.
Expand All @@ -27,7 +27,7 @@ This repository contains the artifact release for the OSDI paper:
- [1.4. Example usage in a DP ML pipeline](#14-example-usage-in-a-dp-ml-pipeline)
- [2. Getting started with the simulator](#2-getting-started-with-the-simulator)
- [2.1 Setup](#21-setup)
- [Setup python environment](#setup-python-environment)
- [Setup a Python environment](#setup-a-python-environment)
- [Installation from source](#installation-from-source)
- [2.2 Examples](#22-examples)
- [The minimal simulation example](#the-minimal-simulation-example)
Expand Down Expand Up @@ -63,6 +63,7 @@ mkdir ~/.kube
sudo chown -f -R $USER ~/.kube
```
(You can learn more about how to use Microk8s without sudo [here](https://github.com/ubuntu/microk8s/blob/feature/dev-docs/docs/access-without-sudo.md))

You can now start and stop your cluster with:
```bash
Expand Down Expand Up @@ -247,8 +248,8 @@ This simulator is used for prototyping and microbenchmark evaluation of privacy

### 2.1 Setup

#### Setup python environment
Install conda, create and activate an isolated python environment "ae".
#### Setup a Python environment
Install Conda, create and activate an isolated Python environment "ae".
```bash
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh
bash ~/miniconda.sh -b -p $HOME/miniconda
Expand Down
7 changes: 7 additions & 0 deletions evaluation/macrobenchmark/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,21 @@

# Macrobenchmark


## Requirements

The commands in this section have to be run from the `macrobenchmark` directory. You can jump there with:

```bash
cd evaluation/macrobenchmark
```

You should have the `privatekube` Python package installed, as described in the [main README](https://github.com/columbia/PrivateKube).

Please note that the steps below can take several days to run, depending on your hardware. If you want to examine the experimental data without running the preprocessing or the training yourself, you can download some artifacts from this [public bucket](https://storage.googleapis.com/privatekube-public-artifacts).

Training will be faster with a Nvidia GPU, but you can also use your CPU by specifying `--device=cpu` in the script arguments.

## Download and preprocess the dataset

To download a preprocessed and truncated (140Mb instead of 7Gb) version of the dataset, run the following:
Expand Down

0 comments on commit 27d7532

Please sign in to comment.