Skip to content

Commit

Permalink
docs:update installation instructions (#113)
Browse files Browse the repository at this point in the history
* feat: refactored Readme

* feat: more readme improvements

* feat: updated all installation instructions

* docs: fixed path in run demo

* docs: fixed link

* docs: and or

* chore: responding to pr comments
  • Loading branch information
JuanBesa authored Feb 18, 2021
1 parent d3a70be commit 645e0b7
Show file tree
Hide file tree
Showing 5 changed files with 97 additions and 74 deletions.
72 changes: 24 additions & 48 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,69 +1,45 @@
# Welcome to the Fetch.ai Collective Learning

Colearn is a library that enables privacy-preserving decentralized machine learning tasks on the FET network.
Colearn is a library that enables privacy-preserving decentralized machine learning tasks on the [FET network](https://fetch.ai/technology/).

This blockchain-mediated collective learning system enables multiple stakeholders to build a shared
machine learning model without needing to rely on a central authority.
This library is currently in development.

The collective learning protocol allows learners to collaborate on training a model without requiring trust between the participants. Learners vote on updates to the model, and only updates which pass the quality threshold are accepted. This makes the system robust to attempts to interfere with the model by providing bad updates. For more details on the collective learning system see [here](https://fetchai.github.io/colearn/about/)

## Installation
1. Download the source code from github:
```bash
git clone https://github.com/fetchai/colearn.git && cd colearn
```
1. Create and launch a clean virtual environment with Python 3.7.
(This library has currently only been tested with Python 3.7).
```bash
pipenv --python 3.7 && pipenv shell
```
### Current Version

2. Install the package from source:
```bash
pip install -e .[all]
```
3. Run one of the examples:
```bash
examples/pytorch_mnist.py
```

For more instructions, see the documentation at [fetchai.github.io/colearn/](https://fetchai.github.io/colearn/)
We have released *v.0.2* of the Colearn Machine Learning Interface, the first version of an interface that will allow developers to prepare for future releases.
Together with the interface we provide a simple backend for local experiments. This is the first backend with upcoming blockchain ledger based backends to follow.
Future releases will use similar interfaces so that learners built with the current system will work on a different backend that integrates a distributed ledger and provides other improvements.
The current framework will then be used mainly for model development and debugging.
We invite all users to experiment with the framework, develop their own models, and provide feedback!

See the documentation at [fetchai.github.io/colearn/](https://fetchai.github.io/colearn/)

## Build image
## Installation

To build ML server image and push to google cloud use the following command:
```
cd docker
python3 ./build.py --publish --allow_dirty
# Check this worked correctly
docker images
```
To use the latest stable release we recommend installing the [package from PyPi](https://pypi.org/project/colearn/)

To install with support for Keras and Pytorch:
```bash
pip install colearn[all]
```
To install with just support for Keras or Pytorch:

## Documentation
To run the documentation, first install mkdocs and plugins:
```bash
pip install mkdocs==1.1.2 mkdocs-macros-plugin==0.5.0 \
mkdocs-macros-test==0.1.0 mkdocs-material==6.2.3 \
mkdocs-material-extensions==1.0.1 markdown-include==0.6.0
```
```bash
pip install colearn[keras]
pip install colearn[pytorch]
```

Then run:
```
mkdocs serve
```

For more instructions see the documentation at [fetchai.github.io/colearn/installation](https://fetchai.github.io/colearn/installation/)

After installation we recommend [running a demo](https://fetchai.github.io/colearn/demo/)
, or seeing [the examples](https://fetchai.github.io/colearn/examples/)

### Current Version

We have released *v.0.1* of the Colearn Machine Learning Interface, the first version of an interface that will allow developers to prepare for future releases.
Together with the interface we provide a simple backend for local experiments. This is the first backend with upcoming blockchain ledger based backends to follow.
Future releases will use similar interfaces so that learners built with the current system will work on a different backend that integrates a distributed ledger and provides other improvements.
The current framework will then be used mainly for model development and debugging.
We invite all users to experiment with the framework, develop their own models, and provide feedback!



## Quick Overview
The collective learning protocol allows learners to collaborate on training a model without requiring trust between the participants. Learners vote on updates to the model, and only updates which pass the quality threshold are accepted. This makes the system robust to attempts to interfere with the model by providing bad updates. For more details on the collective learning system see [here](docs/about.md)
6 changes: 3 additions & 3 deletions docs/demo.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
# How to run the demo

You can try collective learning for yourself using the simple demo in `./bin/run_demo.py`.
You can try collective learning for yourself using the simple demo in `examples/run_demo.py`.
This demo creates n learners for one of five learning tasks and co-ordinates the collective learning between them.

There are five potential datasets for the demo

* KERAS_MNIST is the Tensorflow implementation of standard handwritten digits recognition dataset
* KERAS_CIFAR10 is the Tensorflow implementation of standard image recognition dataset
* PYTORCH_XRAY is Pytorch implementation of a binary classification task that requires predicting pneumonia from images of chest X-rays.
The data need to be downloaded from [kaggle](https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia)
The data need to be downloaded from [Kaggle](https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia)
* PYTORCH_COVID_XRAY is Pytorch implementation of a 3 class classification task that requires predicting no finding, covid or pneumonia from images of chest X-rays.
This dataset is not currently publicly available.
* FRAUD The fraud dataset consists of information about credit card transactions, and the task is to predict whether
transactions are fraudulent or not.
The data need to be downloaded from [kaggle](https://www.kaggle.com/c/ieee-fraud-detection)
The data need to be downloaded from [Kaggle](https://www.kaggle.com/c/ieee-fraud-detection)

Use the -h flag to see the options:
```bash
Expand Down
9 changes: 9 additions & 0 deletions docs/dev_notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,13 @@ To have access to the google cloud storage you need to setup your google authent
have the $GOOGLE_APPLICATION_CREDENTIALS set up correctly.
For more details ask or see the contract-learn documentation

## Build image

To build ML server image and push to google cloud use the following command:
```
cd docker
python3 ./build.py --publish --allow_dirty
# Check this worked correctly
docker images
```

34 changes: 16 additions & 18 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,41 +20,39 @@ For more information on the Collective Learning Protocol see [here](about.md).

### Current Version

We have released *v.0.1* of the Colearn Machine Learning Interface, the first version of an interface that allows developers to define their own model architectures that can then be used in collective learning.
We have released *v.0.2* of the Colearn Machine Learning Interface, the first version of an interface that allows developers to define their own model architectures that can then be used in collective learning.
Together with the interface we provide a simple backend for local experiments. This is a prototype backend with upcoming blockchain ledger based backends to follow.
Future releases will use similar interfaces so that learners built with the current system will work on a different backend that integrates a distributed ledger and provides other improvements.
The current framework will then be used mainly for model development and debugging.
We invite all users to experiment with the framework, develop their own models, and provide feedback!

## Getting Started
1. Download the source code from github:

To use the latest stable release we recommend installing the [package from PyPi](https://pypi.org/project/colearn/)

To install with support for Keras and Pytorch:
```bash
git clone https://github.com/fetchai/colearn.git && cd colearn
pip install colearn[all]
```
1. Create and launch a clean virtual environment with Python 3.7.
(This library has currently only been tested with Python 3.7).
To install with just support for Keras or Pytorch:

```bash
pipenv --python 3.7 && pipenv shell
pip install colearn[keras]
pip install colearn[pytorch]
```

2. Install the package from source:
```bash
pip install -e .[all]
```
For more installation options see [Installation](./installation.md)
3. Run one of the examples:
For more installation options or get the latest (development) version see [Installation](./installation.md)

Then run one of the examples:

```bash
python examples/pytorch_mnist.py
```
For other examples see [Examples](./examples.md).

For other examples see the [Examples](./examples.md).

## Writing your own models
We encourage users to try out the system by writing their own models.
Models need to implement the collective learning interface, which provides functions for training and voting on updates.
More instructions can be found in the Getting Started section.

## Running the tests
Tests can be run with:
```
tox
```
50 changes: 45 additions & 5 deletions docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,25 +3,65 @@ The core package, `colearn`, contains only the [MachineLearningInterface](about.
implements the Collective Learning Protocol.
To install only the core package:
```
pip install .
pip install colearn
```

To make collective learning easier to use we have defined extra packages with helpers
for model development in Keras and Pytorch.

To install with Keras/Pytorch extras:
```
pip install .[keras]
pip install .[pytorch]
pip install colearn[keras]
pip install colearn[pytorch]
```

To install both the keras and pytorch extras use:
To install both the Keras and Pytorch extras use:
```
pip install .[all]
pip install colearn[all]
```

## Installing From Source

Alternatively, to install the latest code from the repo:

1. Download the source code from github:
```bash
git clone https://github.com/fetchai/colearn.git && cd colearn
```
1. Create and launch a clean virtual environment with Python 3.7.
(This library has currently only been tested with Python 3.7).
```bash
pipenv --python 3.7 && pipenv shell
```

2. Install the package from source:
```bash
pip install -e .[all]
```
3. Run one of the examples:
```bash
examples/pytorch_mnist.py
```

If you are developing the colearn library then install it in editable mode so that new
changes are effective immediately:
```
pip install -e .[all]
```
### Running the tests
Tests can be run with:
```
tox
```
## Documentation
To run the documentation, first install [mkdocs](https://www.mkdocs.org) and plugins:
```bash
pip install .[docs]
```

Then run:
```
mkdocs serve
```

0 comments on commit 645e0b7

Please sign in to comment.