Merge branch 'development'

.travis.yml

@@ -1,26 +1,46 @@
-language: generic
+language: python
+python: 
+# We don't actually use the Travis Python, but this keeps it organized.
+- "3.5"
+- "3.6"
+
 notifications:
   email: false
 
 matrix:
     include:
-        - os: linux
-          sudo: required
-          dist: trusty
         - os: linux
           sudo: required
         - os: osx
-          osx_image: xcode7
 
 install:
-  - bash ./installation/installation_script.sh
-
+  # install anaconda
+  - sudo apt-get update
+  - wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh;
+  - bash miniconda.sh -b -p $HOME/miniconda
+  - export PATH="$HOME/miniconda/bin:$PATH"
+  - hash -r
+  - conda config --set always_yes yes --set changeps1 no
+  - conda update -q conda
+  # Useful for debugging any issues with conda
+  - conda info -a
+
+  # create testing enviroment
+  - conda config --add channels conda-forge 
+  - conda create -q -n test-environment python=$TRAVIS_PYTHON_VERSION 
+    numpy matplotlib pytables pandas gitpython pyqt=5 
+    h5py scipy scikit-learn scikit-image seaborn xlrd 
+    cython statsmodels keras opencv
+  - source activate test-environment
+  - pip install tensorflow 
+
+  # install tierpsy
+  - bash installation/installation_script.sh --setup_modules
+
 before_script:
-  #add conda path to the system. For some reason it does not work if it is done in the installation script.
-  - ls /home/travis/miniconda/bin
-  - export PATH=/home/travis/miniconda/bin:$PATH
+  #download examples
+  - bash installation/installation_script.sh --download_examples
 
 script:
   - which python3 
-  - python3 ./installation/installation_test.py #test installation libraries
-  - python3 ./Tests/run_tests.py #run tests
+  - python tests/run_tests.py 2 4 5

docs/EXPLANATION.md

@@ -35,15 +35,13 @@ The second step is to join the identified particles into trajectories. We link t
 
 In a subsequent step, Tierpsy Tracker tries to join trajectories that have a small time gap between them *i.e.* the worm was lost for a few frames. Additionally we will remove any spurious trajectories shorter than `min_track_size` .
 
-Additionally, if a valid [keras model](https://keras.io/) was trained. It is possible to use a neural network to filter worms from eggs and other spurious particles.
-
 Below there is an example of how the trajectories look after tracking.
 
 ![trajectories](https://cloud.githubusercontent.com/assets/8364368/26301795/25eb72ac-3eda-11e7-8a52-99dd6c49bc07.gif)
 
 ### SKE_INIT
 This is a refinement step to clean [/plate_worms](OUTPUTS.md/#plate_worms). For each trajectory we interpolate any time gap, calculate a fixed region of interest size, and smooth the threshold and centroid over time. The purpose of these modifications is to make the thresholding more robust and the data suitable for the [next step](#calculate-skeletons). The data is stored in the [trajectories_data](OUTPUTS.md/#trajectories_data) table. This is the main table since it contains the data of the trajectories that are used in the subsequent steps and displayed by the [viewer](HOWTO.md/#tierpsy-tracker-viewer).
-
+If a valid [keras model](https://keras.io/) was trained. It is possible to use a neural network to filter worms from eggs and other spurious particles.
 
 ### BLOB_FEATS
 

docs/HOWTO.md

@@ -1,8 +1,26 @@
-# Manual
+## Example Data
+
+Example files can be found [here](https://imperiallondon-my.sharepoint.com/:u:/g/personal/ajaver_ic_ac_uk/EazhY5HZ4zRIhrCYIm_p4VcBq50QYBPzeaBVF2DRgRe2IQ?e=fdrkj7). The zip file contains a multiworm video recorded using a high resolution fixed camera and a single worm video recorded using the [WT2.0](https://www.mrc-lmb.cam.ac.uk/wormtracker/).
+
+You can analyze the videos using the [Batch Processing Multiple Files](#batch-processing-multiple-files) App. The videos require different analysis parameters since they belong to different setups, therefore they cannot be processed together.
+
+For the multiworm video the `Parameters File` must be set to `MULTI_RIG.json` and the `File Pattern to Include` as `*.mov` as shown below:
+
+<img width="450" alt="screen shot 2018-04-25 at 09 04 05" src="https://user-images.githubusercontent.com/8364368/39233808-6344ddca-4869-11e8-9c34-d3db9102109f.png">
+
+For the multiworm video the `Parameters File` must be set to `WT2_clockwise.json` and the `File Pattern to Include` as `*.avi` as shown below:
+
+<img width="450" alt="screen shot 2018-04-25 at 09 13 19" src="https://user-images.githubusercontent.com/8364368/39233903-9a07ec62-4869-11e8-921e-27769f3dc87c.png">
+
+The processing times for in MacBook Pro (15-inch, 2017) were 04:31 minutes for the multiworm video and 11:43 minutes for the singleworm video.
+
+# Detailed Instructions
 
 ## Getting Started
 
-After running the installation script in OSX or Windows there should be a clickable executable named `TierpsyTracker` on the Desktop. If the executable is missing you can re-create it by running:
+If you are using any of the [binary executables](https://github.com/ver228/tierpsy-tracker/releases) you only have to double click the file and the program should start after a few seconds.
+
+If you [installed tierpsy tracker from the source](https://github.com/ver228/tierpsy-tracker/blob/master/docs/INSTALLATION.md) in OSX or Windows there should be a clickable executable named `TierpsyTracker` on the Desktop. If the executable is missing you can re-create it by running:
 
 ```bash
 #(OSX/Linux)

docs/INSTALLATION.md

@@ -1,31 +1,36 @@
 *The instructions below are to install Tierpsy Tracker from the source code. I would recommend to do this only if you are using Linux or want to run the development version, otherwise use the double-click executables available for Windows (7 or latest) and OSX (Yosemite or latest) in the [releases page](https://github.com/ver228/tierpsy-tracker/releases).*
 
 # System Requirements 
-- Freshly installed [miniconda] (https://conda.io/miniconda.html) or at least setup up a new enviroment.
+- Freshly installed [miniconda](https://conda.io/miniconda.html) or at least setup up a new enviroment.
 - Optional [ffmpeg](https://ffmpeg.org/download.html): ffprobe must be accessible from the command line to calculate the video timestamps.
 - [C compiler compatible with cython](http://cython.readthedocs.io/en/latest/src/quickstart/install.html). In Windows, you can use [Visual C++ 2015 Build Tools](http://landinghub.visualstudio.com/visual-cpp-build-tools). In OSX, if you install [homebrew](https://brew.sh/) it will setup the C compiler without the need to download XCode from the appstore. 
 - [Git](https://git-scm.com/). [Here](https://gist.github.com/derhuerst/1b15ff4652a867391f03) are some instructions to install it.
 
 # Installation
 
-1. Clone this repository either using the [Github Desktop](https://desktop.github.com/) or from the command line as:
-```bash
-git clone https://github.com/ver228/tierpsy-tracker
-```
-
-2. Install the conda dependencies from the conda-forge channel:
+1. Install the conda dependencies from the conda-forge channel:
+
 ```bash
 conda config --add channels conda-forge 
 
 conda install -y numpy matplotlib pytables pandas gitpython pyqt=5 \
-h5py scipy scikit-learn scikit-image seaborn xlrd cython statsmodels
+h5py scipy scikit-learn scikit-image seaborn xlrd cython statsmodels numba
+
+conda install -y -c conda-forge keras opencv tensorflow
+```
 
-conda install -y -c conda-forge keras opencv
-pip install tensorflow 
+2. Clone this repository either using the [Github Desktop](https://desktop.github.com/) or from the command line as:
+
+```bash
+git clone https://github.com/ver228/tierpsy-tracker
 ```
 
-3. Install the rest of the modules:
-On the tierpsy-tracker root folder (the folder with the cloned repository) type:
+
+3. Install the rest of the modules. Firts move to the tierpsy-tracker root folder (the folder with the cloned repository) as:
+```bash
+cd tierpsy-tracker
+```
+Then install the rest of the modules using the following command according to your operative system.
 ```bash
 bash installation/installation_script.sh #OSX or Linux
 
@@ -38,8 +43,8 @@ installation/installation_script.bat #Windows
 - You do not need to install manually the [Open Worm Analysis Toolbox](https://github.com/openworm/open-worm-analysis-toolbox). However if you do (and I do not recommend it), be aware that there is a bug with the pip installer: it is missing some dependencies and it will create a corrupt [.egg](https://stackoverflow.com/questions/2051192/what-is-a-python-egg) in your packages folder. Manually delete the .egg (use the error traceback to find the its location) and re-run `installation_script.sh`. The script will download the Open Worm Analysis Toolbox repository and install it using `python setup.py develop`. 
 
 
-# Test Examples
-On Mac OSX or Linux, some test examples can be downloaded by running 
+# Test Data
+On Mac OSX or Linux, some test data can be downloaded by running 
 
 ```bash
 installation/installation_script.sh --download_examples