Merge pull request #46 from ver228/development

merge for 1.5.0

docs/EXPLANATION.md

@@ -31,7 +31,11 @@ The first step is to identify possible particles. We divide the image into regio
 
 ### TRAJ_JOIN
 
-The second step is to join the identified particles into trajectories. We link the particles' trajectories by using their nearest neighbor in consecutive frames. The nearest neighbor must be less than `max_allowed_dist` away and the fractional change in area must be less than `area_ratio_lim`. One particle can only be joined to a single particle in consecutive frames, no split or merged trajectories are allowed. If these conditions are not satisfied it means that there was a problem in the trajectory *e.g.* two worms collided and therefore in the next frame the closest object is twice the area, or a worm disapeared from the field of view. If there is any conflict, the trajectory will be broken and a new label will be assigned to any unassigned particle. 
+The second step is to join the identified particles into trajectories. The algorithm changes depending if the analysis is selected for a single-worm or for multi-worm. See the instructions of the widget [Set Parameters](HOWTO.md/#set-parameters) for more information.
+
+In the **single-worm**, the algorithm the particles are first filter by area. The filter threshold is calculated with the assumption that in most of the frames the worm is the largest object. Only one trajectory is linked using the closest neighbors in consecutive frames.
+
+In the **multi-worm** case, we link the particles' trajectories by using their nearest neighbor in consecutive frames. The nearest neighbor must be less than `max_allowed_dist` away and the fractional change in area must be less than `area_ratio_lim`. One particle can only be joined to a single particle in consecutive frames, no split or merged trajectories are allowed. If these conditions are not satisfied it means that there was a problem in the trajectory *e.g.* two worms collided and therefore in the next frame the closest object is twice the area, or a worm disapeared from the field of view. If there is any conflict, the trajectory will be broken and a new label will be assigned to any unassigned particle. 
 
 In a subsequent step, Tierpsy Tracker tries to join trajectories that have a small time gap between them *i.e.* the worm was lost for a few frames. Additionally we will remove any spurious trajectories shorter than `min_track_size` .
 

docs/HOWTO.md

@@ -58,7 +58,18 @@ Other important parameters to set are:
 
 * `Frame per Second` (fps) is the frame rate of your video. An important value since it is used to calculate several other parameters. If `Extract Timestamp` is set to `true`, the software will try to extract the frame rate from the video timestamp. However, keep in mind that it is not always possible to recover the correct timestamp, and therefore it is recommended that you provide the value here.
 * `Frames to Average` is used to calculate the background mask. This value can significantly speed up the compression step. However, it will not work if the particles are highly motile. Use the buttons `Play` and `Next Chunk` to see how a selected value affects the mask. Note that the average is used only for the background mask, the foreground regions are kept intact for each individual frame. 
-* `Microns per Pixel`. This value is only used to calculate the [skeleton features](EXPLANATION.md/#feat_create), but the results will be in pixels instead of micrometers if the conversion factor is not set.
+* `Microns per Pixel`. This value is only used to in the steps to calculate the final features](EXPLANATION.md). If this value is set to be less than zero the features results will be in pixels instead of micrometers.
+* `Analysis Type`. The selected analysis type will determine the series of [steps](EXPLANATION.md) executed by the program according to the table below:
+
+ | Extension | Description | 
+ ---------|-------------------------------------------------------
+ | BASE\* | No features, only steps up to the skeleton orientation | 
+ | TIERPSY\* | Add the steps for the [tierpsy features](EXPLANATION.md/#extract-features-tierpsy-features-route) calculation. | 
+ | OPENWORM\* | Add the steps for the [openworm features](EXPLANATION.md/#extract-features-openworm-route) calculation. |
+ | \*WT2 | Add the necessary steps to analyze videos recorded using the [WormTracker 2.0](https://www.mrc-lmb.cam.ac.uk/wormtracker/).| 
+ | \*SINGLE | Same steps as BASE but the trajectories will be joined with the assumption that there is only a single worm in the video.  | 
+ | \*AEX | Add the steps to filter worms and obtain the food contour using deep learning models. This models might only work from data from the [Behavioural Genomics Laboratory](https://lms.mrc.ac.uk/research-group/behavioural-genomics/). | 
+
 
 You can access further parameters by clicking `Edit More Parameters`. The explanation of each parameter can be found by using the [contextual help](https://en.wikipedia.org/wiki/Tooltip). It is not always trivial to effectively adjust these other parameters, but if you believe you need too, I recommend using a small video (~100 frames) for testing.
 

tierpsy/analysis/feat_tierpsy/get_tierpsy_features.py

@@ -75,15 +75,19 @@ def _display_progress(n):
 
                 args = []
                 for p in ('skeletons', 'widths', 'dorsal_contours', 'ventral_contours'):
-                    node = fid.get_node('/coordinates/' + p)
 
-                    dat = np.full((traj_size, *node.shape[1:]), np.nan)
-                    if skel_id_val.size > 0:
-                        if len(node.shape) == 3:
-                            dd = node[skel_id_val, :, :]
-                        else:
-                            dd = node[skel_id_val, :]
-                        dat[good_id] = dd
+                    node_str = '/coordinates/' + p
+                    if node_str in fid:
+                        node = fid.get_node(node_str)
+                        dat = np.full((traj_size, *node.shape[1:]), np.nan)
+                        if skel_id_val.size > 0:
+                            if len(node.shape) == 3:
+                                dd = node[skel_id_val, :, :]
+                            else:
+                                dd = node[skel_id_val, :]
+                            dat[good_id] = dd
+                    else:
+                        dat = None
 
                     args.append(dat)
 
@@ -115,7 +119,7 @@ def save_feats_stats(features_file, derivate_delta_time):
     with pd.HDFStore(features_file, 'r') as fid:
         fps = fid.get_storer('/trajectories_data').attrs['fps']
         timeseries_data = fid['/timeseries_data']
-        blob_features = fid['/blob_features']    
+        blob_features = fid['/blob_features'] if '/blob_features' in fid else None
 
 
     #Now I want to calculate the stats of the video

tierpsy/analysis/traj_join/joinBlobsTrajectories.py

@@ -152,6 +152,7 @@ def _validRowsByArea(plate_worms):
 
     # median location, it is likely the worm spend more time here since the
     # stage movements tries to get it in the centre of the frame
+    # TODO this assuption could case errors if the videos were not taken using WT2
     CMx_med = plate_worms_f['coord_x'].median()
     CMy_med = plate_worms_f['coord_y'].median()
     L_med = plate_worms_f['box_length'].median()

tierpsy/gui/TrackerViewerAux.py

@@ -28,7 +28,7 @@
     )
 
 #%%
-def _estimate_trajectories_data(ow_feat_file, timestamp, microns_per_pixel, stage_position_pix):
+def _estimate_trajectories_data(ow_feat_file, timestamp, microns_per_pixel, stage_position_pix = None):
     '''
     I want to estimate the trajectorires_data table from the features_timeseries in the old features
     so I can used them with the viewer.
@@ -64,7 +64,7 @@ def _estimate_trajectories_data(ow_feat_file, timestamp, microns_per_pixel, stag
 
             if stage_position_pix is not None:
                 #subtract stage motion if necessary
-                ss = stage_position_pix[new_data['frame_number']]
+                ss = stage_position_pix[new_data['frame_number'].astype(np.int)]
                 new_data['coord_x'] -= ss[:, 0]
                 new_data['coord_y'] -= ss[:, 1]
 
@@ -334,7 +334,7 @@ def _drawSkel(self, worm_qimg, skel_dat, skel_colors = GOOD_SKEL_COLOURS):
                     qPlg[tt].append(QPointF(*p))
 
 
-        if len(qPlg['skeleton']) == 0:
+        if not qPlg or len(qPlg['skeleton']) == 0:
             return
 
         pen = QPen()

tierpsy/version.py

@@ -1,5 +1,5 @@
 # # -*- coding: utf-8 -*-
-__version__ = '1.5.0-beta'
+__version__ = '1.5.0'
 
 try:
     import os
@@ -12,6 +12,9 @@
     pass
 
 '''
+1.5.0
+- Bug corrections.
+
 1.5.0-beta
 - Complete the integration with tierpsy features formalizing two different feature paths.
 - Reorganize the types of analysis, and deal with deprecated values.