All notable changes to this project will be documented in this file.
- See release notes on Github
- Commit code used to create new VBN data release.
- Update VBN changelog..
- See release notes on Github
- Update testing
- Update readthedocs configuration
- Fix bug in trials change time loading.
- Python 3.10 and 3.11 support
- Support for VBO update release.
- Uspport for new VBO release
- Various bug fixes.
- See release notes on Github
- Support for updated vbn release containing probe, lfp and behavior only data.
- Updates to Ophys data in anticipation of a forthcoming updated data release
- Various notebook updates
- Python 3.9 support
- Various bug fixes and quality of life improvements.
- bugfix when accessing stimulus presentations table for vcn data
- updates vbn notebooks
- Add support for visual behavior neuropixels data
- Bug fix in ecephys
- Added support for VBN source density jobs.
- Bug fix for pg8000
- Add ability to extract running speed from mulit-stimulus experiments
- Compatible with pandas 1.4
- Fixes bug that caused file paths on windows machines to be incorrect in Visual behavior user-facing classes
- Updates to support MESO.2
- Loosens/updates required versions for several dependencies
- Updates in order to generate valid NWB files for Neuropixels Visual Coding data collected between 2019 and 2021
- Fixes bug that was preventing the BehaviorSession from properly instantiating passive sessions.
- Major internal refactor to BehaviorSession, BehaviorOphysExperiment classes. Implements DataObject pattern for fetching and serialization of data.
- Documentation changes ahead of SWDB 2021
- Bugfix to CloudCache; it is now possible for multiple users to share a cache.
- Reordered columns in Visual Behavior metadata tables to be more helpful
- fix to how from_lims API gets OPhys experiment metadata. Preserves relationship between OPhys experiments and failed containers
- minor fix to cloud cache
- Added ability to specify a static cache directory (use_static_cache=True) to instantiate VisualBehaviorOphysProjectCache.from_local_cache()
- Added 'experience_level', 'passive' and 'image_set' columns to ophys_experiments_table
- Added 'ophys_cells_table' metadata table to track the relationship between ophys_experiment_id and cell_specimen_id
- Bugfixes related to NWB creation for BehaviorSessions
- Fixed mkdir error for non-existing ecephys upload directory
- Refactored the schema for the ecephys copy utility to avoid raising an error when a previous output file already exists.
- python 3.8 compatibility
- CloudCache (the class supporting cloud-based data releases) is now smart enough to construct symlinks between files that are identical across dataset versions (rather than downloading duplicate copies of files).
- VisualBehavioOphysProjectCache supports user-controlled switching between dataset versions.
- Adds restriction to require hdmf version to be strictly less than 2.5.0 which accidentally introduced a major version breaking change
- This version marks the release of Visual Behavior Optical Physiology data! For more details please visit the: Visual Behavior - Optical Physiology Project Page
- update documentation to support visual behavior data release
- Fixes a bug with the dictionary returned by BehaviorSession get get_performance_metrics() method
- Adds docstrings to the BehaviorSession get_performance_metrics(), get_rolling_performance_df(), and get_reward_rate() methods
- changes name of BehaviorProjectCache to VisualBehaviorOphysProjectCache
- changes VisualBehaviorOphysProjectCache method get_session_table() to get_ophys_session_table()
- changes VisualBehaviorOphysProjectCache method get_experiment_table() to get_ophys_experiment_table()
- VisualBehaviorOphysProjectCache is enabled to instantiate from_s3_cache() and from_local_cache()
- Improvements to BehaviorProjectCache
- Adds project metadata writer
- Improvements to BehaviorProjectCache
- Updates to Session metadata; refactors implementation to use class rather than dict internally
- Fixes a bug that was preventing Allen Institute Windows users from accessing gratings images
- Created lookup table to get monitor_delay for cases where calculation from data fails
- If sync timestamp file has more timestamps than eye tracking moving has frame, trim excess timestamps (up to 15)
- Session API returns both warped and unwarped stimulus images, and both are written to NWB
- Refactored behavior and ophys session and data APIs to remove a circular inheritance issue
- Fixed segmentation mask and roi_mask misregistration in 'BehaviorOphysSession'
- Replaces BehaviorOphysSession.get_roi_masks() method with roi_masks property
- Fixes bug which prevented the SDK from loading stimuli dataframes for static gratings
- Return event detection data through session API
- Read/write event detection data from/to NWB
- Time stamps for events in trial_log are set to the exact sync timestamp of the corresponding frame.
- For behavior-only sessions, sync-like timestamp of the first frame is set to zero.
- stimulus_templates in Session API returns new object with key of image name
- Refactored BehaviorOphysSession to inherit methods and properties from BehaviorSession
- Fixed a test for checking that Behavior and BehaviorOphysSessions contain the same data regardless of which API (LIMS/JSON/NWB) is used. Also fixed resulting failure cases.
- Adds ability to write and read behavior only NWB files
- Adds eye tracking ellipse fits and metadata as new NWB data stream
- OPhys Behavior data retrieval methods no longer depend on ROIs being ordered identically in different files.
- Adds unfiltered running speed as new data stream
- run_demixing gracefully ignores any ROIs that are not in the input trace file
- update deprecated call to scipy.spatial.transform.Rotation.as_dcm() to .as_matrix()
- When running raster_plot on a spike_times dataframe, the spike times from each unit are plotted twice. (thank you @dgmurx)
- improvements and fixes to behavior ophys NWB files.
- improvements and fixes to BehaviorProjectCache tables including new column "donor_id"
- implemented a timeout to obtaining an ecephys session. (thank you @wesley-jones)
- big overhaul of how Behavior and BehaviorOphys classes are structured for the visual behavior project. See #1789
- (Internal) Fixed a bug in mesoscope processing where the ophys acquisition frames were being truncated prior to splitting, resulting in many fewer than expected acquisition frames.
- (Internal) Fixed a running_processing bug for behavior ophys experiments when the input data would have one more encoder entry than timestamp. The behavior of the code now matches what the warning says.
- (Internal) Fixed a write_nwb bug for behavior ophys experiments involving the BehaviorOphysJsonApi expecting a mesoscope-specific method.
- Adds load sync data for individual plane sets to relate accurate event timings to mesoscope data.
- Adds public API method to access the behavior_session_id from an instance of BehaviorOphysSession.
- Visual behavior running speed is now low-pass filtered at 10Hz. The raw running speed data is still available. The running speed is corrected for encoder threshold croissing artifacts.
- Support for stimulus gratings for visual behavior.
- Updates to some visual behavior pynwb implementations.
- Fixed an eye-tracking sync problem.
- AllenSDK HTTP engine streaming requests now include a progress bar.
- (Internal) Behavior Ophys Sessions no longer have a dependence on the
segmentation_mask_imagefile (provided by LIMS) when trying to write NWB files.
- (Internal)
response_timeof a trial in behavior-only or behavior + ophys sessions is now the first lick of the trial (for non-"aborted" trials). If no lick occurred or if the trial is "aborted",response_timeisNaN. - Resolve
ImportError: cannot import name 'MultiContainerInterface' from 'hdmf.container'errors by removing explicit version bounds on thehdmfpackage. - The optical physiology 2-photon trace demixer has been modified to be more memory friendly and should no longer result in out of memory errors when trying to demix very large movie stacks.
- (Internal) Docker image definitions have been updated so that internal continuous integration tests can work properly
- Behavior Ophys NWB File writing capability fixes for updated PyNWB and HDMF versions
- Added warning if using outdated Visual Coding Neuropixels NWB files
- Added documentation file for Visual Behavior terms in AllenSDK for quick lookup
- CCF locations for ecephys neuropixels electrodes have been added to their respective nwb electrodes tables
- Examples for accessing eye tracking ellipse fit and screen gaze location data have been added to ecephys example notebooks
- pynwb and hdmf version pinning has been relaxed
- The organization of data for ecephys neuropixels Neurodata Without Borders (NWB) files has been significantly changed to conform with NWB specifications and best practices
Important Note: Due to newer versions of pynwb/hdmf having issues reading previously released Visual Coding Neuropixels NWB files and due to the significant reorganization of their NWB file contents, this release contains breaking changes that necessitate a major version revision. NWB files released prior to 6/11/2020 are not guaranteed to work with the 2.0.0 version of AllenSDK. If you cannot or choose not to re-download the updated NWB files, you can continue using a prior version of AllenSDK (< 2.0.0) to access them. However, no further features or bugfixes for AllenSDK (< 2.0.0) are planned. Data released for other projects (Cell Types, Mouse Connectivity, etc.) are NOT affected and will NOT need to be re-downloaded
- The biophysical module can now run both current and legacy all-active models.
- Internal users can now access
date_of_acquisitionfor behavior-only Session data. - A pull request template was added to the repository.
- The CSV log was removed from
BehaviorProjectCache(internal users). - Duplicated demixer module was deprecated, and test coverage was added.
- Docker image for AllenSDK was updated.
- Internal LIMS data served to
BehaviorDataSessionclass now all use the same timestamp source
- Time sync tests were failing because of removed content. Tests were running only on nightly.
- Notebook tests failing on nightly because of down server, switched tests to production server.
- Internal users can now access
eye_trackingellipse fit data from behavior + ophys Session objects - A new mixin for managing processing parameters for Session objects
- Added support for additional sync file line labels
- Monitor delay calculation is updated to properly handle photodiode streams that end on a rising edge. We are no longer providing a default delay value in case of error.
- experiment_table from behavior project cache has NaNs in the 'imaging_depth' column for MultiScope experiments due to incorrect join in behavior_project_lims_api.py and 4 other places where ophys_sessions was incorrectly queried for imaging_depth_id
- get_keys method for sync datasets was returning the wrong line labels and creating incorrect key, value pairs for data loading from sync files
- tutorial for optotagging for ecephys notebook
- get_receptive_field() method in ecephys receptive field mapping
- remove redundant sham_change column in behavior sessions.trials table
- versions for NWB output for ecephys and ophys behavior.
- monitor delay is now calculated for BehaviorOphysLimsApi rather than defaulting to 0.0351
- Fixed a bug where auto-rewarded trials were not properly attributed in the rewards property of a visual behavior
- return None rather than raise exception if no container id was returned from lims id for given ophys id
- Project caches no longer accept arbitrary keywords
- matplotloib.pyplot.hist parameter normed no longer supported
- Users have an option to provide credentials for accessing the database either explicitly via public API or by setting up the environment variables
- Allow users to modify BehaviorDataSession and BehaviorOphysSession data
- morphology.apply_affine correctly rescales radii
- invalid extracellular electrophysiology spikes no longer show up as spikes at time -1
- (internal) When loading a behavior session, behavior and eye tracking video frame times are assessed from the correct lines
- Improved Neuropixels data download performance by enabling asynchronous transfers. Data downloads will now raise timeout errors when data cannot be retrieved in a reasonable timeframe.
- Updated AllenSDK readme and contributing documentation
- #1214 Fix hanging downloads for Neuropixels NWB files
- Support for Brain Observatory - Visual Coding Neuropixels data.
- #447 Implemented improved eye tracking module based on DeepLabCut
- Python 2.x is no longer supported.
- statsmodels is now pinned at 0.9.0
- https://github.com/AllenInstitute/AllenSDK/commit/f76678 Fix integer division bug in receptive field analysis
- #549 Transforms for tissuecyte registration accessible from allensdk
- #568 Release instructions for future maintainers.
- #582 Documentation builds from a yml config in python 3.7
- #523 pip no longer fails on new conda environment because of tables
- Added tools for working with itksnap label descriptions (#312)
- Update pytest version number (now requires pytest>=4.1.1)
- Update numpy version number (now requires numpy>=1.15.1)
- Removed deprecated functions from BrainObservatoryNwbDataSet
- Updated documentation
- #256 Examples python3 compatible
- #267 Fix get_cell_specimens filter argument
- #226 Fix pandas sorting behavior warning
- #207 Fix "two celltypes features"
- #273 Fix "make notebook"
- #275 Fix inconsistent numpy requirement
- #295 Fix BiophysicalApi.get_neuronal_models
- #309 Fix tests to use updated pytest
- #307 Fix openjpeg dependency in python3 in the CI build
- #311 Fix locally sparse noise test
- #330 Fix numpy warning message
- #330 Fix Pillow dependency install
- #405 Fix session analysis regression test (passing CI in python2 and python3)
- #420 Fix installation bug resulting from pytables 3.5
- https://github.com/AllenInstitute/AllenSDK/commit/1ef88046a58a36d870d3f1048a778806d1db2954 Fix scikit-image dependency install
- BrainObservatoryCache.get_ophys_experiment_events
- BrainObservatoryCache.get_ophys_experiments and get_experiment_containers accept reporter_lines argument
- BrainObservatoryCache.ge_ophys_experiment_analysis
- Cache subclasses can have their manifest locations set via environment variable
- bumped lower limit of pandas to 0.17, removed upper limit
- changed CellTypesCache cell data structure to be less nested, more flat. Requires update to manifest and re-download of cells.json.
- simplified MouseConnectivityCache data structure. Requires update to manifest and re-download of experiments.json.
- stopped using deprecated function in SimpleTree
- regression tests for Brain Observatory analysis
- Numerous Python 2/3 compatibility fixes
- Improved spike filtering in ephys.ephys_features
- CellTypesCache.get_cells has a new
speciesargument.
- MouseConnectivityCache downloads structure masks instead of computing them.
- ABS-64; Jupyterhub Dockerfiles
- PR #13; Manifest error check
ABS-132; changed to Allen Institute Software License ABS-129; StructureTree converts color hex triplet to 8-bit color triplet ABS-125; update expected data for unmocked tests PBS-358; update extract_trace module to deal with motion border case ABS-85; allow variable long square start time in ephys feature extractor
- BrainObservatoryNwbDataSet.get_demixed_traces
- BrainObservatoryNwbDataSet.get_pupil_location
- BrainObservatoryNwbDataSet.get_pupil_size
- BrainObservatoryApi.get_cell_specimen_id_mapping
- allensdk.brain_observatory.receptive_field_analysis
- allensdk.brain_observatory.demixer
- BrainObservatoryCache.get_ophys_experiments returns "acquisition_age_days" instead of "age_days". The new field describes the age of the animal on the day of experiment acquisition.
- BrainObservatoryCache.get_experiment_containers no longer returns "age_days".
- BrainObservatoryCache.get_ophys_experiments accepts a list of cell_specimen_ids for additional filtering
- json_utilities uses simplejson for better NaN handling
- issue #42: Caching behavior in MouseConnectivityCache for CCF volumes fixed
- GLIF examples, documentation fixed
- ReferenceSpace is a new class for relating structure graphs and annotation volumes.
- Standardized caching and paging decorators
- Ontology has been deprecated in favor of StructureTree.
- MouseConnectivityCache uses StructureTree internally (ontology-based methods are deprecated)
- CellTypesCache and MouseConnectivityCache use cacher decorator
- GlifApi has stateless methods now. Old methods are deprecated.
- Github issue #35 - MouseConnectivityCache's manifest now uses CCF-version- and resolution-specific file names for masks. The masks now live inside the CCF version directory. Users must download a new manifest.
- Github issues #23, #28 - added a new dependency "requests_toolbelt" and upgraded API database for more reliable large file downloads.
- Github issue #26 - better documentation for structure unionize records.
- Github issue #25 - documentation errors in brain observatory analysis.
- New CCF annotation volume with complete cortical areas and layers.
- Mouse Connectivity structure unionize records have been computed for new CCF. Previous records are available here: http://download.alleninstitute.org/informatics-archive/june-2016/mouse_projection/
- Github issue #27 - MouseConnectivityCache.get_structure_unionizes returns only requested structures, not all descendants. Added a separate argument for descendant inclusion.
- MouseConnectivityCache has a new constructor argument for specifying CCF version.
- Github issue #16 (jinja2 requirement)
- Github pull request #21 (spurious "i" typo) in r_neuropil.py
- neuropil subtraction algorithm (brain_observatory.r_neuropil) faster and more robust
- formatting changes for better PEP8 compliance
- preparation for Python 3 support
- updated Dockerfiles
- Github issue #17 (scipy requirement)
- Support for the Allen Brain Observatory data (BrainObservatoryCache and BrainObservatoryApi classes).
- Code for neurpil subtraction, dF/F estimation, and tuning analysis.
- New ephys feature extractor (ephys_features.py, ephys_extractor.py). The old one is still there (feature_extractor.py) but should be considered deprecated.
- CellTypesCache.get_cells has a new argument 'reporter_status', which accepts one or more ReporterStatus values.
- CellTypesApi.save_reconstruction_marker and CellTypesCache.get_reconstruction_marker download and open files containing positions that mark special points in reconstruction (truncation, early termination of reconstruction, etc).
- swc.read_marker_file for reading in marker files
- Morphology has new methods for manipulating the morphology tree structure
- allensdk.model.biophysical package supports active and passive models
- Morphology compartments are now Compartment objects that behave like dictionaries
- Compartment.children stores references to immediate descendant Compartments
- spike times in NWB files are stored in "analysis/spike_times" instead of "analysis/aibs_spike_times"
- NwbDataSet looks for spike times in both locations ("spike_times" first)
- glif_neuron_methods.py function names have been changed to be more standardized
- allensdk.model.biophysical_perisomatic package renamed to allensdk.model.biophysical
- NEURON dependency updated to 7.4 release 1370
- MouseConnectivityCache.get_structure_mask bug fixed (github issue 8)
- CellTypesApi.get_ephys_sweeps returns sweeps in sweep number order (github issue 11)
- NwbDataSet.set_spike_times added maxshape(None,) to create_dataset (github issue 7)
- MouseConnectivityCache.get_projection_matrix, method for building a signal matrix from injection structure to projection structure.
- CellTypesCache.get_morphology_features, method for retrieving morphology features for all cells
- CellTypesCache.get_all_features, method for retrieving both morphology and ephys features for all cells in a single table
- new RmaTemplate class enables construction of queries with jinja2 template library.
- Jupyter notebook examples added to documentation.
- Api.retrieve_parsed_json_over_http respects post parameter.
- improved installation of dependencies.
- Ontology.get_child_ids and Ontology.get_descendant_ids accept a list of ids instead of a variable length argument list.
- API Access/Data API Client documentation better reflects new 0.10.x allensdk.api.query modules.
- Cache.wrap method defaults to save_as_json=False.
- Cache.wrap method defaults to returning json rather than a pandas dataframe (new parameter return_dataframe=False).
- BiophysicalApi.cache_data throws an exception if no data is found for a neuronal model id.
- Replaced brainscales Dockerfile with neuralenseble Dockerfiles.
- MouseConnectivityCache.get_projection_matrix, method for building a signal matrix from injection structure to projection structure.
- CellTypesCache.get_morphology_features, method for retrieving morphology features for all cells
- CellTypesCache.get_all_features, method for retrieving both morphology and ephys features for all cells in a single table
- Ontology.get_child_ids and Ontology.get_descendant_ids accept a list of ids instead of a variable length argument list.
- Manifest and Cache classes for keeping track of files
- MouseConnectivityApi class for downloading data from the Mouse Connectivity Atlas
- MouseConnectivityCache fclass or keeping track of files
- CellTypesCache for keeping track of cell types files
- Ontology class for manipulating structures
- Rma class for formalizing API queries
- GridDataApi for downloading expression and projection grid volumes
- EphysFeatureExtractor module for computing ephys features used in Cell Types Database
- json_utilities has better numpy data type serialization support
- Documentation updated
- Installation/Makefile bug
- Everything: initial release!