This software presents diverse utilities to develop the first post-processing layer using the neuromorphic auditory sensors (NAS) information. The NAS used implements a cascade filters architecture in FPGA, imitating the behavior of the basilar membrane and inner hair cells, working with the sound information decomposed into its frequency components as spike streams. The neuromorphic hardware interface Address-Event-Representation (AER) is used to propagate auditory information out of the NAS, emulating the auditory vestibular nerve. Using the packetized information (aedat files) generated with jAER software plus an AER to USB computer interface, NAVIS implements a set of graphs that allows to represent the auditory information as cochleograms, histograms, sonograms, etc. It can also split the auditory information into different sets depending on the activity level of the spike streams. The main contribution of this software tool is its capability to apply complex audio post-processing treatments and representations, which is a novelty for spike-based systems in the neuromorphic community. This software will help neuromorphic engineers to build sets for the training of spiking neural networks (SNN).
The following step-by-step guide will show you how to download, install and start using NAVIS. If you prefer a more user-friendly tutorial, you can watch the NAVIS' Getting Started video by clicking on the next image.
NAVIS requires Microsoft .NET Framework 4.5 or greater to be executed. The .NET Framework 4.5 and later versions are not supported on Windows XP, but on Windows Vista, Windows 7 and later versions of Windows.
In addition to Microsoft .NET Framework 4.5 or greater, the NAVIS' Visual Studio project requires Microsoft Visual Studio to compile the code (NAVIS was programmed using Visual Studio Community 2015).
To use NAVIS, first you need to download the latest release. This can be done by clicking on the "releases" button on the home page or just by cloning/downloading the repository.
Then, select the latest release (it has a green button next to it containing the text "Latest release") and click on the source code download link.
After the file has been downloaded, decompress it. NAVIS executable file (NAVIS.exe) is located at the "NAVIS_LatestBuild" folder. Place this folder wherever you prefer and double-click NAVIS.exe to run this tool. If you get an error when trying to execute NAVIS, please check that the folder from which it is being run contains each of the files that are inside the "NAVIS_LatestBuild" folder (no file has been deleted or moved to another folder) and that you have already installed Microsoft .NET Framework 4.5 or greater in your computer. If the problem persist, please send me an email explaining the situation.
Double-click on the NAVIS.exe file to run it. Three main sections can be seen when the main window is opened: the menu, the toolbar and the cochleogram section:
- The menu is located at the upper side of the window. It allows the user to access each of the NAVIS’s functionalities.
- The toolbar is located at the left side of the application. It has some quick buttons (shortcuts) for the most-common NAVIS’s utilities. All of these buttons are shown by default when using NAVIS for the first time. Although, the user can select which of them should appear or not, based on his /her interests. This can be done in the settings, which are deeply explained in the NAVIS wiki.
- The cochleogram section is the big empty space that can be seen after opening NAVIS. This will be used to display the cochleogram of the audio file (in .aedat format) after we load it.
The following picture shows the main window that you should be seeing after running the application.
The folder “AedatSampleFiles” contains several aedat files that can be used to test the application and its functionalities. These files have been captured using jAER, the USBAERmini2 platform and a NAS (with different configurations: mono, stereo, 32-channel and 64-channel).
In this step-by-step tutorial we will be using an aedat file that was captured using 64-channel binaural NAS: “en_un_lugar_de_la_mancha_aedat”, which contains a stream of events that correspond to a young woman reading the first sentence of the famous Spanish novel The Ingenious Gentleman Don Quixote of La Mancha: “En un lugar de La Mancha”.
Before loading this file, we need to configure NAVIS first. Click on File->Settings in the menu or on the button with the gear symbol in the toolbar. This will open the settings window, which is organized into different tabs. Each of the parameters that appear on every tab of the settings window is explained on the “Settings” page on the Wiki. For loading a file we only need to focus on the “Main” tab, specifically on three values: “Channels”, “Audio” and “Address Length”. The file that we will use in this tutorial was captured using a 64-channel binaural NAS and jAER; hence, we will select: “64” as “Channels”, “STEREO” as “Audio” and “32” as “Address Length” (jAER use 32 bits for storing the event’s address information). After this settings have been selected, click on the “Save as default settings” button.
After the settings have been modified and saved, we are now ready to load our first aedat file. Click on File->”Load Aedat” or on the button with the folder symbol in the left toolbar. Navigate to the folder where the file is and select it (this aedat can be found in the “Stereo->32-bit address length” folder from the AedatSampleFiles folder of the GitHub project). The loading process will now begin. Take into account that the file will take longer to load depending on its size and length. After the file is loaded correctly, its cochleogram will be shown in the cochleogram section that we mentioned before.
Now that the file is loaded, you can use any of the functionalities that NAVIS provides: sonogram, histogram, average activity, disparity between left and right cochleae, etc. This software also allows to split the original file into different single aedat files both manually and automatically based on the channel’s average activity.
In this tutorial we have introduced you on how to install, configure and start using NAVIS. We have shown how to load aedat files and the basics for further processing of the information using the tools that this software provides. A more detailed explanation of the main NAVIS’ functionalities (sonogram, histogram, average activity, disparity between cochleae, Manual Aedat Splitter and Automatic Aedat Splitter) and how to use them can be found in the Wiki of this project.
New functionalities or improvements to the existing project are welcome. To contribute to this project please follow these guidelines:
- Search previous suggestions before making a new one, as yours may be a duplicate.
- Fork the project.
- Create a branch.
- Commit your changes to improve the project.
- Push this branch to your GitHub project.
- Open a Pull Request.
- Discuss, and optionally continue committing.
- Wait untill the project owner merges or closes the Pull Request.
Thank you for contributing in the NAVIS project!
The authors of the NAVIS' original idea are: Juan P. Dominguez-Morales, Angel F. Jimenez-Fernandez, Manuel J. Dominguez-Morales and Gabriel Jimenez-Moreno.
The authors would like to thank and give credit to:
- Jiménez-Fernández, A., Cerezuela-Escudero, E., Miró-Amarante, L., Domínguez-Morales, M. J., de Asís Gómez-Rodríguez, F., Linares-Barranco, A., & Jiménez-Moreno, G. (2016). "A binaural neuromorphic auditory sensor for FPGA: a spike signal processing approach". IEEE Transactions on Neural Networks and Learning Systems.Year: 2016, Volume: PP, Issue: 99. Pages: 1 - 15, DOI: 10.1109/TNNLS.2016.2583223.
- Delbrück, T., “jAER Open Source Project,” 2007. [Online]. Available: http://sourceforge.net/p/jaer/wiki/Home/.
- R. Berner et al., “A 5 Meps $100 USB2.0 Address-Event Monitor-Sequencer Interface,” 2007 IEEE ISCAS, 2007.
- Alejandro Linares-Barranco, for his support and contribution.
This project is licensed under the GPL License - see the LICENSE.md file for details.
Copyright © 2015 Juan P. Dominguez-Morales
jpdominguez@atc.us.es
APA: Dominguez-Morales, J. P., Jimenez-Fernandez, A., Dominguez-Morales, M., & Jimenez-Moreno, G. (2017). NAVIS: Neuromorphic Auditory VISualizer Tool. Neurocomputing, 237, 418-422.
ISO 690: DOMINGUEZ-MORALES, Juan P., et al. NAVIS: Neuromorphic Auditory VISualizer Tool. Neurocomputing, 2017, vol. 237, p. 418-422.
MLA: Dominguez-Morales, Juan P., et al. "NAVIS: Neuromorphic Auditory VISualizer Tool." Neurocomputing 237 (2017): 418-422.
BibTeX: @article{dominguez2017navis, title={NAVIS: Neuromorphic Auditory VISualizer Tool}, author={Dominguez-Morales, Juan P and Jimenez-Fernandez, A and Dominguez-Morales, M and Jimenez-Moreno, G}, journal={Neurocomputing}, volume={237}, pages={418--422}, year={2017}, publisher={Elsevier} }