From 99ee699bfb93c530bffdfbdffbdcef3fade9c08d Mon Sep 17 00:00:00 2001 From: thesamovar Date: Sat, 22 Jun 2024 19:49:29 +0100 Subject: [PATCH] tweaks to paper and updated intro of science section --- paper/paper.bib | 14 ++++++++++++- paper/sections/basicmodel/basicmodel.md | 7 +++++++ paper/sections/delays/Delays.md | 7 +++++++ paper/sections/meta_science.md | 2 ++ .../new_inh_model/inhibition_model.md | 7 +++++++ paper/sections/science.md | 21 +++++++------------ 6 files changed, 44 insertions(+), 14 deletions(-) diff --git a/paper/paper.bib b/paper/paper.bib index da735c6..6449363 100644 --- a/paper/paper.bib +++ b/paper/paper.bib @@ -297,4 +297,16 @@ @article{Zenke2018 journal = {Neural Computation}, author = {Friedemann Zenke and Surya Ganguli}, year = {2018}, -} \ No newline at end of file +} + +@article{goodman_decoding_2013, + title = {Decoding neural responses to temporal cues for sound localization}, + volume = {2013}, + issn = {2050084X}, + doi = {10.7554/eLife.01312}, + abstract = {The activity of sensory neural populations carries information about the environment. This may be extracted from neural activity using different strategies. In the auditory brainstem, a recent theory proposes that sound location in the horizontal plane is decoded from the relative summed activity of two populations in each hemisphere, whereas earlier theories hypothesized that the location was decoded from the identity of the most active cells. We tested the performance of various decoders of neural responses in increasingly complex acoustical situations, including spectrum variations, noise, and sound diffraction. We demonstrate that there is insufficient information in the pooled activity of each hemisphere to estimate sound direction in a reliable way consistent with behavior, whereas robust estimates can be obtained from neural activity by taking into account the heterogeneous tuning of cells. These estimates can still be obtained when only contralateral neural responses are used, consistently with unilateral lesion studies. © Goodman et al.}, + number = {2}, + journal = {eLife}, + author = {Goodman, D.F.M. and Benichoux, V. and Brette, R.}, + year = {2013}, +} diff --git a/paper/sections/basicmodel/basicmodel.md b/paper/sections/basicmodel/basicmodel.md index 16ccd57..d790bca 100644 --- a/paper/sections/basicmodel/basicmodel.md +++ b/paper/sections/basicmodel/basicmodel.md @@ -1,6 +1,13 @@ (basic-model)= ## A minimal trainable model of IPD processing +```{list-table} +* - Section authors + - Dan Goodman +* - Notebooks + - [](../research/time-constant-solutions.ipynb) +``` + This section describes the initial model developed for the [Cosyne tutorial that served as the starting point for this project](https://neural-reckoning.github.io/cosyne-tutorial-2022/) {cite:p}`10.5281/zenodo.7044500`. It also describes some small variants of this basic model produced in the course of the project, which can be seen in the notebook [](../research/time-constant-solutions.ipynb). The aim of the model was to address a long standing question about how and why the brain localises sounds in the way it does, restricted specifically in this case to interaural phase differences (IPDs) cues used at low frequencies. A common strand in this research is to consider a population of binaurally responsive neurons, each with a different frequency and IPD tuning, summarised by their best frequency (BF) and best delay (BD), i.e. the frequency and delay/phase at which they give their strongest response. From this spatial-temporal *encoding* of the sound, a second network attempts to *decode* the sound location. In the place theory of {cite:t}`10.1037/h0061495`, the encoding is done by coincidence detection neurons arrayed so that each neuron receives a sound from the left and right ear with different conduction delays. Decoding proceeds as follows. When the conduction delays match the acoustic delays induced by the arrival time difference of the sound at the two ears, the neuron fires at a maximal rate because of the coincidence detection. Doubt was cast on this theory by {cite:t}`10.1038/86049`, who argued that it was not robust to neural noise, and proposed instead a "hemispheric code" that encodes the sound in the difference in the average firing rates of neurons whose best delay is positive versus negative. While this optimises robustness to neural noise, {cite:t}`10.7554/eLife.01312` showed that it was not efficient at integrating across frequencies, was biased in the presence of acoustic noise, and generalised poorly to sounds outside of the training set. diff --git a/paper/sections/delays/Delays.md b/paper/sections/delays/Delays.md index 681987f..0c35227 100644 --- a/paper/sections/delays/Delays.md +++ b/paper/sections/delays/Delays.md @@ -1,6 +1,13 @@ (delay-section)= ## Learning Delays +```{list-table} +* - Section authors + - Karim Habashy +* - Notebooks + - [](../research/Learning_delays.ipynb), [](../research/Learning_delays_major_edit2.ipynb), [](../research/Solving_problem_with_delay_learning.ipynb) +``` + This section introduces a simple method to solve the sound localization problem with only learnable delays. ### Introduction diff --git a/paper/sections/meta_science.md b/paper/sections/meta_science.md index 4b83e72..bea7746 100644 --- a/paper/sections/meta_science.md +++ b/paper/sections/meta_science.md @@ -10,6 +10,8 @@ We provided a [](../research/3-Starting-Notebook.ipynb) to give participants an Our starting notebook used a combination of NumPy [@harris2020array], Matplotlib [@Hunter2007], and PyTorch [@paszke_pytorch_2019]. The code for surrogate gradient descent was based on Friedemann Zenke's SPyTorch tutorial [@zenke_spytorch_2019;@Zenke2018]. +Note that we didn't require participants to use our starting notebook, and indeed in [](#inhib-model), De Santis and Antonietti implemented a very different model from scratch. + ### GitHub Like many open-source efforts, [our public GitHub repository](https://github.com/comob-project/snn-sound-localization) was the heart of our project. This provided us with three main benefits. First, it made joining the project as simple as cloning and committing to the repository. Second, it allowed us to collaborate asynchronously. That is, we could easily complete work in our own time, and then share it with the group. Third, it allowed us to track contributions to the project. Measured in this way, 26 individuals contributed to the project. However, interpreting this number is challenging, as these contributions vary significantly in size, and participants who worked in pairs or small groups, often contributed under a single username. We return to this point in the [](#discussion). diff --git a/paper/sections/new_inh_model/inhibition_model.md b/paper/sections/new_inh_model/inhibition_model.md index 0bf88a0..85ef229 100644 --- a/paper/sections/new_inh_model/inhibition_model.md +++ b/paper/sections/new_inh_model/inhibition_model.md @@ -1,6 +1,13 @@ (inhib-model)= ## Contralateral glycinergic inhibition as a key factor in creating ITD sensitivity +```{list-table} +* - Section authors + - Francesco De Santis, Alberto Antonietti +* - Notebooks + - [](../research/new_inh_model.ipynb) +``` + ### Highlights Here we use a more biologically inspired model to overcome some limitations that have been highlighted in the classic Jeffress model, whose existence, in mammals, is still debated. In fact, axonal delay lines have not been found in the mammalian MSO. We focused then our attention on the inhibitory inputs to the MSO, which were found both anatomically (there are inhibitory ipsilateral and contralateral pathways) and computationally (in our model) to have a central role for coding ITDs. Experiments with inhibition blocked (by a glycine antagonist: strychnine) shown indeed a loss of ITD-coding in the MSO. diff --git a/paper/sections/science.md b/paper/sections/science.md index bf7ddff..27494fd 100644 --- a/paper/sections/science.md +++ b/paper/sections/science.md @@ -1,17 +1,12 @@ -## Collective introduction +## Introduction -* (Based on new_inh_model.md) -* For most animals, sound localization is realised through two classes of acoustic signals: binaural and spectral cues. -* In humans, binaural cues are sufficient to discriminate sounds in the azimuth plane, while spectral cues help to discriminate vertical angles and resolve font-back ambiguities (ref). -* Focussing on binaural cues, animals exploit two differences in the signals arriving at each ear: their interaural time and level difference (ITD and ILD). -* Though the relative importance of these two cues for a given animal, depends on it's head size and range of audible frequencies (ref). -* As our audible range is centred on lower frequencies than other mammals (ref) and y (ref), it has long been assumed that we mainly use ITDs when discriminating between different azimuth angles, -* and, the predominant model of human sound localisation, known as the Jeffress model, relies solely on ITDs (ref). -* In this model, the acoustic signals arriving at each ear are converted into spike trains, and then passed along monoaural delay lines to a bank of binaural coincidence detectors, via excitatory synapses. -* As each detector receives each monoaural signal with some delay, each is sensitive to a particular range of ITDs. For example, a sound heard from the left, will proceed through the left delays ahead of the right, until they converge at a coincidence detector tuned to that temporal offset. -* Finally, the outputs of these coincidence detectors are summed over time and used to estimate the sounds location (Fig. x). -* However, while such delay lines were found in the avian Nucleus Laminaris (ref) equivalent structures are yet to be found in mammals, and experiments suggest a role for inhibition, as well as excitation in sound localisation (refs). -* With these caveats in mind, we set out to explore sound localisation through both simple, and more biologically plausible spiking neural network models. +We chose sound localisation using interaural time differences as the topic of the original [Cosyne tutorial that kicked off this project](https://neural-reckoning.github.io/cosyne-tutorial-2022/) {cite:p}`10.5281/zenodo.7044500`. The reasoning was that the tutorial was about spiking neural networks, and their unique distinguishing feature is the way that time is processed, something that is particularly important in the sound localisation circuit. + +We are able to localise sound by detecting location- or direction-specific cues in the signals that arrive at our ears. One of the most important source of cues (although not the only one) come from differences in the signal between two ears, including both timing and level differences (interaural timing difference or ITD, and interaural level difference or ILD). Humans are able to detect arrival time differences in some cases as small as 20 $\mu$s. + +The classic model of ITD sensitivity is the delay line model of {cite:t}`Jeffress1948` in which an array of binaural coincidence detector neurons receive inputs from the two ears with different delays. When the neural delays exactly match the acoustic delays induced by the sound location, the neuron is most active, and therefore the identity of the most active neuron can tell you the direction of the sound. This model was shown to be inefficient with respect to neural noise by {cite:t}`McAlpine2003`, who proposed an alternative model based on average firing rates of the two binaural hemispheres. This model was shown to be optimally robust to neural noise. {cite:t}`goodman_decoding_2013` showed that this model performed too poorly to account for behavioural data, especially in situations where sounds had complex and unknown spectral properties, or in the presence of background noise, and proposed an alternative based on a perceptron-like neural network that was both robust to neural noise and performed well across a range of conditions. + +The starting point of this project was to ask: what solutions would you find if you directly optimised a spiking neural network to localise sounds? How would those solutions depend on the available neural mechanisms and statistics of the sound? Could we understand the solutions found in a simple way? What properties would the solution have in terms of robustness to noise, generalisation, and so forth? Could the solutions found by optimisation throw light on features found in the auditory systems of different animals? ## A simple spiking neural network model * To explore how networks of spiking neurons can localise sound, we first considered a simple model, akin to the Jeffress model (ref).