variable time step yields different results in different platforms

[anilbey]

Hi,

Quoting @alexsavulescu:

It is known that variable time step can yield different results (i.e. see comment #1328 (comment)).

The figure below shows the differences in the variable time step in Ubuntu and Centos platforms.

Is there a plan to make variable time step yield consistent results across different platforms?

[nrnhines]

The plan is to upgrade to the latest sundials distribution. It is presently unknown whether that will be a complete solution. I do know that the use of the "pow" function can cause round off error differences between windows and linux. What are the ordinate and abscissa of the above plot?

[anilbey]

Thanks for the reply @nrnhines. I look forward to the sundials update.

The ordinate is the voltage in mV. The abscissa is the time trace of the simulation. It is in fact a zoomed-in version of this plot.

[pramodk]

The figure below shows the differences in the variable time step in Ubuntu and Centos platforms.

@anilbey : just as an extra information, could you tell us how you have installed neuron here? e.g. same version via official wheels on the both platforms?

[anilbey]

Yes correct, I installed via pip install.

[pramodk]

Ok. And what versions of Ubuntu and CentOS?

Not sure if we would be able to track down everything (or it's worth doing it), but if there is a small repducer you can share here then we can take a look when have some additional time.

[anilbey]

Yes there's a reproducer, I had put it on the original ticket (NRN-126) in our internal system before creating this. Let me try write the instructions here without depending on our system.

[anilbey]

Instructions on how to reproduce it on Github's CI (docker).

This is the repository: https://github.com/BlueBrain/SSCxEModelExamples

A tag, namely "neuron-os-debug" is created on Github to mark the state of the repository.

https://github.com/BlueBrain/SSCxEModelExamples/releases/tag/neuron-os-debug

If you just run the CI code (either the gitlab-ci.yaml instructions or the .github/workflows/python-app.yml instructions) on an Ubuntu system, you will see that it fails because the result was created on Centos. You might as well trigger the CI system on github that runs on ubuntu-latest. When you run the code on a Centos system however, you will notice there is no difference in the results and the tests pass.

A github ci result (this may become unavailable in time, if that's the case re-run the ci from the tag or ask me to do it): https://github.com/BlueBrain/SSCxEModelExamples/actions/runs/3272203221/jobs/5382928268
It says

ValueError: operands could not be broadcast together with shapes (5062,2) (4611,2)

[arnaudon]

Hello, I think I have the same issue with some tests. On bb5 and my MacBook I have the same numerical values, but on github CI (with ubuntu) I have different ones.