Python requirements and versions matching

[jennykim1016]

Hello!

We are trying to run this script on Mac (both M1 / M2 / Apply Silicon -- newest). We have just realized that:

• Apple Silicon doesn't natively support Python 3.8
• Conda does not support Python 3.6 but still supports Python 3.8, but the required package versions cannot be met. For instance, conda install -c tensorflow-gpu=2.2.0 or keras=2.3.1. would not work (with conda-forge)[1]. With python 3.8, import statements such as import tensorflow.compat.v2 as tf in /opt/anaconda3/envs/py38/lib/python3.8/site-packages/keras/src/engine/functional.py would not work as it requires lower-version tensorflow and keras, which again Conda doesn't support now.

Given this, what would be the best way to run the SynthSeg script, locally on the Macbook? Is there a plan to support this script in later Python versions (3.10, 3.11) that would be even more compatible with the newer hardware structures in the future?

[1] Log for conda create -n synthseg_38 python=3.8 tensorflow-gpu=2.2.0 keras=2.3.1 nibabel matplotlib -c anaconda -c conda-forge

Channels:

• anaconda
• conda-forge
• defaults
  Platform: osx-arm64
  Collecting package metadata (repodata.json): done
  Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

• tensorflow-gpu=2.2.0*
• keras=2.3.1*

Current channels:

• https://conda.anaconda.org/anaconda
• https://conda.anaconda.org/conda-forge
• defaults

[hvgazula]

Hi, @jennykim1016 I noticed this issue while working on my own. While I haven't tested the full range of synthseg's functionality, my preliminary work suggests you should be able to use this code (barring a minor infraction here and there) with a newer version of TF/Keras and Python. The farthest I could go was TF=2.15 and Python=3.11 with generation (at least). I am currently looking into prediction and will look into training very soon. If my tests turn out well on these three aspects (generation, training, and prediction), it will suffice to say SynthSeg can run on newer versions seamlessly.

[phcerdan]

@hvgazula It would be awesome if you share your requirements_python311.txt. Good luck with training!

[hvgazula]

@hvgazula It would be awesome if you share your requirements_python311.txt. Good luck with training!

Here they are:
ss_reks_centos.txt
ss_reks_osx.txt

[BBillot]

Hi,

thanks all for the questions and answers!

I think I started looking into this a while ago and I was able to find a stable list of requirements for python 3.x (x>10), but I dropped it because I didn't like the new way of debugging keras models (it didn't allow anymore to access the values inside the call methods of custom keras layers).

I seem to remember that using this new list of requirements only needed minor changes in the code to make it run for training/testing. Unfortunately I don't think I saved this list of requirements... But using conda, it should be kind of straightforward to establish for any newer Python version.

Let me know if @hvgazula's list of requirements works, and then I can add it to the installation instructions :)

Thanks all!
Benjamin

[jennykim1016]

Thank you so much everyone for these helpful discussions! We have actually decided to move forward with Windows + appropriate versions for now, but if we do decide to test with a new Python version, I will make sure to test it and update the thread. Thanks!

[nimzodisaster]

Im testing this now. As an aside, given that synthseg and mri_easy_reg are both part of freesurfer, is it likely that if I get the gpu to work in synthseg, that I can get it to work on the easy-reg software (after telling its torch code to use the gpu device).?

[BBillot]

Well I'm sure there are some dependency differences between SynthSeg and EasyReg, so you might to use two different venv etc.