Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Happy path installation #80

Open
maxmunzel opened this issue Oct 23, 2023 · 5 comments
Open

Happy path installation #80

maxmunzel opened this issue Oct 23, 2023 · 5 comments

Comments

@maxmunzel
Copy link
Contributor

I needed a week to get my fancy_gym install working. The solution that worked in the end for me was installing conda and using a fresh env with python=3.8.

This worked both on macOS 13 and Fedora 38.

All other attempts ended in obscure cmake errors in some sub-sub-sub-dependency like abseil-strings.

Talking with Bruce and Hongyi, it seems like conda is also the standard way to go
within the chair, so we should maybe add this to the readme?

One thing in particular is, that once you installed the master branch inside an env, you
can't install the legacy branch and vice versa, as the dependencies get messed up. This
tripped me over, hard.
@D-o-d-o-x
Copy link
Collaborator

Following the installation instructions should 'just work'. I tested this on multiple machines (two private machines, BwUni2 and HoreKa), but only on linux.
Only the optional dependency 'mujoco-legacy' is somewhat tricky as it requires mujoco to be installed natively beforehand, but that's not really something we can address on our end. (Also only required to access some not-yet-updated envs in gymnasium)

I can't easily test on MacOS. Did you also run into issue on Fedora? Can you provide me with replication instructions and the raised errors?

Yes, there are a lot of differences between the dependency-version for legacy and master. They should not be installed into the same venv. I can write a notice about this into the legacy README...

I never used conda and always just use vanilla Python (and the builtin virtual envs). We currently do not plan to provide conda packages. Installing via pip into a conda env should work and I can update the README accordingly in the future...

@maxmunzel
Copy link
Contributor Author

maxmunzel commented Oct 23, 2023 •
edited
Loading

EDIT: Trying it again, the .[metaworld] install also fails on a fresh env, let me come up with another example -.-

I tried various things (macOS python, fedora python, the python docker container, the conda docker container) with various problems. (The macOS/fedora + conda + fresh env combo seems reliable though.)

Here's an example what happens if you install "." first, then ".[metaworld]". For some reason, if you install in this order, it tries to build mujoco even though the 3.0.0 version has wheels. I understand this is mostly the sad state of python dependency management and not directly related to fancy_gym.

casparfriedrich@littlebird /t/fancy_gym (master)> conda create -n test python=3.8                                                                                                                                                                     (base)
casparfriedrich@littlebird /t/fancy_gym (master)> conda activate test                                                                                                                                                                                 (base)
casparfriedrich@littlebird /t/fancy_gym (master)> pip3 install .                                                                                                                                                                                      (test)


casparfriedrich@littlebird /t/fancy_gym (master)> pip3 install ".[metaworld]"                                                                                                                                                                         (test)
Processing /private/tmp/fancy_gym
  Preparing metadata (setup.py) ... done
Collecting metaworld@ git+https://github.com/Farama-Foundation/Metaworld.git@d155d0051630bb365ea6a824e02c66c068947439#egg=metaworld
  Cloning https://github.com/Farama-Foundation/Metaworld.git (to revision d155d0051630bb365ea6a824e02c66c068947439) to /private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-install-ejatjnie/metaworld_05dbd15a89f9466d8afb7e87aa6a6aa4
  Running command git clone --filter=blob:none --quiet https://github.com/Farama-Foundation/Metaworld.git /private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-install-ejatjnie/metaworld_05dbd15a89f9466d8afb7e87aa6a6aa4
  Running command git rev-parse -q --verify 'sha^d155d0051630bb365ea6a824e02c66c068947439'
  Running command git fetch -q https://github.com/Farama-Foundation/Metaworld.git d155d0051630bb365ea6a824e02c66c068947439
  Running command git checkout -q d155d0051630bb365ea6a824e02c66c068947439
  Resolved https://github.com/Farama-Foundation/Metaworld.git to commit d155d0051630bb365ea6a824e02c66c068947439
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: gymnasium>=0.26.0 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (0.29.1)
Requirement already satisfied: mp_pytorch<=0.1.3 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (0.1.3)
Requirement already satisfied: numpy>=1.21.0 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from gymnasium>=0.26.0) (1.24.4)
Requirement already satisfied: cloudpickle>=1.2.0 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from gymnasium>=0.26.0) (3.0.0)
Requirement already satisfied: typing-extensions>=4.3.0 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from gymnasium>=0.26.0) (4.8.0)
Requirement already satisfied: farama-notifications>=0.0.1 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from gymnasium>=0.26.0) (0.0.4)
Requirement already satisfied: importlib-metadata>=4.8.0 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from gymnasium>=0.26.0) (6.8.0)
Requirement already satisfied: matplotlib in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from mp_pytorch<=0.1.3) (3.7.3)
Requirement already satisfied: torch in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from mp_pytorch<=0.1.3) (2.1.0)
Requirement already satisfied: addict in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from mp_pytorch<=0.1.3) (2.4.0)
Collecting mujoco (from metaworld@ git+https://github.com/Farama-Foundation/Metaworld.git@d155d0051630bb365ea6a824e02c66c068947439#egg=metaworld)
  Downloading mujoco-3.0.0.tar.gz (655 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 655.9/655.9 kB 4.5 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting scipy>=1.4.1 (from metaworld@ git+https://github.com/Farama-Foundation/Metaworld.git@d155d0051630bb365ea6a824e02c66c068947439#egg=metaworld)
  Downloading scipy-1.10.1-cp38-cp38-macosx_10_9_x86_64.whl (35.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 35.0/35.0 MB 11.1 MB/s eta 0:00:00
Collecting imageio (from metaworld@ git+https://github.com/Farama-Foundation/Metaworld.git@d155d0051630bb365ea6a824e02c66c068947439#egg=metaworld)
  Using cached imageio-2.31.6-py3-none-any.whl.metadata (4.6 kB)
Requirement already satisfied: zipp>=0.5 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from importlib-metadata>=4.8.0->gymnasium>=0.26.0) (3.17.0)
Collecting pillow<10.1.0,>=8.3.2 (from imageio->metaworld@ git+https://github.com/Farama-Foundation/Metaworld.git@d155d0051630bb365ea6a824e02c66c068947439#egg=metaworld)
  Using cached Pillow-10.0.1-cp38-cp38-macosx_10_10_x86_64.whl.metadata (9.5 kB)
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from matplotlib->mp_pytorch<=0.1.3) (1.1.1)
Requirement already satisfied: cycler>=0.10 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from matplotlib->mp_pytorch<=0.1.3) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from matplotlib->mp_pytorch<=0.1.3) (4.43.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from matplotlib->mp_pytorch<=0.1.3) (1.4.5)
Requirement already satisfied: packaging>=20.0 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from matplotlib->mp_pytorch<=0.1.3) (23.2)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from matplotlib->mp_pytorch<=0.1.3) (3.1.1)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from matplotlib->mp_pytorch<=0.1.3) (2.8.2)
Requirement already satisfied: importlib-resources>=3.2.0 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from matplotlib->mp_pytorch<=0.1.3) (6.1.0)
Collecting absl-py (from mujoco->metaworld@ git+https://github.com/Farama-Foundation/Metaworld.git@d155d0051630bb365ea6a824e02c66c068947439#egg=metaworld)
  Using cached absl_py-2.0.0-py3-none-any.whl.metadata (2.3 kB)
Collecting etils[epath] (from mujoco->metaworld@ git+https://github.com/Farama-Foundation/Metaworld.git@d155d0051630bb365ea6a824e02c66c068947439#egg=metaworld)
  Downloading etils-1.3.0-py3-none-any.whl.metadata (5.5 kB)
Collecting glfw (from mujoco->metaworld@ git+https://github.com/Farama-Foundation/Metaworld.git@d155d0051630bb365ea6a824e02c66c068947439#egg=metaworld)
  Using cached glfw-2.6.2-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-macosx_10_6_intel.whl.metadata (5.4 kB)
Collecting pyopengl (from mujoco->metaworld@ git+https://github.com/Farama-Foundation/Metaworld.git@d155d0051630bb365ea6a824e02c66c068947439#egg=metaworld)
  Using cached PyOpenGL-3.1.7-py3-none-any.whl.metadata (3.2 kB)
Requirement already satisfied: filelock in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from torch->mp_pytorch<=0.1.3) (3.12.4)
Requirement already satisfied: sympy in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from torch->mp_pytorch<=0.1.3) (1.12)
Requirement already satisfied: networkx in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from torch->mp_pytorch<=0.1.3) (3.1)
Requirement already satisfied: jinja2 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from torch->mp_pytorch<=0.1.3) (3.1.2)
Requirement already satisfied: fsspec in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from torch->mp_pytorch<=0.1.3) (2023.10.0)
Requirement already satisfied: six>=1.5 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from python-dateutil>=2.7->matplotlib->mp_pytorch<=0.1.3) (1.16.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from jinja2->torch->mp_pytorch<=0.1.3) (2.1.3)
Requirement already satisfied: mpmath>=0.19 in /usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages (from sympy->torch->mp_pytorch<=0.1.3) (1.3.0)
Using cached imageio-2.31.6-py3-none-any.whl (313 kB)
Using cached Pillow-10.0.1-cp38-cp38-macosx_10_10_x86_64.whl (3.7 MB)
Using cached absl_py-2.0.0-py3-none-any.whl (130 kB)
Using cached glfw-2.6.2-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-macosx_10_6_intel.whl (96 kB)
Using cached PyOpenGL-3.1.7-py3-none-any.whl (2.4 MB)
Downloading etils-1.3.0-py3-none-any.whl (126 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.4/126.4 kB 6.8 MB/s eta 0:00:00
Building wheels for collected packages: fancy-gym, metaworld, mujoco
  Building wheel for fancy-gym (setup.py) ... done
  Created wheel for fancy-gym: filename=fancy_gym-1.0-py3-none-any.whl size=12138140 sha256=5d6b49225ae14db1dae3617e128cd985c423501fe900dc743e4db3b86578d183
  Stored in directory: /private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-ephem-wheel-cache-9xaoawhb/wheels/49/4e/02/74bf07aeaa68d492cddfbe4e7f1632a577e34322a4dd34a341
  Building wheel for metaworld (pyproject.toml) ... done
  Created wheel for metaworld: filename=metaworld-2.0.0-py3-none-any.whl size=84323613 sha256=023e4cffdd174b5c1be9c09a57aeab69fe6e38e405b47f525ade5e4ce36f2962
  Stored in directory: /Users/casparfriedrich/Library/Caches/pip/wheels/5e/6f/a9/09b585500a75b37ee7936df2681c9bec046616235ec6b36eef
  Building wheel for mujoco (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for mujoco (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [77 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-10.9-x86_64-cpython-38
      creating build/lib.macosx-10.9-x86_64-cpython-38/mujoco
      copying mujoco/viewer.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco
      copying mujoco/renderer.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco
      copying mujoco/renderer_test.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco
      copying mujoco/__init__.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco
      copying mujoco/msh2obj_test.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco
      copying mujoco/msh2obj.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco
      copying mujoco/rollout.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco
      copying mujoco/rollout_test.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco
      copying mujoco/bindings_test.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco
      copying mujoco/viewer_test.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco
      copying mujoco/render_test.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco
      copying mujoco/gl_context.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco
      creating build/lib.macosx-10.9-x86_64-cpython-38/mujoco/cgl
      copying mujoco/cgl/cgl.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco/cgl
      copying mujoco/cgl/__init__.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco/cgl
      creating build/lib.macosx-10.9-x86_64-cpython-38/mujoco/egl
      copying mujoco/egl/__init__.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco/egl
      copying mujoco/egl/egl_ext.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco/egl
      creating build/lib.macosx-10.9-x86_64-cpython-38/mujoco/osmesa
      copying mujoco/osmesa/__init__.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco/osmesa
      creating build/lib.macosx-10.9-x86_64-cpython-38/mujoco/glfw
      copying mujoco/glfw/__init__.py -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco/glfw
      creating build/lib.macosx-10.9-x86_64-cpython-38/mujoco/testdata
      copying mujoco/testdata/msh.xml -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco/testdata
      copying mujoco/testdata/abdomen_1_body.msh -> build/lib.macosx-10.9-x86_64-cpython-38/mujoco/testdata
      running build_ext
      Traceback (most recent call last):
        File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 434, in build_wheel
          return self._build_with_temp_dir(
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 419, in _build_with_temp_dir
          self.run_setup()
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 334, in <module>
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/normal/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 364, in run
          self.run_command("build")
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/private/var/folders/rp/_d4nsgyj2pb5bqg544p0p6hw0000gp/T/pip-build-env-k7esxbls/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 152, in run
        File "<string>", line 169, in _find_mujoco
      RuntimeError: MUJOCO_PLUGIN_PATH environment variable is not set
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for mujoco
Successfully built fancy-gym metaworld
Failed to build mujoco
ERROR: Could not build wheels for mujoco, which is required to install pyproject.toml-based projects

@maxmunzel
Copy link
Contributor Author

I'll try to replicate some of the failed installations attempts later, I threw aways the failed VMs after I had a running setup...

@D-o-d-o-x
Copy link
Collaborator

Ok. If you have a working setup, I would classify this as low-priority.
We will provide a PyPI package soon and installation will be as easy as pip install fancy_gym (No git clone necessary).

I will investigate this further, if these issues also arrise on this new installation method.

@maxmunzel
Copy link
Contributor Author

Sounds good 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@maxmunzel @D-o-d-o-x