Installation fails due to port being removed from URL of private PyPi repository when generating lock file

[eflebus]

Checklist

• I added a descriptive title
• I searched open reports and couldn't find a duplicate

What happened?

I cannot install a Python package hosted on a private PyPi server listening on a generic port (e.g. 8080). This seems to happen due to a wrong URL in the lock file. Specifically, it fails to establish a connection with the server (port 80 is used instead of 8080).

Given the following environment.yml file:

channels:
  - conda-forge
pip-repositories:
  - http://127.0.0.1:8080/simple
dependencies:
  - python=3.11
  - pip:
    - mypkg==0.1.0

I run conda-lock lock -p linux-64 -f environment.yml to generate the lock file. However, the URL reported in conda-lock.yml lacks the specified port 8080.

...
- name: mypkg
  version: 0.1.0
  manager: pip
  platform: linux-64
  dependencies: {}
  url: http://127.0.0.1/packages/mypkg-0.1.0-py3-none-any.whl
  hash:
    sha256: 6f3840e23f0256ee8b8e1d01e63947e984ccb60c69cceb9da083f85f3686a34c
  category: main
  optional: false

If I run conda-lock install -n myenv conda-lock.yml, the installation fails and the following error messages are shown (I cut out a few lines of the stack trace).

...
ERROR:root:ERROR: Could not install packages due to an OSError: HTTPConnectionPool(host='127.0.0.1', port=80): Max retries exceeded with url: /packages/mypkg-0.1.0-py3-none-any.whl (Caused by NewConnectionError('<pip._vendor.urllib3.connection.HTTPConnection object at 0x727103f7d510>: Failed to establish a new connection: [Errno 111] Connection refused'))
ERROR:root:ERROR conda.cli.main_run:execute(49): `conda run pip install --no-deps -r /tmp/tmpk1ny9od7` failed. (See above for error)
INFO:root:Collecting mypkg@ http://127.0.0.1/packages/mypkg-0.1.0-py3-none-any.whl#sha256=6f3840e23f0256ee8b8e1d01e63947e984ccb60c69cceb9da083f85f3686a34c (from -r /tmp/tmpk1ny9od7 (line 1))
Traceback (most recent call last):
...
conda_lock._vendor.poetry.utils._compat.CalledProcessError: Command '['/home/ema/miniconda3/condabin/mamba', 'run', '--name', 'myenv', 'pip', 'install', '--no-deps', '-r', '/tmp/tmpk1ny9od7']' returned non-zero exit status 1.

After replacing 127.0.0.1 with 127.0.0.1:8080 in the lock file, the installation succeeds.

...
INFO:root:Collecting mypkg@ http://127.0.0.1:8080/packages/mypkg-0.1.0-py3-none-any.whl#sha256=6f3840e23f0256ee8b8e1d01e63947e984ccb60c69cceb9da083f85f3686a34c (from -r /tmp/tmpm5ednid5 (line 1))
INFO:root:  Downloading http://127.0.0.1:8080/packages/mypkg-0.1.0-py3-none-any.whl (1.2 kB)
INFO:root:Installing collected packages: mypkg
INFO:root:Successfully installed mypkg-0.1.0
...

I tried to track down the code responsible for removing the port from the original URL and I think I found it. As far as I can tell, it is line 428 inside function _get_url (module pypi_solver.py).

def _get_url(link: Link) -> str:
    parsed_url = urlsplit(link.url)
    link.url = link.url.replace(parsed_url.netloc, str(parsed_url.hostname))  # <--- line 428
    return link.url_without_fragment

I installed the dev environment as explained in the contributing guidelines and replaced that line as show below:

def _get_url(link: Link) -> str:
    parsed_url = urlsplit(link.url)
    link.url = link.url.replace(
        parsed_url.netloc,
        f"{parsed_url.hostname}:{parsed_url.port}" if parsed_url.port is not None else str(parsed_url.hostname)
    )
    return link.url_without_fragment

After this change, the URL in conda-lock.yml includes the port as expected.

I also run pytest tests/test_pip_repositories.py::test_it_uses_pip_repositories_with_env_var_substitution and the tests passed. I cannot tell if this proves that my change does not break anything else in the codebase since I couldn't run the entire suite.

I looked at the currently open issues and pull requests but I couldn't find anything similar. Hope this report is detailed and clear enough. Let me know if further information is needed.

Many thanks for this awesome tool!

Conda Info

active environment : pypi
    active env location : /home/ema/miniconda3/envs/pypi
            shell level : 2
       user config file : /home/ema/.condarc
 populated config files : /home/ema/.condarc
          conda version : 23.7.3
    conda-build version : not installed
         python version : 3.9.16.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.35=0
                          __linux=6.6.10=0
                          __unix=0=0
       base environment : /home/ema/miniconda3  (writable)
      conda av data dir : /home/ema/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://conda.anaconda.org/nodefaults/linux-64
                          https://conda.anaconda.org/nodefaults/noarch
          package cache : /home/ema/miniconda3/pkgs
                          /home/ema/.conda/pkgs
       envs directories : /home/ema/miniconda3/envs
                          /home/ema/.conda/envs
               platform : linux-64
             user-agent : conda/23.7.3 requests/2.31.0 CPython/3.9.16 Linux/6.6.10-76060610-generic pop/22.04 glibc/2.35
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

Conda Config

No response

Conda list

No response

Additional Context

Setup I used to reproduce the issue:

• Conda lock 2.5.7
• Python 3.11.9

# Create Python package
poetry new mypkg
cd mypkg
poetry build
cd ..
mkdir packages
cp mypkg/dist/mypkg-0.1.0* packages

# Run PyPi server locally using Docker
docker run -p 8080:8080 -v $(pwd)/packages:/data/packages pypiserver/pypiserver:latest run