Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

run in ubuntu #10

Open
lcxsnow opened this issue May 1, 2023 · 2 comments
Open

run in ubuntu #10

lcxsnow opened this issue May 1, 2023 · 2 comments

Comments

@lcxsnow
Copy link

lcxsnow commented May 1, 2023

when I run the code in ubuntu, which has more than one gpu, it appeared the below error:

(pytorch) m11113013@htsc-Server3:~/ProjectCode/MasterProject2$ python -m torch.distributed.run --standalone --nproc_per_node 1 train.py --outdir ./training-runs/ --cfg lite --data /dataset/MSCOCO/mscoco.zip --img-resolution 64 --batch 8 --batch-gpu 2 --kimg 25000 --metrics fid50k_full
Error: --metrics can only contain the following values:
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 4694) of binary: /home/m11113013/miniconda3/envs/pytorch/bin/python
Traceback (most recent call last):
File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/run.py", line 765, in
main()
File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 345, in wrapper
return f(*args, **kwargs)
File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/run.py", line 761, in main
run(args)
File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/run.py", line 752, in run
elastic_launch(
File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

train.py FAILED

Failures:
<NO_OTHER_FAILURES>

Root Cause (first observed failure):
[0]:
time : 2023-04-30_22:44:16
host : htsc-Server3
rank : 0 (local_rank: 0)
exitcode : 1 (pid: 4694)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

When running on Window, it appeared the below errors:

(pytorch) C:\Users\snow\Desktop\python project\stylegan-t-main>python -m torch.distributed.run --standalone --nproc_per_node 1 train.py --outdir ./training-runs/ --cfg
lite --data ./data/coco_val256.zip --img-resolution 256 --batch 128 --batch-gpu 2 --kimg 25000 --metrics fid50k_full
NOTE: Redirects are currently not supported in Windows or MacOs.
Traceback (most recent call last):
File "train.py", line 25, in
from training import training_loop
File "C:\Users\snow\Desktop\python project\stylegan-t-main\training\training_loop.py", line 32, in
from metrics import metric_main
File "C:\Users\snow\Desktop\python project\stylegan-t-main\metrics\metric_main.py", line 20, in
from metrics import precision_recall
File "C:\Users\snow\Desktop\python project\stylegan-t-main\metrics\precision_recall.py", line 36, in
def compute_pr(opts, max_real: int, num_gen: int, nhood_size: int, row_batch_size: int, col_batch_size: int) -> tuple[float, float]:
TypeError: 'type' object is not subscriptable
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 8652) of binary: C:\Users\snow\miniconda3\envs\pytorch\python.exe
Traceback (most recent call last):
File "C:\Users\snow\miniconda3\envs\pytorch\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "C:\Users\snow\miniconda3\envs\pytorch\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\snow\miniconda3\envs\pytorch\lib\site-packages\torch\distributed\run.py", line 728, in
main()
File "C:\Users\snow\miniconda3\envs\pytorch\lib\site-packages\torch\distributed\elastic\multiprocessing\errors_init
.py", line 345, in wrapper
return f(*args, **kwargs)
File "C:\Users\snow\miniconda3\envs\pytorch\lib\site-packages\torch\distributed\run.py", line 724, in main
run(args)
File "C:\Users\snow\miniconda3\envs\pytorch\lib\site-packages\torch\distributed\run.py", line 718, in run
)(*cmd_args)
File "C:\Users\snow\miniconda3\envs\pytorch\lib\site-packages\torch\distributed\launcher\api.py", line 131, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "C:\Users\snow\miniconda3\envs\pytorch\lib\site-packages\torch\distributed\launcher\api.py", line 247, in launch_agent
failures=result.failures,
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

train.py FAILED

Failures:
<NO_OTHER_FAILURES>

Root Cause (first observed failure):
[0]:
time : 2023-05-01_13:33:36
host : DESKTOP-FIA0K2K
rank : 0 (local_rank: 0)
exitcode : 1 (pid: 8652)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

@snd-ml
Copy link

snd-ml commented May 15, 2023

I am also getting the same error. Did you get any reply yet?

@ADAMS12345678
Copy link

+1

I am also getting the same error. Did you get any reply yet?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants