Is it possible to use multi controlnet through the API?

[monis24]

Now that the /controlnet routes are depreciated, can we still use multi-controlnet through the sdapi/txt2img routes with alwayson_scripts ?

[ljleb]

Yes. That is in fact the recommended way to call the web API now. You can still use the old /controlnet/*2img routes but it is not recommended. More details in the docs: https://github.com/Mikubill/sd-webui-controlnet/wiki/API#web-api

[Vespinian]

Could be case sensitive string checks spread across the code. The default ui unit at L157 uses "None" for the model and "none" for the module.

[spectralshift]

I tried and got the same non-expected result. Here's my checklist:

1. Updated Auto1111
2. Updated ControlNet extension
3. Shut down and restarted Auto1111, using only --api

I am using jupyter notebook for this, so I opened that up.

1. Tried running it with no module included in the payload (commented out "module")

Error running process: C:\stable-diffusion\automatic\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
Traceback (most recent call last):
  File "C:\stable-diffusion\automatic\stable-diffusion-webui\modules\scripts.py", line 409, in process
    script.process(p, *script_args)
  File "C:\stable-diffusion\automatic\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 720, in process
    preprocessor = self.preprocessor[unit.module]
KeyError: None

2. Tried running it with "None":

Loading preprocessor: None
Error running process: C:\stable-diffusion\automatic\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
Traceback (most recent call last):
  File "C:\stable-diffusion\automatic\stable-diffusion-webui\modules\scripts.py", line 409, in process
    script.process(p, *script_args)
  File "C:\stable-diffusion\automatic\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 720, in process
    preprocessor = self.preprocessor[unit.module]
KeyError: None

3. Tried running it with null, Null

Code won't run, null not being recognized in python.

4. Tried running it with "null", "Null"

Loading preprocessor: Null
Error running process: C:\stable-diffusion\automatic\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
Traceback (most recent call last):
  File "C:\stable-diffusion\automatic\stable-diffusion-webui\modules\scripts.py", line 409, in process
    script.process(p, *script_args)
  File "C:\stable-diffusion\automatic\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 720, in process
    preprocessor = self.preprocessor[unit.module]
KeyError: 'Null'

5. Running it with "none"

No issue, it runs with no errors. However, the output is not the output I would expect. My test was

a) I have an image (512x512), and I use it for all three preprocessors to generate a depth, a pose, and a canny image. I save those preprocessor raw images.

b) I run

• no module and pass in the custom depth/post/canny to ControlNet -> render
• with each module active, using the original image as the source, let the preprocessor produce images for ControlNet -> render

I would expect the results to match, and I do get the expected results if I drag the depth/pose/canny preprocessor images into the UI and run it with no module.

I haven't reported it because my code is cobbled together in an ugly mess and I assumed there was something I had messed up. After testing here, I'm a bit more confident something is wonky.

[ljleb]

In python the equivalent of json value null is None. It seems that omitting the key breaks by default. The code may need to change to take into consideration not specifying the module?

[ljleb]

In any case this needs to be reproduced and narrowed down in the code. I'm busy atm so I can't take care of it right now...

[spectralshift]

To mostly conclude this, I ran a series of tests.

1. No issue with outcomes. Just watch for only canny needing to be inverted.

2. The default value should change from "None" to "none", or something similar. Using "none" for the module/preprocessor works, leaving it blank/default value/"None" fails.

[redwolf3245]

Can someone help me with an example request using multiple controlnet settings? I'm confused about how to use it correctly now.