[Feature]: Uploading ControlNet Images via API

[dyn-do]

Expected behavior

I am currently creating animations using the Web UI API in the following manner without animateDiff:
・Upload raw images of each frame's ControlNet, such as open_pose, segment, depth, etc., and generate images with img2img.
・Repeat the above step for the number of animation frames.

Is it possible to achieve this using the animateDiff API? Upon checking animatediff_ui.py, I only see options to upload data via video_source or video_path.

[continue-revolution]

Yes. You do not need to upload anything to AnimateDiff API. You can stick to your original workflow. You can also choose to upload a path to video via video_source or a path to dir with a series of frames via video_path - this will be broadcasted to all ControlNet units that you do not provide a path to frames via ControlNet API.

However, this feature is currently only available in txt2img. I’m planning to implement i2i batch in a week, via #224 , after finishing some personal real-life responsibilities.

[continue-revolution]

For txt2img, if you want to specify different control images for different ControlNet units, upload path to dir with a series of frames via Batch tab on ControlNet UI. I don’t know the corresponding API item in ControlNet.

[dyn-do]

Thank you for your response.
It seems that the API for the expansion of ControlNet doesn't support multiple frames, so it looks tough. 💦

[continue-revolution]

Where do you upload your frames via api?

[dyn-do]

Thank you for your response. I am posting images to the Web UI's img2img API with the ControlNet argument using the method below:
https://github.com/Mikubill/sd-webui-controlnet/wiki/API#integrating-sdapiv12img

[continue-revolution]

this is finally implemented by someone else and improved by me.