feat(backend): add stablediffusion-ggml #4289
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| func (sd *SDGGML) GenerateImage(opts *pb.GenerateImageRequest) error { | ||
| t := C.CString(opts.PositivePrompt) | ||
| defer C.free(unsafe.Pointer(t)) |
Check warning
Code scanning / gosec
Use of unsafe calls should be audited Warning
| defer C.free(unsafe.Pointer(t)) | ||
|
|
||
| dst := C.CString(opts.Dst) | ||
| defer C.free(unsafe.Pointer(dst)) |
Check warning
Code scanning / gosec
Use of unsafe calls should be audited Warning
✅ Deploy Preview for localai ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
mudler
force-pushed
the
feat/stablediffusion-ggml
branch
from
597a649 to
b5bcda2
Compare
|
Testing with: name: sd-ggml
backend: stablediffusion-ggml
parameters:
model: huggingface://city96/FLUX.1-dev-gguf/flux1-dev-Q2_K.ggufWIP as doesn't work still |
|
Testing it with: name: sd-ggml
backend: stablediffusion-ggml
parameters:
model: huggingface://city96/FLUX.1-dev-gguf/flux1-dev-Q2_K.gguf
options:
- "diffusion_model"
- "clip_l_path:clip_l.safetensors"
- "t5xxl_path:t5xxl_fp16.safetensors"
- "vae_path:ae.safetensors"
- "sampler:euler"
download_files:
- filename: ae.safetensors
sha256: afc8e28272cd15db3919bacdb6918ce9c1ed22e96cb12c4d5ed0fba823529e38
uri: https://huggingface.co/ChuckMcSneed/FLUX.1-dev/resolve/main/ae.safetensors
- filename: clip_l.safetensors
sha256: 660c6f5b1abae9dc498ac2d21e1347d2abdb0cf6c0c0c8576cd796491d9a6cdd
uri: https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/clip_l.safetensors
- filename: t5xxl_fp16.safetensors
sha256: 6e480b09fae049a72d2a8c5fbccb8d3e92febeb233bbe9dfe7256958a9167635
uri: https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/t5xxl_fp16.safetensors |
| size := C.size_t(unsafe.Sizeof((*C.char)(nil))) | ||
| length := C.size_t(len(opts.Options)) | ||
| options = (**C.char)(C.malloc(length * size)) | ||
| view := (*[1 << 30]*C.char)(unsafe.Pointer(options))[0:len(opts.Options):len(opts.Options)] |
Check warning
Code scanning / gosec
Use of unsafe calls should be audited Warning
| defer C.free(unsafe.Pointer(dst)) | ||
|
|
||
| negative := C.CString(opts.NegativePrompt) | ||
| defer C.free(unsafe.Pointer(negative)) |
Check warning
Code scanning / gosec
Use of unsafe calls should be audited Warning
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
mudler
force-pushed
the
feat/stablediffusion-ggml
branch
from
765da54 to
606f215
Compare
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
|
finally working now: name: sd-ggml
backend: stablediffusion-ggml
parameters:
model: huggingface://city96/FLUX.1-dev-gguf/flux1-dev-Q2_K.gguf
options:
- "diffusion_model"
- "clip_l_path:clip_l.safetensors"
- "t5xxl_path:t5xxl_fp16.safetensors"
- "vae_path:ae.safetensors"
- "sampler:euler"
step: 25
download_files:
- filename: ae.safetensors
sha256: afc8e28272cd15db3919bacdb6918ce9c1ed22e96cb12c4d5ed0fba823529e38
uri: https://huggingface.co/ChuckMcSneed/FLUX.1-dev/resolve/main/ae.safetensors
- filename: clip_l.safetensors
sha256: 660c6f5b1abae9dc498ac2d21e1347d2abdb0cf6c0c0c8576cd796491d9a6cdd
uri: https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/clip_l.safetensors
- filename: t5xxl_fp16.safetensors
sha256: 6e480b09fae049a72d2a8c5fbccb8d3e92febeb233bbe9dfe7256958a9167635
uri: https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/t5xxl_fp16.safetensorscurl http://localhost:8080/v1/images/generations -H "Content-Type: application/json" -d '{
"prompt": "A cute baby sea otter", "model": "sd-ggml",
"size": "256x256"
}' |
| sd.threads = int(opts.Threads) | ||
|
|
||
| modelFile := C.CString(opts.ModelFile) | ||
| defer C.free(unsafe.Pointer(modelFile)) |
Check warning
Code scanning / gosec
Use of unsafe calls should be audited Warning
leejet/stable-diffusion.cpp#366 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR is a first attempt into adding a very straightforward backend for stablediffusion.cpp (https://github.com/leejet/stable-diffusion.cpp which is using ggml-based engine). Part of #1126
To test it:
Notes for Reviewers
It does not wire up all the options available to stablediffusion.cpp, the purpose of this PR is to introduce the backend first, optimizations are left as an exercise for later.
Signed commits