Add quantization #1773
Add quantization #1773
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
aleksandr-mokrov
requested review from
a team,
apaniukov and
itrushkin
and removed request for
a team
|
View / edit / reply to this conversation on ReviewNB l-bat commented on 2024-02-29T14:30:58Z Please add section with comparing inference time between FP16 and INT8 pipelines aleksandr-mokrov commented on 2024-03-04T22:16:00Z I'm not sure that here is needed any specific function to calculate inference time. Generation an image in the example takes about 1 minute for fp16 model and about 30 seconds for int8 model and the notebook demonstrates it. Renamed |
|
View / edit / reply to this conversation on ReviewNB l-bat commented on 2024-02-29T14:30:59Z please fix the link view aleksandr-mokrov commented on 2024-03-04T22:06:46Z fixed |
|
View / edit / reply to this conversation on ReviewNB l-bat commented on 2024-02-29T14:31:00Z Line #7. def __init__(self, compiled_model, prob: float, data_cache: List[Any] = None):
aleksandr-mokrov commented on 2024-03-04T22:06:56Z removed |
|
View / edit / reply to this conversation on ReviewNB l-bat commented on 2024-02-29T14:31:01Z Line #18. lcm_pipeline.unet = CompiledModelDecorator(original_unet)
aleksandr-mokrov commented on 2024-03-04T22:07:04Z fixed |
|
View / edit / reply to this conversation on ReviewNB l-bat commented on 2024-02-29T14:31:02Z I think it would be better to compare the images side by side. Look at aleksandr-mokrov commented on 2024-03-04T22:07:14Z Changed |
|
View / edit / reply to this conversation on ReviewNB l-bat commented on 2024-02-29T14:31:03Z Please add an option to use quantized model. Look at aleksandr-mokrov commented on 2024-03-04T22:07:28Z Added |
|
View / edit / reply to this conversation on ReviewNB eaidova commented on 2024-03-04T05:03:10Z Line #1. from openvino.runtime import Core please align openvino imports with other notebooks aleksandr-mokrov commented on 2024-03-04T22:07:40Z Fixed eaidova commented on 2024-03-05T05:20:09Z I mean "import openvino as ov" aleksandr-mokrov commented on 2024-03-05T13:49:12Z done |
|
View / edit / reply to this conversation on ReviewNB eaidova commented on 2024-03-04T05:03:11Z Line #16. def collect_calibration_data(lcm_pipeline) -> List[Dict]: lcm has no relation to this notebook, such variable name may confuse users, please give more netral naming aleksandr-mokrov commented on 2024-03-04T22:07:53Z renamed |
|
View / edit / reply to this conversation on ReviewNB eaidova commented on 2024-03-04T05:03:12Z Line #5. logger = logging.getLogger(__name__) I do not see logger usage in notebook, why do you need to create its instance? aleksandr-mokrov commented on 2024-03-04T22:08:04Z removed |
|
@aleksandr-mokrov, I believe this PR should be closed #1703? |
|
fixed View entire conversation on ReviewNB |
|
removed View entire conversation on ReviewNB |
|
fixed View entire conversation on ReviewNB |
|
Changed View entire conversation on ReviewNB |
|
Added View entire conversation on ReviewNB |
|
Fixed View entire conversation on ReviewNB |
|
renamed View entire conversation on ReviewNB |
|
removed View entire conversation on ReviewNB |
|
I'm not sure that here is needed any specific function to calculate inference time. Generation an image in the example takes about 1 minute for fp16 model and about 30 seconds for int8 model and the notebook demonstrates it. Renamed View entire conversation on ReviewNB |
|
I mean "import openvino as ov" View entire conversation on ReviewNB |
|
View / edit / reply to this conversation on ReviewNB eaidova commented on 2024-03-05T05:23:25Z Line #115. btn.click(fn=predict, inputs=[image, reference, seed, steps, use_quantize_model], outputs=[image_out],) I think model selection should be before running otherwise it maybe big memory cost having 2 compiled models or even pipelines during generation) aleksandr-mokrov commented on 2024-03-05T13:51:03Z I added a multiselector. Unused pipeline is deleted. |
|
done View entire conversation on ReviewNB |
|
I added a multiselector. Unused pipeline is deleted. View entire conversation on ReviewNB |
|
View / edit / reply to this conversation on ReviewNB eaidova commented on 2024-03-06T05:20:18Z Line #3. if to_quantize.value: I think it is better to check that int8 model files exists instead of this flag. if notebook will be used just as demo, there is no need to quantize model every run, but using it in demo still possible aleksandr-mokrov commented on 2024-03-06T16:40:19Z Changed. Now it is possible to choose pipeline for demo, run demo, stop demo and choose another pipeline. |
|
View / edit / reply to this conversation on ReviewNB eaidova commented on 2024-03-06T05:20:19Z Line #63. pipeline = ov_pipe_inpaint if not use_quantize_model else int8_ov_pipe_inpaint it seems for me that this line will not work after removing one from pipelines. I recommend to not provide flag inside function, but use pipeline variable that will be equal to int8 or fp16 pipeline in place where you remove pipe that is not used anymore. aleksandr-mokrov commented on 2024-03-06T16:40:36Z removed |
|
Changed. Now it is possible to choose pipeline for demo, run demo, stop demo and choose another pipeline. View entire conversation on ReviewNB |
|
removed View entire conversation on ReviewNB |
CVS-129098