Wav2Lip: Accurately Lip-syncing Videos #2417
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
aleksandr-mokrov
requested review from
a team and
eaidova
and removed request for
a team
notebooks/wav2lip/wav2lip.ipynb
Outdated
| @@ -0,0 +1,326 @@ | |||
| { | |||
There was a problem hiding this comment.
notebooks/wav2lip/wav2lip.ipynb
Outdated
| @@ -0,0 +1,326 @@ | |||
| { | |||
There was a problem hiding this comment.
Line #17. pip_install("-q", "openvino>=2024.3.0")
2024.4.0,
also not sure how contrib and general opencv packages interact with each other (if I right remember contrib includes base and override content, but I may mistaken)
Reply via ReviewNB
There was a problem hiding this comment.
openvino version is updated. I checked opencv packages, they are installing independently.
notebooks/wav2lip/wav2lip.ipynb
Outdated
| @@ -0,0 +1,326 @@ | |||
| { | |||
There was a problem hiding this comment.
Line #12. sys.path.append(str(wav2lip_path))
as we discussed sometime ago, it is better to use insert(0) instead to append for avoiding mixing packages in env and too long variables cutting on windows
Reply via ReviewNB
notebooks/wav2lip/wav2lip.ipynb
Outdated
| @@ -0,0 +1,326 @@ | |||
| { | |||
There was a problem hiding this comment.
can you please show reference audio too? it is also better to include logs for this cells that users can see expected result (or maybe provide some static example of expected result in markdown if you think that using videos makes notebook size too large)
Reply via ReviewNB
There was a problem hiding this comment.
Output is added, they are not too large: 2643KB is size of notebook in total
notebooks/wav2lip/ov_inference.py
Outdated
| @@ -0,0 +1,554 @@ | |||
| from os import listdir, path | |||
There was a problem hiding this comment.
do we need separated file for inference? maybe it can be combined into one file with helper?
There was a problem hiding this comment.
bbox and ov_inference are merged
notebooks/wav2lip/ov_inference.py
Outdated
| img = torch.from_numpy(img).float().to(device) | ||
| BB, CC, HH, WW = img.size() | ||
|
|
||
| results = net({"x": img.numpy()}) |
There was a problem hiding this comment.
slightly strange manipulation from numpy to torch and back without any actions between, ov model accepts torch tensors as well as numpy as input
There was a problem hiding this comment.
converting to numpy is removed
notebooks/wav2lip/bbox.py
Outdated
| @@ -0,0 +1,126 @@ | |||
| from __future__ import print_function | |||
There was a problem hiding this comment.
not sure that this import required, it is for old pytorch compatibility <3.6) and the same comment, can this file content be combined with other? too much helpers requires download more files for running notebook as standalone and makes code understanding less clear. it makes sense to separate them only for case if some functional may be optional (e.g. quantization utilities, user may skip quantization step and in this case this fucntions will not be used)
There was a problem hiding this comment.
removed, bbox and ov_inference are merged
| - Convert the original model to OpenVINO Intermediate Representation (IR) format | ||
| - Compiling models and prepare pipeline | ||
| - Interactive inference | ||
|
|
There was a problem hiding this comment.
can you please put some images/animations related to inference results?
There was a problem hiding this comment.
This animation was added. In notebook properties previous one was replaced by this
CVS-152780