Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Turn on Audio Sync for audio extraction process #2648

Merged
merged 2 commits into from
Sep 22, 2024

Conversation

dvh312
Copy link
Contributor

@dvh312 dvh312 commented Sep 2, 2024

Problem description

  • I used Whisper AI and recognized that the subtitles generated didn't have the correct timestamps, especially near the end of the video.

Cause

  1. The video files I'm using have some corrupted frames.
  2. ffmpeg try to "skip" these frames, causing the duration of the extracted audio to be significantly shorter than the original video duration.

Solution

  • Turning on audio sync to make the extracted audio matches the video timestamps.

Screenshots

Original command

ffmpeg  -i input.mp4  -acodec pcm_s16le -ac 1 -ar 16000 -f s16le output.wav 

ffmpeg-before.log

Imported to Audacity, only shows the duration of 22m52s532ms. The video duration is 22m59s960ms. This means subtitles will be shifted by almost 7 seconds ahead.

Screenshot 2024-09-01 at 5 23 34 PM Screenshot 2024-09-01 at 5 24 23 PM

Updated command

ffmpeg  -i input.mp4  -acodec pcm_s16le -ac 1 -ar 16000 -f s16le -af aresample=async=1 output.wav 

ffmpeg-after.log

Audacity now shows the duration of 22m59s349ms, which is approximately the original video duration of 22m59s960ms

Screenshot 2024-09-01 at 5 28 37 PM

Copy link
Owner

@morpheus65535 morpheus65535 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can't patch files in libs directory since they'll get overwritten when update update dependencies. You must find a way to do it from the provider itself or make a PR to ffprobe module itself, get it merged and we'll get it in next dependencies update cycle.

@anderson-oki
Copy link
Collaborator

@dvh312 Morpheus means you can't, there is a typo.

@dvh312
Copy link
Contributor Author

dvh312 commented Sep 16, 2024

@morpheus65535 Thanks, let me find a way to do it in the provider.

@dvh312
Copy link
Contributor Author

dvh312 commented Sep 18, 2024

@morpheus65535 I've reverted the change in libs and will make the update to ffsubsync directly.

@morpheus65535 morpheus65535 merged commit c69be93 into morpheus65535:development Sep 22, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants