KeyError: 'position_dependent_phones' #18

TakayukiNagamine · 2023-01-31T15:43:31Z

TakayukiNagamine
Jan 31, 2023

Hello everyone, I would be grateful if I could seek any help regarding the error "KeyError: 'position_dependent_phones'" when aligning Japanese data. I'm trying to forced-align audio files, each of which contains a single word in Japanese, but I keep getting the error message KeyError: 'position_dependent_phones'. I thought it might be to do with the dictionary, as the pre-trained dictionary does not contain the words in my corpus, but developing a new dictionary didn't help. Does anyone possibly know what may possibly be causing this? I'm not sure what levels of details I need to give but I'd be happy to provide further information if helpful.

Version: MFA 2.0.6
Language/acoustic model: v2_0_1a
Dictionary: Japanese MFA dictionary v2_0_1a then I trained my own based on the corpus using Japanese MFA G2P model v2.0.1a

Thanks,

Takayuki Nagamine

mmcauliffe · 2023-01-31T21:46:12Z

mmcauliffe
Jan 31, 2023
Maintainer

This would be a bug in MFA not with the dictionary. Can you run with --verbose and --debug and paste the full error message to see where it's happening? I'm trying wrap up MFA 2.1 which is the version the japanese model was trained, so this should get resolved once I can release 2.1.

0 replies

mmcauliffe · 2023-01-31T22:14:03Z

mmcauliffe
Jan 31, 2023
Maintainer

Ok I think I tracked down where it was throwing the error, and I've updated the zip file for release. Can you redownload the acoustic model via mfa or downloading from https://github.com/MontrealCorpusTools/mfa-models/releases/tag/acoustic-japanese_mfa-v2.0.1a and see if that fixed it?

5 replies

TakayukiNagamine Jan 31, 2023
Author

Thank you so much for looking into this so quickly! I've tried running it after downloading the acoustic model again, but unfortunately it doesn't seem working. Here is the full error message:

/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/Bio/pairwise2.py:278: BiopythonDeprecationWarning: Bio.pairwise2 has been deprecated, and we intend to remove it in a future release of Biopython. As an alternative, please consider using Bio.Align.PairwiseAligner as a replacement, and contact the Biopython developers if you still need the Bio.pairwise2 module.
warnings.warn(
DEBUG - Beginning run for pretrained_aligner on corpus
DEBUG - Using multiprocessing with 3
DEBUG - Set up logger for MFA version: 2.0.6
DEBUG - Cleaned previous run
WARNING - Skipped the following configuration keys: dither, energy_floor, num_coefficients, num_mel_bins,
cepstral_lifter, preemphasis_coefficient, and use_voicing
DEBUG - There were some differences in the current run compared to the last one. This may cause issues, run with
--clean, if you hit an error.
ERROR - There was an error in the run, please see the log.
Exception ignored in atexit callback: <bound method ExitHooks.history_save_handler of <montreal_forced_aligner.command_line.mfa.ExitHooks object at 0x109efba90>>
Traceback (most recent call last):
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/command_line/mfa.py", line 103, in history_save_handler
raise self.exception
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/bin/mfa", line 11, in
sys.exit(main())
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/command_line/mfa.py", line 1062, in main
run_align_corpus(args, unknown)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/command_line/align.py", line 107, in run_align_corpus
align_corpus(args, unknown_args)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/command_line/align.py", line 40, in align_corpus
aligner.align()
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/alignment/pretrained.py", line 347, in align
self.setup()
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/alignment/pretrained.py", line 191, in setup
self.setup_acoustic_model()
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/alignment/pretrained.py", line 93, in setup_acoustic_model
self.position_dependent_phones = dict_info["position_dependent_phones"]
KeyError: 'position_dependent_phones'

mmcauliffe Jan 31, 2023
Maintainer

And just to double check, can you rerun the download command with --ignore_cache that should force the redownload (https://montreal-forced-aligner.readthedocs.io/en/latest/user_guide/models/index.html#cmdoption-mfa-model-download-ignore_cache)

TakayukiNagamine Jan 31, 2023
Author

Thanks, I redownloaded the acoustic model flagging --ignore cache it seems getting better! It indeed managed to go further, but one final error prevented it from completing the process, with the error message below:

INFO - Exporting TextGrids to /Volumes/Samsung_T5/PhD_Data_Analysis/data/8_forced_alignment/02_batabata/output...
INFO - Collecting phone and word alignments from alignment lattices...
0%| | 0/138 [00:00<?, ?it/s]/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/Bio/pairwise2.py:278: BiopythonDeprecationWarning: Bio.pairwise2 has been deprecated, and we intend to remove it in a future release of Biopython. As an alternative, please consider using Bio.Align.PairwiseAligner as a replacement, and contact the Biopython developers if you still need the Bio.pairwise2 module.
warnings.warn(
0%| | 0/138 [00:02<?, ?it/s]
ERROR - There was an error in the run, please see the log.
Exception ignored in atexit callback: <bound method ExitHooks.history_save_handler of <montreal_forced_aligner.command_line.mfa.ExitHooks object at 0x1097e3a90>>
Traceback (most recent call last):
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/command_line/mfa.py", line 103, in history_save_handler
raise self.exception
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/bin/mfa", line 11, in
sys.exit(main())
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/command_line/mfa.py", line 1062, in main
run_align_corpus(args, unknown)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/command_line/align.py", line 107, in run_align_corpus
align_corpus(args, unknown_args)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/command_line/align.py", line 42, in align_corpus
aligner.export_files(
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/alignment/base.py", line 655, in export_files
self.export_textgrids(output_format, include_original_text)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/alignment/base.py", line 506, in export_textgrids
self._collect_alignments()
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/alignment/base.py", line 441, in _collect_alignments
raise v
montreal_forced_aligner.exceptions.MultiprocessingError: MultiprocessingError:

Job 0 encountered an error:
Traceback (most recent call last):

File 
"/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/abc.py", 
line 73, in run
  yield from self._run()

File 
"/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/alignment/multiprocessing.py",
 line 997, in _run
  self.check_call(nbest_proc)

File 
"/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/abc.py", 
line 100, in check_call
  raise KaldiProcessingError([self.log_path])

montreal_forced_aligner.exceptions.KaldiProcessingError: KaldiProcessingError:

There were 1 job(s) with errors when running Kaldi binaries.
See the log files below for more information.
/Users/takayukinagamine/Documents/MFA/corpus_pretrained_aligner/pretrained_aligner/log/get_phone_ctm.0.lo

mmcauliffe Feb 2, 2023
Maintainer

Ok, I should have 2.1 out later tonight, so that should address this since I have been generating alignments with the Japanese model on my local machine.

TakayukiNagamine Feb 2, 2023
Author

Thanks so much! Will try it out and let you know how things go!

TakayukiNagamine · 2023-02-06T15:33:49Z

TakayukiNagamine
Feb 6, 2023
Author

It seems now that I have installed the upgraded MFA (2.1), but it starts spitting another kind of error message. Now the error persists for the English annotation files that worked well in the previous version. This may not be to do with the issue of the Japanese model we're talking about, but would you mind looking into the error message if possible?

The corpus structure should not be a problem - it has annotated textgrids for each wav file, the file names corresponding to each other. I haven't changed anything from running the aligner in the previous version. It used to both validate and align the folder ok.

Thanks so much in advance for your help!

(aligner) takayukinagamine@TakayukinoMacBook-Pro ~ % mfa validate /Volumes/Samsung_T5/PhD_Data_Analysis/data/8_forced_alignment/believe/corpus english_us_arpa english_us_arpa
ERROR - There was an error in the run, please see the log.
Exception ignored in atexit callback: <bound method ExitHooks.history_save_handler of <montreal_forced_aligner.command_line.mfa.ExitHooks object at 0x10dd22c80>>
Traceback (most recent call last):
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/command_line/mfa.py", line 93, in history_save_handler
raise self.exception
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/bin/mfa", line 11, in
sys.exit(mfa_cli())
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/command_line/validate.py", line 119, in validate_corpus_cli
validator.cleanup()
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/abc.py", line 646, in cleanup
self.save_worker_config()
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/abc.py", line 652, in save_worker_config
with mfa_open(self.worker_config_path, "w") as f:
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/contextlib.py", line 135, in enter
return next(self.gen)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/helper.py", line 64, in mfa_open
file = open(path, mode, encoding=encoding, newline=newline)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/takayukinagamine/Documents/MFA/corpus/corpus.yaml'

4 replies

mmcauliffe Feb 6, 2023
Maintainer

It looks like something's going wrong before it has a chance to create any of the temporary directories. I'll fix that up, but in the mean time, can you manually create the folder "/Users/takayukinagamine/Documents/MFA/corpus/" and rerun it to see what the original error is?

mmcauliffe Feb 6, 2023
Maintainer

Can you also try creating a whole new environment from scratch to make sure everything's correct there? The steps I got working on a brand new set up are:

conda create -n mfa python=3.9 montreal-forced-aligner -c conda-forge
mfa configure --database_port 5433 (I'll update MFA to use this port by default to make sure there are no conflicts with other servers)
mfa models download dictionary japanese_mfa
mfa models download acoustic japanaese_mfa
mfa validate /path/to/a/japanese/corpus japanese_mfa

TakayukiNagamine Feb 7, 2023
Author

Thanks so much for the suggestions! I've tried it but they didn't work unfortunately. Thought it was an issue with iCloud - the document folder is also uploaded to iCloud for backup, but downloading MFA onto the local computer didn't solve the issue. I continue to have the same error message: FileNotFoundError: [Errno 2] No such file or directory: '/Users/takayukinagamine/Documents/MFA/corpus/corpus.yaml'

TakayukiNagamine Feb 7, 2023
Author

If this helps, MFA keeps deleting the directory /Users/takayukinagamine/Documents/MFA/corpus/ whenever I create the folder manually, returning the 'No such file or directory' error above. And note that I'm installing/trying to run the aligner on the local hard drive, having disconnected it from iCloud now.

mmcauliffe · 2023-02-07T16:42:11Z

mmcauliffe
Feb 7, 2023
Maintainer

Can you try updating to the latest 2.1.2? I did add an extra check so that it wouldn't try to write the file if the directly doesn't exist, and fixed a few other bugs.

…

On Tue, Feb 7, 2023, 5:22 AM Takayuki Nagamine ***@***.***> wrote: If this helps, MFA keeps deleting the directory /Users/takayukinagamine/Documents/MFA/corpus/ whenever I create the folder manually, returning the 'No such file or directory' error above. And note that I'm installing/trying to run the aligner on the local hard drive, having disconnected it from iCloud now. — Reply to this email directly, view it on GitHub <#18 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVJOT6MSJQGR6MVA7EMKH3WWJD75ANCNFSM6AAAAAAUMTABRI> . You are receiving this because you commented.Message ID: <MontrealCorpusTools/mfa-models/repo-discussions/18/comments/4893507@ github.com>

5 replies

TakayukiNagamine Feb 7, 2023
Author

Thanks, I'm trying it now. Just some stupid questions, but does running the installation code conda create -n aligner -c conda-forge montreal-forced-aligner update MFA automatically? And is there any command that I could run to check the installed version? (I randomly type in something like 'mfa version', which somehow worked for 2.0.6 but not currently.)

mmcauliffe Feb 7, 2023
Maintainer

Oh yes, the version subcommand didn't get transferred, I'll fix that for the next release. But you should be able to see the version in the conda list command, but yes creating a new environment should update MFA automatically. You might want delete the temporary ~/Documents/MFA (and redownload the model and dictionary), just to make sure some lingering configuration or files from the previous versions don't cause issues.

TakayukiNagamine Feb 8, 2023
Author

I see! Now I can see that my version is indeed 2.1.1. using conda list.

I did it again - the directory error has been suppressed but now there's a new one I'm afraid. I looked at the suggested URL to see what the problem is and they say that the issue might be an unexpected one.

Is there any chance that these errors are caused by the local settings of my laptop? If possible, would you be able to try validating/aligning one of my corpus folders to see if it works on your side? (Don't worry if this is too much for you!)

Thank you so much once again for all your help!

Takayuki

(aligner) takayukinagamine@TakayukinoMacBook-Pro ~ % mfa validate /Volumes/Samsung_T5/PhD_Data_Analysis/data/8_forced_alignment/02_batabata/corpus japanese_mfa japanese_mfa --clean
ERROR - There was an error in the run, please see the log.
Exception ignored in atexit callback: <bound method ExitHooks.history_save_handler of <montreal_forced_aligner.command_line.mfa.ExitHooks object at 0x10def2c50>>
Traceback (most recent call last):
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/command_line/mfa.py", line 93, in history_save_handler
raise self.exception
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/bin/mfa", line 11, in
sys.exit(mfa_cli())
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/command_line/validate.py", line 114, in validate_corpus_cli
validator.validate()
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/validation/corpus_validator.py", line 656, in validate
self.setup()
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/validation/corpus_validator.py", line 600, in setup
self.initialize_database()
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/abc.py", line 226, in initialize_database
conn.execute(sqlalchemy.text("CREATE EXTENSION IF NOT EXISTS vector"))
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1414, in execute
return meth(
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 489, in _execute_on_connection
return connection._execute_clauseelement(
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1638, in _execute_clauseelement
ret = self._execute_context(
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1842, in _execute_context
return self._exec_single_context(
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1983, in _exec_single_context
self._handle_dbapi_exception(
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2325, in _handle_dbapi_exception
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1964, in _exec_single_context
self.dialect.do_execute(
File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 747, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.

[SQL: CREATE EXTENSION IF NOT EXISTS vector]
(Background on this error at: https://sqlalche.me/e/20/e3q8)

mmcauliffe Feb 8, 2023
Maintainer

So I'm seeing this on another Mac user's set up: MontrealCorpusTools/Montreal-Forced-Aligner#552, so I'm going to try to get a mac install set up and try to debug it there. Sorry about all these issues! Thanks for being patient and testing it out for me!

TakayukiNagamine Feb 8, 2023
Author

Ah, this takes us to that thread! I've seen it before - didn't think it was relevant then but now it is! Thank you from me as well for patiently tackling the issues for me and I'm glad that I've been able to help you out only a tiny bit!

TakayukiNagamine · 2023-02-09T14:38:13Z

TakayukiNagamine
Feb 9, 2023
Author

I've tried conda update pgvector montreal-forced-aligner and tried validating the Japnaese corpus on the latest version of MFA (2.1.3) but unfortunately am still getting the same error reported above and also in MontrealCorpusTools/Montreal-Forced-Aligner#552. The corpus folder issue seems to have been resolved as I can see the folder in the directory ~/Document/MFA/. I do have the vector.control file in /Users/takayukinagamine/opt/anaconda3/envs/aligner/share/extension/vector.control.

5 replies

TakayukiNagamine Feb 9, 2023
Author

Sorry for spamming, but just curious if there's a way to revert back to an older version of MFA? I have some English data I aligned using 2.0.6 and while the error persists for the new version I thought it might be useful for me to be able to work on the older version for some more alignment work. Thanks so much once again for all your help and efforts in troubleshooting!

mmcauliffe Feb 9, 2023
Maintainer

No worries! The command conda create -n mfa_206 montreal-forced-aligner=2.0.6 praatio=5.1.1 sqlalchemy<2.0 should give you a workable 2.0.6 environment.

mmcauliffe Feb 11, 2023
Maintainer

Not sure if you saw on MontrealCorpusTools/Montreal-Forced-Aligner#552, but the latest versions via conda update pgvector montreal-forced-aligner should be working for apple M1's now.

TakayukiNagamine Feb 14, 2023
Author

Thanks so much Michael! It's reassuring that I could continue to discuss this with you here (but sorry for taking your time for this!).

I've tried the line conda update pgvector montreal-forced-aligner and run version 2.1.7 but it throws another error for the Japanese files as well as for the English corpus (using the english_us_arpa acoustic model and dictionary) that used to have no issues with MFA 2.0.6. I also saw a chat on Twitter and this may be to do with my M1 mac, so maybe I'd wait for another update to come into effect? Here is what I get in case this illustrates the issue. Any inputs from you would be much appreciated - after all it might just be me. Thanks a lot!

Here are my codes and the error I got for the validation command:


(aligner) takayukinagamine@TakayukinoMacBook-Pro ~ % conda update pgvector montreal-forced-aligner
(aligner) takayukinagamine@TakayukinoMacBook-Pro ~ % mfa model download acoustic japanese_mfa --ignore_cache
INFO - Saved model to /Users/takayukinagamine/Documents/MFA/pretrained_models/acoustic/japanese_mfa.zip, you can now use japanese_mfa in place of acoustic paths in mfa commands.
(aligner) takayukinagamine@TakayukinoMacBook-Pro ~ % mfa model download dictionary japanese_mfa --ignore_cache
INFO - Saved model to /Users/takayukinagamine/Documents/MFA/pretrained_models/dictionary/japanese_mfa.dict, you can now use japanese_mfa in place of dictionary paths in mfa commands.
(aligner) takayukinagamine@TakayukinoMacBook-Pro ~ % conda list | grep pgvector
pgvector                  0.4.0                hec0c9b1_3    conda-forge
pgvector-python           0.1.6              pyhb2dc1fd_2    conda-forge
(aligner) takayukinagamine@TakayukinoMacBook-Pro ~ % mfa validate /Volumes/Samsung_T5/PhD_Data_Analysis/data/8_forced_alignment/02_batabata/corpus japanese_mfa japanese_mfa --clean```

 
Exception ignored in atexit callback: <bound method ExitHooks.history_save_handler of <montreal_forced_aligner.command_line.mfa.ExitHooks object at 0x10de3ac20>>
Traceback (most recent call last):
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/command_line/mfa.py", line 93, in history_save_handler
    raise self.exception
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/bin/mfa", line 11, in <module>
    sys.exit(mfa_cli())
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/command_line/validate.py", line 110, in validate_corpus_cli
    validator = TrainingValidator(
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/validation/corpus_validator.py", line 524, in __init__
    super().__init__(**kwargs)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/acoustic_modeling/trainer.py", line 189, in __init__
    super().__init__(**kwargs)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/transcription/transcriber.py", line 141, in __init__
    super().__init__(**kwargs)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/alignment/base.py", line 107, in __init__
    super().__init__(**kwargs)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/corpus/acoustic_corpus.py", line 1170, in __init__
    super().__init__(**kwargs)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/corpus/acoustic_corpus.py", line 110, in __init__
    super().__init__(**kwargs)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/corpus/base.py", line 114, in __init__
    super().__init__(**kwargs)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/abc.py", line 506, in __init__
    self.setup_logger()
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/abc.py", line 736, in setup_logger
    conf = load_configuration(self.worker_config_path)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/montreal_forced_aligner/helper.py", line 91, in load_configuration
    data = yaml.load(f, Loader=yaml.SafeLoader)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/yaml/constructor.py", line 51, in get_single_data
    return self.construct_document(node)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/yaml/constructor.py", line 60, in construct_document
    for dummy in generator:
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/yaml/constructor.py", line 413, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/yaml/constructor.py", line 218, in construct_mapping
    return super().construct_mapping(node, deep=deep)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/yaml/constructor.py", line 143, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/yaml/constructor.py", line 100, in construct_object
    data = constructor(self, node)
  File "/Users/takayukinagamine/opt/anaconda3/envs/aligner/lib/python3.10/site-packages/yaml/constructor.py", line 427, in construct_undefined
    raise ConstructorError(None, None,
yaml.constructor.ConstructorError: could not determine a constructor for the tag 'tag:yaml.org,2002:python/object/apply:pathlib.PosixPath'
  in "/Users/takayukinagamine/Documents/MFA/corpus/corpus.yaml", line 1, column 19```

mmcauliffe Feb 16, 2023
Maintainer

Ok, I think everything's more stable now with 2.2.1. I also added tokenizer models that should reflect the tokenization that I've done on the Japanese training pretty well, along with a short guide to using it here: https://montreal-forced-aligner.readthedocs.io/en/latest/first_steps/index.html#first-steps-tokenize. Hopefully this time it works for you!

TakayukiNagamine · 2023-05-12T15:44:02Z

TakayukiNagamine
May 12, 2023
Author

Hello Michael, Sorry that it's been a while since the last post but been a little busy getting something else sorted from February until last week. Good news, MFA is working for my Japanese data! Thank you so much for all this. There are some glitches but they may be to do with the wav files. Some of the words I'm forced-aligning do not seem to be included in the dictionary so I may have to do them separately. Will try this a little more and let you know if any issues persist.

0 replies

mmcauliffe · 2023-05-13T03:00:11Z

mmcauliffe
May 13, 2023
Maintainer

Sounds good, glad to hear it! I am planning on rolling out some better Japanese (and other languages) support with doing tokenization via spacy's https://spacy.io/usage/models. There are a couple of bigger prerequisite changes that have to happen before then, but ideally you should just be able to use regular Japanese text as input and it'll generate tokenized transcripts.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KeyError: 'position_dependent_phones' #18

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 7 comments 19 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

KeyError: 'position_dependent_phones' #18

TakayukiNagamine Jan 31, 2023

Replies: 7 comments · 19 replies

mmcauliffe Jan 31, 2023 Maintainer

mmcauliffe Jan 31, 2023 Maintainer

TakayukiNagamine Jan 31, 2023 Author

mmcauliffe Jan 31, 2023 Maintainer

TakayukiNagamine Jan 31, 2023 Author

mmcauliffe Feb 2, 2023 Maintainer

TakayukiNagamine Feb 2, 2023 Author

TakayukiNagamine Feb 6, 2023 Author

mmcauliffe Feb 6, 2023 Maintainer

mmcauliffe Feb 6, 2023 Maintainer

TakayukiNagamine Feb 7, 2023 Author

TakayukiNagamine Feb 7, 2023 Author

mmcauliffe Feb 7, 2023 Maintainer

TakayukiNagamine Feb 7, 2023 Author

mmcauliffe Feb 7, 2023 Maintainer

TakayukiNagamine Feb 8, 2023 Author

mmcauliffe Feb 8, 2023 Maintainer

TakayukiNagamine Feb 8, 2023 Author

TakayukiNagamine Feb 9, 2023 Author

TakayukiNagamine Feb 9, 2023 Author

mmcauliffe Feb 9, 2023 Maintainer

mmcauliffe Feb 11, 2023 Maintainer

TakayukiNagamine Feb 14, 2023 Author

mmcauliffe Feb 16, 2023 Maintainer

TakayukiNagamine May 12, 2023 Author

mmcauliffe May 13, 2023 Maintainer

TakayukiNagamine
Jan 31, 2023

Replies: 7 comments 19 replies

mmcauliffe
Jan 31, 2023
Maintainer

mmcauliffe
Jan 31, 2023
Maintainer

TakayukiNagamine Jan 31, 2023
Author

mmcauliffe Jan 31, 2023
Maintainer

TakayukiNagamine Jan 31, 2023
Author

mmcauliffe Feb 2, 2023
Maintainer

TakayukiNagamine Feb 2, 2023
Author

TakayukiNagamine
Feb 6, 2023
Author

mmcauliffe Feb 6, 2023
Maintainer

mmcauliffe Feb 6, 2023
Maintainer

TakayukiNagamine Feb 7, 2023
Author

TakayukiNagamine Feb 7, 2023
Author

mmcauliffe
Feb 7, 2023
Maintainer

TakayukiNagamine Feb 7, 2023
Author

mmcauliffe Feb 7, 2023
Maintainer

TakayukiNagamine Feb 8, 2023
Author

mmcauliffe Feb 8, 2023
Maintainer

TakayukiNagamine Feb 8, 2023
Author

TakayukiNagamine
Feb 9, 2023
Author

TakayukiNagamine Feb 9, 2023
Author

mmcauliffe Feb 9, 2023
Maintainer

mmcauliffe Feb 11, 2023
Maintainer

TakayukiNagamine Feb 14, 2023
Author

mmcauliffe Feb 16, 2023
Maintainer

TakayukiNagamine
May 12, 2023
Author

mmcauliffe
May 13, 2023
Maintainer