java.lang.AssertionError

[GoogleCodeExporter]

I'm using the unsupervised version 2.1 available from the repository. Here is 
my command line

/usr/bin/java -server \
  -Xms1024m \
  -Xmx2048m \
  -Xss768k \
  -ea \
  -jar /usr/local/bin/berkeleyaligner.jar \
  -EMWordAligner.numThreads 6 \
  -Data.trainSources /opt/library/BUILDS/tm/demo_tm/bitext.list \
  -Data.foreignSuffix nl \
  -Data.englishSuffix en \
  -Data.testSources \
  -exec.execDir /opt/library/TRAININGS/alignments/align-demo_tm-en-nl/berk.classes \
  -exec.create True \
  -Evaluator.writeGIZA True \
  -Main.SaveParams True \
  -Main.alignTraining True \
  -Main.forwardModels MODEL1 HMM \
  -Main.reverseModels MODEL1 HMM \
  -Main.iters 5 5 \
  -Main.mode JOINT JOINT

This is a small 40,000 phrase pair corpus for testing and development. The 
machine is a server with a 6-core AMD Opteron and 16 GB RAM and 1TB available 
hard drive space. Jave/OS version as follows:

  user@moses0:~$ java -version
  java version "1.6.0_20"
  OpenJDK Runtime Environment (IcedTea6 1.9.10) (6b20-1.9.10-0ubuntu1~10.04.3)
  OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)

I'm testing a workflow. So, I run the same command on the same corpus multiple 
times. Each time, the previous output folder is deleted. 

Most times, this command completes training successfully. Sometimes, however, 
it fails with an AssertionError. The location of the failure is a different 
phrase-pair each time. It always, however, fails during the first iteration of 
model 1.

main() {
  Execution directory: /opt/library/TRAININGS/alignments/align-demo_tm-en-nl/berk.classes
  Preparing Training Data [2.3s, cum. 2.4s]
  41410 training, 0 test
  Training models: 2 stages {
    Training stage 1: MODEL1 and MODEL1 jointly for 5 iterations {
      Initializing forward model [9.1s, cum. 9.1s]
      Initializing reverse model [7.9s, cum. 17s]
      Joint Train: 41410 sentences, jointly {
        Iteration 1/5 {
          Sentence 2/41410
          Sentence 1/41410
          Sentence 5/41410
          Sentence 13/41410
          WARNING: Translation model update concurrency error
          Sentence 54/41410
          WARNING: Translation model update concurrency error
          Sentence 207/41410
          WARNING: Translation model update concurrency error
          WARNING: Translation model update concurrency error
          ERROR: java.lang.AssertionError:
fig.basic.StringDoubleMap.find(StringDoubleMap.java:397)
fig.basic.StringDoubleMap.incr(StringDoubleMap.java:78)
fig.basic.String2DoubleMap.incr(String2DoubleMap.java:51)
edu.berkeley.nlp.wordAlignment.SentencePairState.updateTransProbs(SentencePairSt
ate.java:79)
edu.berkeley.nlp.wordAlignment.distortion.Model1or2SentencePairState.updateNewPa
rams(Model1or2SentencePairState.java:91)
edu.berkeley.nlp.wordAlignment.EMWordAligner$1.run(EMWordAligner.java:231)
edu.berkeley.nlp.concurrent.WorkQueue$1.run(WorkQueue.java:70)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
java.lang.Thread.run(Thread.java:636)
1 errors, 4 warnings
          ... 585 lines omitted ...
        }

Do you have any suggestions to solve this intermittent problem?

Thanks,
Tom

Original issue reported on code.google.com by tah...@gmail.com on 19 Feb 2012 at 12:44