improvelog.txt

Gmk0
NN Policy
 input 15*15*2
 conv1 3*3*2*16 SAME relu
 conv2 3*3*16*48 SAME relu
 conv3 3*3*48*96 SAME relu
 conv4 3*3*96*96 SAME relu
 conv5 3*3*96*96 SAME relu
 conv6 3*3*96*96 SAME relu
 conv7 3*3*96*96 SAME relu
 conv8 3*3*96*1 SAME relu
 fc1 reshape conv8 15*15
 y softmax fc1
Loss cross_enptroy
NN Value
 input 15*15*2
 conv1 3*3*2*16 SAME relu
 conv2 3*3*16*48 SAME relu
 conv3 3*3*48*96 SAME relu
 conv4 3*3*96*96 SAME relu
 conv5 3*3*96*96 SAME relu
 conv6 3*3*96*96 SAME relu
 conv7 3*3*96*96 SAME relu
 conv8 3*3*96*1 SAME relu
 fc1 reshape conv8 225 relu
 y matmul fc1
Loss avg_sqr

GameRecord
  int stepcount
  int[stepcount] move form 0 to 225
  int winner 0,1,2

GameTrainData
  int stepcount
  int[stepcount] move
  pos
    prob, move
  pos policy[stepcount][225]   sigma policy=1
  int winner for z
  
mini_batch
  int count
  GameTrainData[count]

12-30
  target: conv->res
  +bn, +dual 

1-22
  starting SL test
  problem: speed of mcts too slow

1-23
  SL test ok
  completed NN estimate
  generate data test
  optimized mcts selection, 2-3x speed up

1-24
  self dataset ok, but found a few problem
  add dirlect noise
  start iteration 0

1-25
  found small fault at iteration 0
  output of winner reversed(correct by retransdata)
  a small fault in SL data transform and a big fault in It0 data trans.(fixed)

1-26
  iteration 0 ok
  test shows decay_reward is important (decay=0.85)
  weight_0 vs random
w    41         9
b    44         6
85% PASS

1-27
  optimzed tree search, 3x speed up

1-28
  training I1, decay=0.88 le=0.05 step=16k
  weight_1 vs weight_0
w    29         21
b    32         18
61% PASS
  tringing I2, decay=0.88 le=0.05 step=16k

1-29
  weight_2 vs weight_1
w    33         17
b    32         18
65% PASS
  training I3, decay=0.88 le=0.05 step=16k
weight_3 vs weight_2
w    30         20
b    29         21
59% PASS
weight_3 vs weight_1
w    37         13
b    37         13
extra test ok
training I4, decay=0.88 le=0.05 step=16k
weight_4 vs weight_3
w    31         19
b    43         7
74% PASS
1-30
training I5, decay=0.88 le=0.05 step=16k
weight_5 vs weight_4
w    29         21
b    30         20
59% PASS
training I6, decay=0.89 le=0.05 step=16k
w    36         14
b    32         18
68% PASS
1-31
training I7, decay=0.89 le=0.05 step=16k
w    49         1
b    35         15
84% PASS
training I8, decay=0.89 le=0.05 step=16k
w    47         3
b    26         24
73% PASS
2-1
fpu frac changed to 1.3
training I9, decay=0.89 le=0.05 step=16k
w    36         14
b    26         24
62%PASS
training I10, decay=0.89 le=0.05 step=16k
w    45         5
b    15         35 
60%PASS
2-2
training I11, decay=0.89 le=0.05 step=16k
w    43         7
b    24         26
67%PASS
find huge bug in data_gathering, trying to fix it
(Iteration 1-11 will strongly effected)
bug fixed
training I12, decay=0.89 le=0.05 step=16k
w    43         7
b    41         9
84%PASS
  weight_12 vs  SL   test
w    14         36
b    14         36
good progress!
2-3
training I13, decay=0.9 le=0.05 step=16k
w    41         9
b    48         2
89%PASS
training I14, decay=0.9 le=0.05 step=16k
w    35         15
b    44         6
79%PASS
2-4
training I15, decay=0.92 le=0.05 step=16k
w    35         15
b    41         9
76%PASS
training I16, decay=0.93 le=0.05 step=16k
w    36         14
b    45         5
81%PASS
  weight_16 vs  SL   test
w    23         27
b    27         23
50% maybe overfitting
  weight_16_puct=0.8 vs weight_16_puct=1.6
w    26         24
b    37         13
  weight_16_puct=1.6 vs weight_16_puct=1.6
w    10         40
b    47         3
  weight_16_puct=1.2 vs weight_16_puct=1.6
w    12         38
b    36         14
  weight_16_puct=0.5 vs weight_16_puct=1.6
w    14         36
b    32         18
puct now set to 0.8
2-5
training I17, decay=0.94 le=0.05 step=16k
w    35         15
b    44         6
79%PASS
add random transform in training
training I18, decay=0.95 le=0.05 step=16k
w    21         29
b    26         24
47%
training I19, decay=0.95 le=0.05 step=16k
w    24         26
b    37         13
61%PASS
training I20, decay=0.95 le=0.05 step=16k
w    22         28
b    36         14
58%
2-7
finished cpp search
2-8
add command line paras
2-9
add gmk protrol support
2-10
add match player
2-11
   I0 vs random
b   43    7
w   34    13
77%PASS
2-12
fixed bug, restart
   I0 vs random
b   42    8
w   34    16
76%PASS
training I1 step=16k, lr=0.05, decay=0.87, mse_factor=2.0
   I1 vs I0
b   48    2
w   46    4
94%PASS
2-13
selfplay I1, puct=1
training I2 step=16k, lr=0.05, decay=0.89, mse_factor=2.0
training I2z step=16k, lr=0.05, decay=0.89, mse_factor=1.0
   I2 vs I1
b   50    0
w   47    3
97%PASS
I2z vs I2 failed

training I3 step=16k, lr=0.05, decay=0.89, mse_factor=2.0
training I3z step=16k, lr=0.05, decay=0.89, mse_factor=1.0
training I3-6000 step=6k, lr=0.05, decay=0.89, mse_factor=2.0
training I3p step=16k, lr=0.02, decay=0.89, mse_factor=2.0
training I3d step=16k, lr=0.05, decay=0.92, mse_factor=2.0
    I3 vs I2
b   49    1
w   40    10
89%PASS
    I3z vs I3
b   35    15
w   15    35
    I3-6000 vs I3
b   13    37
w   4     46
    I3p vs I3
b   41    9
w   12    38
    I3d vs I3
b   35    15
w   14    36
2-14
training I4 step=16k, lr=0.03, decay=0.91, mse_factor=1.5
    I4 vs I3
b   47    3
w   21    29
68% PASS
training I5 step=16k, lr=0.03, decay=0.92, mse_factor=1.5
    I5 vs I4
b   46    4
w   37    13
83% PASS
2-15
training I6 step=16k, lr=0.03, decay=0.92, mse_factor=1.5
    I6 vs I5
b   38    12
w   27    23
65% PASS
training I7 step=16k, lr=0.03, decay=0.93, mse_factor=1.5
    I7 vs I6
b   36    14
w   21    29
57% PASS
training I8 step=16k, lr=0.03, decay=0.95, mse_factor=1.5
    I8 vs I7
b   39    11
w   30    20
69% PASS
2-16
training I9 step=16k, lr=0.03, decay=0.95, mse_factor=1.5, bs=192
    I9 vs I8
b   33    17
w   34    16
67% PASS
training I10 step=16k, lr=0.03, decay=0.96, mse_factor=1.5, bs=192
    I10 vs I9
b   42    8
w   29.5  20.5
71.5% PASS
2-17
training I11 step=16k, lr=0.03, decay=0.97, mse_factor=1.5, bs=192
    I11 vs I10
b   31    19
w   31.5  18.5
62.5% PASS
2-18
training I12 step=16k, lr=0.03, decay=0.97, mse_factor=1, bs=192
    I12 vs I11
b   35    15
w   36    14
71% PASS
training I13 step=16k, lr=0.03, decay=0.975, mse_factor=0.6, bs=192
    I13 vs I12
b   37    13
w   28    22
62% PASS
2-19
training I14_0 step=16k, lr=0.03, decay=0.975, mse_factor=1, bs=192 56%
training I14_0 step=16k, lr=0.03, decay=0.975, mse_factor=0.6, bs=192 55%
training I14 step=16k, lr=0.03, decay=0.98, mse_factor=1, bs=192
    I14 vs I13
b   35  
w   26 
puct:1.3->1.4
training I15 step=16k, lr=0.03, decay=0.98, mse_factor=0.6, bs=192
    I15 vs I14
b   41  
w   29 
2-20
training I16 step=16k, lr=0.03, decay=0.98, mse_factor=0.6, bs=192
    I16 vs I15
b   34  
w   31
2-21
training I17 step=16k, lr=0.06, decay=0.97, mse_factor=1, bs=192
    I17 vs I16
b   39
w   23
training I18 step=16k, lr=0.06, decay=0.97, mse_factor=1, bs=192
    I18 vs I17
b   39
w   24
2-22
training I20 step=16k, lr=0.06, decay=0.97, mse_factor=1, bs=192
    I20 vs I19
b   41
w   26.5
2-23
changed to 6b64f
training I21 step=16k, lr=0.05, decay=0.97, mse_factor=1, bs=192
    I21 vs I20
b   38
w   31.5
    I22 vs I21
b   36
w   25
2-24
training I23 step=16k, lr=0.05, decay=0.97, mse_factor=0.6, bs=192
    I23 vs I22
b   31
w   26
    I24 vs I23
b   41
w   26
2-25
    I25 vs I24
b   45
w   16
2-26
    I26 vs I25
b   50
w   11.5
    I27 vs I26
b   41
w   19
2-27
    I28 vs I27
b   40
w   18
training I29t1 step=2k, lr=0.005, decay=0.97, mse_factor=1, bs=128
2-28
    I29t1 vs I28
b   40
w   26
training I30 step=4k, lr=0.002, decay=0.98, mse_factor=1, bs=128
    I30 vs I29t1
b   41
w   29

3-1
    I31 vs I30
b   41
w   28

    I32 vs I31
b   35
w   23
3-2
    I33 vs I32
b   38
w   17
3-3
training I34s step=2k, lr=0.001, decay=0.98, mse_factor=1, bs=192
    I34s vs I33
b   38
w   20.5
training I35 step=4k, lr=0.001, decay=0.98, mse_factor=1, bs=128
    I35 vs I34s
b   41
w   14

3-8
puct changed to 1.6
training I38s step=2k, lr=0.002, decay=0.98, mse_factor=1, bs=128
    I38s vs J37t
b   38
w   19

training I39s step=2k, lr=0.002, decay=0.98, mse_factor=1, bs=192
    I39s vs I38s
b   36
w   17

3-9
add mc_win output
current target_Q: z^decay
new target_Q(test): alpha * z^dacay + (1-alpha)*(mc_win^mc_decay)
alpha 0.6
decay first_z 0.4
mcdecay 0.75
tau1 0.8 tau2 0.6

training I40f step=2k, lr=0.002, decay=0.98, mse_factor=1, bs=128
    I40s vs I39s
b   38
w   16