Dataset Tensorboard metrics

[sethtroisi]

We have a number of metrics that measure a bunch of things

• metrics about the model weights
  • l2_cost
• model accuracy on constant data
  • value_cost on pro holdout
• model accuracy on recent RL data
  • , policy_cost, policy_entropy, value_cost (diff between target and output)
  • value_confidence (what was the average value output)
• RL data
  • policy target top 1 (what percents of readouts went to the top move?)
  • average_winrate_observed

I think we should add

• value_cost_bias = value_cost(target_value, average_winrate_observed)
• average_move = average move number of the position (requires adding to the tf examples)
• search_q_error = |target_value - search_q| (requires adding to the tf example)

Anything else that would be interesting?

[amj]

this is a great idea, i'd love to have those metrics. Let's be sure to keep backwards compatibility with all the old data we have.

i can't think of other things i'd add right now.