frameworks/tensorflow/adanet_objective.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting adanet\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/04/c4/11ac106b2f8946ebe1940ebe26ef4dd212d655c4a2e28bbcc3b5312268e4/adanet-0.3.0-py2.py3-none-any.whl (65kB)\n",
      "\u001b[K    100% |################################| 71kB 353kB/s ta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: protobuf>=3.6.0 in /usr/local/lib/python3.5/dist-packages (from adanet) (3.6.1)\n",
      "Requirement already satisfied: numpy>=1.12.0 in /usr/local/lib/python3.5/dist-packages (from adanet) (1.15.2)\n",
      "Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.5/dist-packages (from adanet) (1.11.0)\n",
      "Requirement already satisfied: setuptools in /usr/local/lib/python3.5/dist-packages (from protobuf>=3.6.0->adanet) (39.1.0)\n",
      "Installing collected packages: adanet\n",
      "Successfully installed adanet-0.3.0\n",
      "\u001b[33mYou are using pip version 18.0, however version 18.1 is available.\n",
      "You should consider upgrading via the 'pip install --upgrade pip' command.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install adanet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "g9QvCK6Su4hY"
   },
   "source": [
    "##### Copyright 2018 The AdaNet Authors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "cellView": "both",
    "colab": {
     "test": {
      "output": "ignore",
      "timeout": 900
     }
    },
    "colab_type": "code",
    "id": "EUman0Uju6SR"
   },
   "outputs": [],
   "source": [
    "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "# https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "l5CQ48UOxrqO"
   },
   "source": [
    "# The AdaNet objective"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3tw35taebIOz"
   },
   "source": [
    "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/adanet/blob/master/adanet/examples/tutorials/adanet_objective.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://github.com/tensorflow/adanet/blob/master/adanet/examples/tutorials/adanet_objective.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "7kL896ITmpoR"
   },
   "source": [
    "One of key contributions from *AdaNet: Adaptive Structural Learning of Neural\n",
    "Networks* [[Cortes et al., ICML 2017](https://arxiv.org/abs/1607.01097)] is\n",
    "defining an algorithm that aims to directly minimize the DeepBoost\n",
    "generalization bound from *Deep Boosting*\n",
    "[[Cortes et al., ICML 2014](http://proceedings.mlr.press/v32/cortesb14.pdf)]\n",
    "when applied to neural networks. This algorithm, called **AdaNet**, adaptively\n",
    "grows a neural network as an ensemble of subnetworks that minimizes the AdaNet\n",
    "objective (a.k.a. AdaNet loss):\n",
    "\n",
    "$$F(w) = \\frac{1}{m} \\sum_{i=1}^{m} \\Phi \\left(\\sum_{j=1}^{N}w_jh_j(x_i), y_i \\right) + \\sum_{j=1}^{N} \\left(\\lambda r(h_j) + \\beta \\right) |w_j| $$\n",
    "\n",
    "where $w$ is the set of mixture weights, one per subnetwork $h$,\n",
    "$\\Phi$ is a surrogate loss function such as logistic loss or MSE, $r$ is a\n",
    "function for measuring a subnetwork's complexity, and $\\lambda$ and $\\beta$\n",
    "are hyperparameters.\n",
    "\n",
    "## Mixture weights\n",
    "\n",
    "So what are mixture weights? When forming an ensemble $f$ of subnetworks $h$,\n",
    "we need to somehow combine the their predictions. This is done by multiplying\n",
    "the outputs of subnetwork $h_i$ with mixture weight $w_i$, and summing the\n",
    "results:\n",
    "\n",
    "$$f(x) = \\sum_{j=1}^{N}w_jh_j(x)$$\n",
    "\n",
    "In practice, most commonly used set of mixture weight is **uniform average\n",
    "weighting**:\n",
    "\n",
    "$$f(x) = \\frac{1}{N}\\sum_{j=1}^{N}h_j(x)$$\n",
    "\n",
    "However, we can also solve a convex optimization problem to learn the mixture\n",
    "weights that minimize the loss function $\\Phi$:\n",
    "\n",
    "$$F(w) = \\frac{1}{m} \\sum_{i=1}^{m} \\Phi \\left(\\sum_{j=1}^{N}w_jh_j(x_i), y_i \\right)$$\n",
    "\n",
    "This is the first term in the AdaNet objective. The second term applies L1\n",
    "regularization to the mixture weights:\n",
    "\n",
    "$$\\sum_{j=1}^{N} \\left(\\lambda r(h_j) + \\beta \\right) |w_j|$$\n",
    "\n",
    "When $\\lambda > 0$ this penalty serves to prevent the optimization from\n",
    "assigning too much weight to more complex subnetworks according to the\n",
    "complexity measure function $r$.\n",
    "\n",
    "## How AdaNet uses the objective\n",
    "\n",
    "This objective function serves two purposes:\n",
    "\n",
    "1.  To **learn to scale/transform the outputs of each subnetwork $h$** as part\n",
    "    of the ensemble.\n",
    "2.  To **select the best candidate subnetwork $h$** at each AdaNet iteration\n",
    "    to include in the ensemble.\n",
    "\n",
    "Effectively, when learning mixture weights $w$, AdaNet solves a convex\n",
    "combination of the outputs of the frozen subnetworks $h$. For $\\lambda >0$,\n",
    "AdaNet penalizes more complex subnetworks with greater L1 regularization on\n",
    "their mixture weight, and will be less likely to select more complex subnetworks\n",
    "to add to the ensemble at each iteration.\n",
    "\n",
    "In this tutorial, in you will observe the benefits of using AdaNet to learn the\n",
    "ensemble's mixture weights and to perform candidate selection.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "test": {
      "output": "ignore",
      "timeout": 900
     }
    },
    "colab_type": "code",
    "id": "t4ptB-vwWEGH"
   },
   "outputs": [],
   "source": [
    "from __future__ import absolute_import\n",
    "from __future__ import division\n",
    "from __future__ import print_function\n",
    "\n",
    "import functools\n",
    "\n",
    "import adanet\n",
    "import tensorflow as tf\n",
    "\n",
    "# The random seed to use.\n",
    "RANDOM_SEED = 42"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ElcMSpyg_dYO"
   },
   "source": [
    "## Boston Housing dataset\n",
    "\n",
    "In this example, we will solve a regression task known as the [Boston Housing dataset](https://www.cs.toronto.edu/~delve/data/boston/bostonDetail.html) to predict the price of suburban houses in Boston, MA in the 1970s. There are 13 numerical features, the labels are in thousands of dollars, and there are only 506 examples.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Mp5TTBOJ_ZTU"
   },
   "source": [
    "## Download the data\n",
    "Conveniently, the data is available via Keras:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "test": {
      "output": "ignore",
      "timeout": 900
     }
    },
    "colab_type": "code",
    "id": "Plx4LtD4_XFY"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/boston_housing.npz\n",
      "57344/57026 [==============================] - 0s 1us/step\n"
     ]
    }
   ],
   "source": [
    "(x_train, y_train), (x_test, y_test) = (\n",
    "    tf.keras.datasets.boston_housing.load_data())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(102, 13)\n",
      "[ 18.0846   0.      18.1      0.       0.679    6.434  100.       1.8347\n",
      "  24.     666.      20.2     27.25    29.05  ]\n",
      "(102,)\n",
      "7.2\n"
     ]
    }
   ],
   "source": [
    "print(x_test.shape)\n",
    "print(x_test[0])\n",
    "print(y_test.shape)\n",
    "print(y_test[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bUSb0JtOA5Xc"
   },
   "source": [
    "## Supply the data in TensorFlow\n",
    "\n",
    "Our first task is to supply the data in TensorFlow. Using the\n",
    "tf.estimator.Estimator convention, we will define a function that returns an\n",
    "input_fn which returns feature and label Tensors.\n",
    "\n",
    "We will also use the tf.data.Dataset API to feed the data into our models.\n",
    "\n",
    "Also, as a preprocessing step, we will apply `tf.log1p` to log-scale the\n",
    "features and labels for improved numerical stability during training. To recover\n",
    "the model's predictions in the correct scale, you can apply `tf.math.expm1` to the\n",
    "prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "test": {
      "output": "ignore",
      "timeout": 900
     }
    },
    "colab_type": "code",
    "id": "H1U6DppVwTPM"
   },
   "outputs": [],
   "source": [
    "FEATURES_KEY = \"x\"\n",
    "\n",
    "\n",
    "def input_fn(partition, training, batch_size):\n",
    "  \"\"\"Generate an input function for the Estimator.\"\"\"\n",
    "\n",
    "  def _input_fn():\n",
    "\n",
    "    if partition == \"train\":\n",
    "      dataset = tf.data.Dataset.from_tensor_slices(({\n",
    "          FEATURES_KEY: tf.log1p(x_train)\n",
    "      }, tf.log1p(y_train)))\n",
    "    else:\n",
    "      dataset = tf.data.Dataset.from_tensor_slices(({\n",
    "          FEATURES_KEY: tf.log1p(x_test)\n",
    "      }, tf.log1p(y_test)))\n",
    "\n",
    "    # We call repeat after shuffling, rather than before, to prevent separate\n",
    "    # epochs from blending together.\n",
    "    if training:\n",
    "      dataset = dataset.shuffle(10 * batch_size, seed=RANDOM_SEED).repeat()\n",
    "\n",
    "    dataset = dataset.batch(batch_size)\n",
    "    iterator = dataset.make_one_shot_iterator()\n",
    "    features, labels = iterator.get_next()\n",
    "    return features, labels\n",
    "\n",
    "  return _input_fn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XXy9pqlq_PQx"
   },
   "source": [
    "## Define the subnetwork generator\n",
    "\n",
    "Let's define a subnetwork generator similar to the one in\n",
    "[[Cortes et al., ICML 2017](https://arxiv.org/abs/1607.01097)] and in\n",
    "`simple_dnn.py` which creates two candidate fully-connected neural networks at\n",
    "each iteration with the same width, but one an additional hidden layer. To make\n",
    "our generator *adaptive*, each subnetwork will have at least the same number\n",
    "of hidden layers as the most recently added subnetwork to the\n",
    "`previous_ensemble`.\n",
    "\n",
    "We define the complexity measure function $r$ to be $r(h) = \\sqrt{d(h)}$, where\n",
    "$d$ is the number of hidden layers in the neural network $h$, to approximate the\n",
    "Rademacher bounds from\n",
    "[[Golowich et. al, 2017](https://arxiv.org/abs/1712.06541)]. So subnetworks\n",
    "with more hidden layers, and therefore more capacity, will have more heavily\n",
    "regularized mixture weights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "hZn9VeFUHvD-"
   },
   "outputs": [],
   "source": [
    "_NUM_LAYERS_KEY = \"num_layers\"\n",
    "\n",
    "\n",
    "class _SimpleDNNBuilder(adanet.subnetwork.Builder):\n",
    "  \"\"\"Builds a DNN subnetwork for AdaNet.\"\"\"\n",
    "\n",
    "  def __init__(self, optimizer, layer_size, num_layers, learn_mixture_weights,\n",
    "               seed):\n",
    "    \"\"\"Initializes a `_DNNBuilder`.\n",
    "\n",
    "    Args:\n",
    "      optimizer: An `Optimizer` instance for training both the subnetwork and\n",
    "        the mixture weights.\n",
    "      layer_size: The number of nodes to output at each hidden layer.\n",
    "      num_layers: The number of hidden layers.\n",
    "      learn_mixture_weights: Whether to solve a learning problem to find the\n",
    "        best mixture weights, or use their default value according to the\n",
    "        mixture weight type. When `False`, the subnetworks will return a no_op\n",
    "        for the mixture weight train op.\n",
    "      seed: A random seed.\n",
    "\n",
    "    Returns:\n",
    "      An instance of `_SimpleDNNBuilder`.\n",
    "    \"\"\"\n",
    "\n",
    "    self._optimizer = optimizer\n",
    "    self._layer_size = layer_size\n",
    "    self._num_layers = num_layers\n",
    "    self._learn_mixture_weights = learn_mixture_weights\n",
    "    self._seed = seed\n",
    "\n",
    "  def build_subnetwork(self,\n",
    "                       features,\n",
    "                       logits_dimension,\n",
    "                       training,\n",
    "                       iteration_step,\n",
    "                       summary,\n",
    "                       previous_ensemble=None):\n",
    "    \"\"\"See `adanet.subnetwork.Builder`.\"\"\"\n",
    "\n",
    "    input_layer = tf.to_float(features[FEATURES_KEY])\n",
    "    kernel_initializer = tf.glorot_uniform_initializer(seed=self._seed)\n",
    "    last_layer = input_layer\n",
    "    for _ in range(self._num_layers):\n",
    "      last_layer = tf.layers.dense(\n",
    "          last_layer,\n",
    "          units=self._layer_size,\n",
    "          activation=tf.nn.relu,\n",
    "          kernel_initializer=kernel_initializer)\n",
    "    logits = tf.layers.dense(\n",
    "        last_layer,\n",
    "        units=logits_dimension,\n",
    "        kernel_initializer=kernel_initializer)\n",
    "\n",
    "    persisted_tensors = {_NUM_LAYERS_KEY: tf.constant(self._num_layers)}\n",
    "    return adanet.Subnetwork(\n",
    "        last_layer=last_layer,\n",
    "        logits=logits,\n",
    "        complexity=self._measure_complexity(),\n",
    "        persisted_tensors=persisted_tensors)\n",
    "\n",
    "  def _measure_complexity(self):\n",
    "    \"\"\"Approximates Rademacher complexity as the square-root of the depth.\"\"\"\n",
    "    return tf.sqrt(tf.to_float(self._num_layers))\n",
    "\n",
    "  def build_subnetwork_train_op(self, subnetwork, loss, var_list, labels,\n",
    "                                iteration_step, summary, previous_ensemble):\n",
    "    \"\"\"See `adanet.subnetwork.Builder`.\"\"\"\n",
    "    return self._optimizer.minimize(loss=loss, var_list=var_list)\n",
    "\n",
    "  def build_mixture_weights_train_op(self, loss, var_list, logits, labels,\n",
    "                                     iteration_step, summary):\n",
    "    \"\"\"See `adanet.subnetwork.Builder`.\"\"\"\n",
    "\n",
    "    if not self._learn_mixture_weights:\n",
    "      return tf.no_op()\n",
    "    return self._optimizer.minimize(loss=loss, var_list=var_list)\n",
    "\n",
    "  @property\n",
    "  def name(self):\n",
    "    \"\"\"See `adanet.subnetwork.Builder`.\"\"\"\n",
    "\n",
    "    if self._num_layers == 0:\n",
    "      # A DNN with no hidden layers is a linear model.\n",
    "      return \"linear\"\n",
    "    return \"{}_layer_dnn\".format(self._num_layers)\n",
    "\n",
    "\n",
    "class SimpleDNNGenerator(adanet.subnetwork.Generator):\n",
    "  \"\"\"Generates a two DNN subnetworks at each iteration.\n",
    "\n",
    "  The first DNN has an identical shape to the most recently added subnetwork\n",
    "  in `previous_ensemble`. The second has the same shape plus one more dense\n",
    "  layer on top. This is similar to the adaptive network presented in Figure 2 of\n",
    "  [Cortes et al. ICML 2017](https://arxiv.org/abs/1607.01097), without the\n",
    "  connections to hidden layers of networks from previous iterations.\n",
    "  \"\"\"\n",
    "\n",
    "  def __init__(self,\n",
    "               optimizer,\n",
    "               layer_size=32,\n",
    "               learn_mixture_weights=False,\n",
    "               seed=None):\n",
    "    \"\"\"Initializes a DNN `Generator`.\n",
    "\n",
    "    Args:\n",
    "      optimizer: An `Optimizer` instance for training both the subnetwork and\n",
    "        the mixture weights.\n",
    "      layer_size: Number of nodes in each hidden layer of the subnetwork\n",
    "        candidates. Note that this parameter is ignored in a DNN with no hidden\n",
    "        layers.\n",
    "      learn_mixture_weights: Whether to solve a learning problem to find the\n",
    "        best mixture weights, or use their default value according to the\n",
    "        mixture weight type. When `False`, the subnetworks will return a no_op\n",
    "        for the mixture weight train op.\n",
    "      seed: A random seed.\n",
    "\n",
    "    Returns:\n",
    "      An instance of `Generator`.\n",
    "    \"\"\"\n",
    "\n",
    "    self._seed = seed\n",
    "    self._dnn_builder_fn = functools.partial(\n",
    "        _SimpleDNNBuilder,\n",
    "        optimizer=optimizer,\n",
    "        layer_size=layer_size,\n",
    "        learn_mixture_weights=learn_mixture_weights)\n",
    "\n",
    "  def generate_candidates(self, previous_ensemble, iteration_number,\n",
    "                          previous_ensemble_reports, all_reports):\n",
    "    \"\"\"See `adanet.subnetwork.Generator`.\"\"\"\n",
    "\n",
    "    num_layers = 0\n",
    "    seed = self._seed\n",
    "    if previous_ensemble:\n",
    "      num_layers = tf.contrib.util.constant_value(\n",
    "          previous_ensemble.weighted_subnetworks[\n",
    "              -1].subnetwork.persisted_tensors[_NUM_LAYERS_KEY])\n",
    "    if seed is not None:\n",
    "      seed += iteration_number\n",
    "    return [\n",
    "        self._dnn_builder_fn(num_layers=num_layers, seed=seed),\n",
    "        self._dnn_builder_fn(num_layers=num_layers + 1, seed=seed),\n",
    "    ]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "vJRH-v5NhnY9"
   },
   "source": [
    "## Train and evaluate\n",
    "\n",
    "Next we create an `adanet.Estimator` using the `SimpleDNNGenerator` we just defined.\n",
    "\n",
    "In this section we will show the effects of two hyperparamters: **learning mixture weights** and **complexity regularization**.\n",
    "\n",
    "On the righthand side you will be able to play with the hyperparameters of this model. Until you reach the end of this section, we ask that you not change them. \n",
    "\n",
    "At first we will not learn the mixture weights, using their default initial value. Here they will be scalars initialized to $1/N$ where $N$ is the number of subnetworks in the ensemble, effectively creating a **uniform average ensemble**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "cellView": "both",
    "colab": {
     "height": 53,
     "test": {
      "output": "ignore",
      "timeout": 900
     }
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 227778,
     "status": "ok",
     "timestamp": 1539111275139,
     "user": {
      "displayName": "",
      "photoUrl": "",
      "userId": ""
     },
     "user_tz": 240
    },
    "id": "8TJHr1k-zYpN",
    "outputId": "0cd3cd50-31f4-47a3-c3cf-233a9687b721"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmplcezpthw\n",
      "INFO:tensorflow:Using config: {'_save_checkpoints_secs': None, '_experimental_distribute': None, '_service': None, '_task_id': 0, '_is_chief': True, '_master': '', '_evaluation_master': '', '_train_distribute': None, '_model_dir': '/tmp/tmplcezpthw', '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f8074e7df28>, '_keep_checkpoint_every_n_hours': 10000, '_global_id_in_cluster': 0, '_keep_checkpoint_max': 5, '_save_checkpoints_steps': 50000, '_tf_random_seed': 42, '_session_config': allow_soft_placement: true\n",
      "graph_options {\n",
      "  rewrite_options {\n",
      "    meta_optimizer_iterations: ONE\n",
      "  }\n",
      "}\n",
      ", '_protocol': None, '_device_fn': None, '_save_summary_steps': 50000, '_num_ps_replicas': 0, '_eval_distribute': None, '_num_worker_replicas': 1, '_log_step_count_steps': 100, '_task_type': 'worker'}\n",
      "INFO:tensorflow:Running training and evaluation locally (non-distributed).\n",
      "INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps 50000 or save_checkpoints_secs None.\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 0\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmplcezpthw/model.ckpt.\n",
      "INFO:tensorflow:loss = 21.773132, step = 1\n",
      "INFO:tensorflow:global_step/sec: 211.077\n",
      "INFO:tensorflow:loss = 0.647101, step = 101 (0.478 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.504\n",
      "INFO:tensorflow:loss = 0.58654255, step = 201 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.494\n",
      "INFO:tensorflow:loss = 0.07683477, step = 301 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.097\n",
      "INFO:tensorflow:loss = 0.08281773, step = 401 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.752\n",
      "INFO:tensorflow:loss = 0.08148783, step = 501 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 459.975\n",
      "INFO:tensorflow:loss = 0.056522056, step = 601 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 458.298\n",
      "INFO:tensorflow:loss = 0.025881834, step = 701 (0.215 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.078\n",
      "INFO:tensorflow:loss = 0.030095303, step = 801 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 455.713\n",
      "INFO:tensorflow:loss = 0.03755439, step = 901 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 444.218\n",
      "INFO:tensorflow:loss = 0.06690022, step = 1001 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 451.699\n",
      "INFO:tensorflow:loss = 0.03615122, step = 1101 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.472\n",
      "INFO:tensorflow:loss = 0.050185308, step = 1201 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 462.55\n",
      "INFO:tensorflow:loss = 0.099214725, step = 1301 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 436.246\n",
      "INFO:tensorflow:loss = 0.026417175, step = 1401 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.357\n",
      "INFO:tensorflow:loss = 0.02078271, step = 1501 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.845\n",
      "INFO:tensorflow:loss = 0.03165562, step = 1601 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.88\n",
      "INFO:tensorflow:loss = 0.041417733, step = 1701 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 447.158\n",
      "INFO:tensorflow:loss = 0.035114042, step = 1801 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 460.335\n",
      "INFO:tensorflow:loss = 0.044721745, step = 1901 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 442.593\n",
      "INFO:tensorflow:loss = 0.029930545, step = 2001 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 459.624\n",
      "INFO:tensorflow:loss = 0.04725883, step = 2101 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 462.613\n",
      "INFO:tensorflow:loss = 0.024880452, step = 2201 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 462.628\n",
      "INFO:tensorflow:loss = 0.024809994, step = 2301 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 442.364\n",
      "INFO:tensorflow:loss = 0.022308666, step = 2401 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 434.897\n",
      "INFO:tensorflow:loss = 0.04762791, step = 2501 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 473.857\n",
      "INFO:tensorflow:loss = 0.03194421, step = 2601 (0.210 sec)\n",
      "INFO:tensorflow:global_step/sec: 461.601\n",
      "INFO:tensorflow:loss = 0.033454657, step = 2701 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 459.118\n",
      "INFO:tensorflow:loss = 0.014480978, step = 2801 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 445.164\n",
      "INFO:tensorflow:loss = 0.031083336, step = 2901 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 447.55\n",
      "INFO:tensorflow:loss = 0.026340332, step = 3001 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 463.595\n",
      "INFO:tensorflow:loss = 0.02651683, step = 3101 (0.213 sec)\n",
      "INFO:tensorflow:global_step/sec: 468.642\n",
      "INFO:tensorflow:loss = 0.027183883, step = 3201 (0.214 sec)\n",
      "INFO:tensorflow:global_step/sec: 463.448\n",
      "INFO:tensorflow:loss = 0.035816483, step = 3301 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 462.425\n",
      "INFO:tensorflow:loss = 0.02551706, step = 3401 (0.214 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.828\n",
      "INFO:tensorflow:loss = 0.049349364, step = 3501 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 463.545\n",
      "INFO:tensorflow:loss = 0.024015253, step = 3601 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 463.739\n",
      "INFO:tensorflow:loss = 0.017241174, step = 3701 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 469.595\n",
      "INFO:tensorflow:loss = 0.020121489, step = 3801 (0.212 sec)\n",
      "INFO:tensorflow:global_step/sec: 468.923\n",
      "INFO:tensorflow:loss = 0.021484237, step = 3901 (0.214 sec)\n",
      "INFO:tensorflow:global_step/sec: 453.121\n",
      "INFO:tensorflow:loss = 0.037488014, step = 4001 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.323\n",
      "INFO:tensorflow:loss = 0.040071916, step = 4101 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.215\n",
      "INFO:tensorflow:loss = 0.021272995, step = 4201 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.032\n",
      "INFO:tensorflow:loss = 0.03338682, step = 4301 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.693\n",
      "INFO:tensorflow:loss = 0.036143243, step = 4401 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.626\n",
      "INFO:tensorflow:loss = 0.039583378, step = 4501 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.591\n",
      "INFO:tensorflow:loss = 0.036702216, step = 4601 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.303\n",
      "INFO:tensorflow:loss = 0.05008479, step = 4701 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 453.169\n",
      "INFO:tensorflow:loss = 0.0439879, step = 4801 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 462.178\n",
      "INFO:tensorflow:loss = 0.023454221, step = 4901 (0.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 468.888\n",
      "INFO:tensorflow:loss = 0.014781383, step = 5001 (0.213 sec)\n",
      "INFO:tensorflow:global_step/sec: 463.829\n",
      "INFO:tensorflow:loss = 0.020877432, step = 5101 (0.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 465.293\n",
      "INFO:tensorflow:loss = 0.028106665, step = 5201 (0.212 sec)\n",
      "INFO:tensorflow:global_step/sec: 447\n",
      "INFO:tensorflow:loss = 0.044017084, step = 5301 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 442.253\n",
      "INFO:tensorflow:loss = 0.015634855, step = 5401 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 468.506\n",
      "INFO:tensorflow:loss = 0.017649759, step = 5501 (0.214 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.122\n",
      "INFO:tensorflow:loss = 0.026881203, step = 5601 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.981\n",
      "INFO:tensorflow:loss = 0.02515915, step = 5701 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.847\n",
      "INFO:tensorflow:loss = 0.03226296, step = 5801 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.411\n",
      "INFO:tensorflow:loss = 0.014366373, step = 5901 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.939\n",
      "INFO:tensorflow:loss = 0.020684633, step = 6001 (0.303 sec)\n",
      "INFO:tensorflow:global_step/sec: 330.503\n",
      "INFO:tensorflow:loss = 0.035918076, step = 6101 (0.299 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.144\n",
      "INFO:tensorflow:loss = 0.052825905, step = 6201 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 355.8\n",
      "INFO:tensorflow:loss = 0.026814178, step = 6301 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.615\n",
      "INFO:tensorflow:loss = 0.03537807, step = 6401 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.551\n",
      "INFO:tensorflow:loss = 0.041909292, step = 6501 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 321.808\n",
      "INFO:tensorflow:loss = 0.025281452, step = 6601 (0.311 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 393.635\n",
      "INFO:tensorflow:loss = 0.020565277, step = 6701 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 451.012\n",
      "INFO:tensorflow:loss = 0.020751141, step = 6801 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 471.442\n",
      "INFO:tensorflow:loss = 0.016550886, step = 6901 (0.213 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.272\n",
      "INFO:tensorflow:loss = 0.026294494, step = 7001 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 452.051\n",
      "INFO:tensorflow:loss = 0.025629986, step = 7101 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 468.433\n",
      "INFO:tensorflow:loss = 0.017876476, step = 7201 (0.213 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.513\n",
      "INFO:tensorflow:loss = 0.048108123, step = 7301 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.219\n",
      "INFO:tensorflow:loss = 0.025422232, step = 7401 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.504\n",
      "INFO:tensorflow:loss = 0.025654687, step = 7501 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.815\n",
      "INFO:tensorflow:loss = 0.012629356, step = 7601 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.862\n",
      "INFO:tensorflow:loss = 0.02279413, step = 7701 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.309\n",
      "INFO:tensorflow:loss = 0.019274494, step = 7801 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.547\n",
      "INFO:tensorflow:loss = 0.017386634, step = 7901 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.327\n",
      "INFO:tensorflow:loss = 0.024868254, step = 8001 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.185\n",
      "INFO:tensorflow:loss = 0.021579651, step = 8101 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.818\n",
      "INFO:tensorflow:loss = 0.040016398, step = 8201 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 481.607\n",
      "INFO:tensorflow:loss = 0.023610089, step = 8301 (0.208 sec)\n",
      "INFO:tensorflow:global_step/sec: 482.181\n",
      "INFO:tensorflow:loss = 0.046594612, step = 8401 (0.209 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.487\n",
      "INFO:tensorflow:loss = 0.020441823, step = 8501 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.699\n",
      "INFO:tensorflow:loss = 0.028941382, step = 8601 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 437.626\n",
      "INFO:tensorflow:loss = 0.022684688, step = 8701 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.342\n",
      "INFO:tensorflow:loss = 0.035483457, step = 8801 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 441.908\n",
      "INFO:tensorflow:loss = 0.054693077, step = 8901 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.749\n",
      "INFO:tensorflow:loss = 0.0427555, step = 9001 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.908\n",
      "INFO:tensorflow:loss = 0.02018222, step = 9101 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.862\n",
      "INFO:tensorflow:loss = 0.03507743, step = 9201 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 455.476\n",
      "INFO:tensorflow:loss = 0.014007289, step = 9301 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.995\n",
      "INFO:tensorflow:loss = 0.017210696, step = 9401 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.353\n",
      "INFO:tensorflow:loss = 0.036349457, step = 9501 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 495.053\n",
      "INFO:tensorflow:loss = 0.032569163, step = 9601 (0.204 sec)\n",
      "INFO:tensorflow:global_step/sec: 468.562\n",
      "INFO:tensorflow:loss = 0.011827829, step = 9701 (0.211 sec)\n",
      "INFO:tensorflow:global_step/sec: 464.403\n",
      "INFO:tensorflow:loss = 0.021425512, step = 9801 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 477.111\n",
      "INFO:tensorflow:loss = 0.022103805, step = 9901 (0.208 sec)\n",
      "INFO:tensorflow:global_step/sec: 449.416\n",
      "INFO:tensorflow:loss = 0.029674655, step = 10001 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 468.488\n",
      "INFO:tensorflow:loss = 0.02097181, step = 10101 (0.214 sec)\n",
      "INFO:tensorflow:global_step/sec: 450.776\n",
      "INFO:tensorflow:loss = 0.04391878, step = 10201 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 462.887\n",
      "INFO:tensorflow:loss = 0.032415442, step = 10301 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 463.603\n",
      "INFO:tensorflow:loss = 0.03412172, step = 10401 (0.213 sec)\n",
      "INFO:tensorflow:global_step/sec: 474.088\n",
      "INFO:tensorflow:loss = 0.038466327, step = 10501 (0.212 sec)\n",
      "INFO:tensorflow:global_step/sec: 461.192\n",
      "INFO:tensorflow:loss = 0.024964295, step = 10601 (0.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 455.837\n",
      "INFO:tensorflow:loss = 0.012220745, step = 10701 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 458.078\n",
      "INFO:tensorflow:loss = 0.02140525, step = 10801 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 494.293\n",
      "INFO:tensorflow:loss = 0.026839105, step = 10901 (0.202 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.35\n",
      "INFO:tensorflow:loss = 0.010016122, step = 11001 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 462.95\n",
      "INFO:tensorflow:loss = 0.027438529, step = 11101 (0.214 sec)\n",
      "INFO:tensorflow:global_step/sec: 473.72\n",
      "INFO:tensorflow:loss = 0.012904949, step = 11201 (0.213 sec)\n",
      "INFO:tensorflow:global_step/sec: 458.671\n",
      "INFO:tensorflow:loss = 0.04179068, step = 11301 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 443.632\n",
      "INFO:tensorflow:loss = 0.017984672, step = 11401 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 449.957\n",
      "INFO:tensorflow:loss = 0.060035847, step = 11501 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.008\n",
      "INFO:tensorflow:loss = 0.014173756, step = 11601 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 465.833\n",
      "INFO:tensorflow:loss = 0.02374209, step = 11701 (0.212 sec)\n",
      "INFO:tensorflow:global_step/sec: 456.728\n",
      "INFO:tensorflow:loss = 0.01310522, step = 11801 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 452.846\n",
      "INFO:tensorflow:loss = 0.03499897, step = 11901 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 463.307\n",
      "INFO:tensorflow:loss = 0.024894122, step = 12001 (0.215 sec)\n",
      "INFO:tensorflow:global_step/sec: 461.083\n",
      "INFO:tensorflow:loss = 0.01472405, step = 12101 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 454.965\n",
      "INFO:tensorflow:loss = 0.010127504, step = 12201 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 465.611\n",
      "INFO:tensorflow:loss = 0.018342067, step = 12301 (0.215 sec)\n",
      "INFO:tensorflow:global_step/sec: 466.85\n",
      "INFO:tensorflow:loss = 0.016512873, step = 12401 (0.213 sec)\n",
      "INFO:tensorflow:global_step/sec: 448.31\n",
      "INFO:tensorflow:loss = 0.023445804, step = 12501 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.459\n",
      "INFO:tensorflow:loss = 0.021017548, step = 12601 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 463.61\n",
      "INFO:tensorflow:loss = 0.018939126, step = 12701 (0.215 sec)\n",
      "INFO:tensorflow:global_step/sec: 442.133\n",
      "INFO:tensorflow:loss = 0.020231271, step = 12801 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 470.697\n",
      "INFO:tensorflow:loss = 0.025940817, step = 12901 (0.212 sec)\n",
      "INFO:tensorflow:global_step/sec: 459.479\n",
      "INFO:tensorflow:loss = 0.023707435, step = 13001 (0.215 sec)\n",
      "INFO:tensorflow:global_step/sec: 467.333\n",
      "INFO:tensorflow:loss = 0.02946925, step = 13101 (0.214 sec)\n",
      "INFO:tensorflow:global_step/sec: 465.031\n",
      "INFO:tensorflow:loss = 0.030435286, step = 13201 (0.215 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.961\n",
      "INFO:tensorflow:loss = 0.015011725, step = 13301 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 469.23\n",
      "INFO:tensorflow:loss = 0.02355712, step = 13401 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 443.878\n",
      "INFO:tensorflow:loss = 0.024404213, step = 13501 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 441.281\n",
      "INFO:tensorflow:loss = 0.016653437, step = 13601 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.627\n",
      "INFO:tensorflow:loss = 0.014666071, step = 13701 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.465\n",
      "INFO:tensorflow:loss = 0.035263278, step = 13801 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 470.53\n",
      "INFO:tensorflow:loss = 0.021573585, step = 13901 (0.209 sec)\n",
      "INFO:tensorflow:global_step/sec: 484.114\n",
      "INFO:tensorflow:loss = 0.015728, step = 14001 (0.207 sec)\n",
      "INFO:tensorflow:global_step/sec: 478.42\n",
      "INFO:tensorflow:loss = 0.015849417, step = 14101 (0.211 sec)\n",
      "INFO:tensorflow:global_step/sec: 477.049\n",
      "INFO:tensorflow:loss = 0.016528312, step = 14201 (0.210 sec)\n",
      "INFO:tensorflow:global_step/sec: 466.428\n",
      "INFO:tensorflow:loss = 0.019497154, step = 14301 (0.213 sec)\n",
      "INFO:tensorflow:global_step/sec: 471.531\n",
      "INFO:tensorflow:loss = 0.012926113, step = 14401 (0.213 sec)\n",
      "INFO:tensorflow:global_step/sec: 465.533\n",
      "INFO:tensorflow:loss = 0.020883435, step = 14501 (0.215 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.742\n",
      "INFO:tensorflow:loss = 0.02792125, step = 14601 (0.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 460.724\n",
      "INFO:tensorflow:loss = 0.024781631, step = 14701 (0.216 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 476.546\n",
      "INFO:tensorflow:loss = 0.027412493, step = 14801 (0.210 sec)\n",
      "INFO:tensorflow:global_step/sec: 500.457\n",
      "INFO:tensorflow:loss = 0.042520046, step = 14901 (0.203 sec)\n",
      "INFO:tensorflow:global_step/sec: 496.861\n",
      "INFO:tensorflow:loss = 0.026533308, step = 15001 (0.200 sec)\n",
      "INFO:tensorflow:global_step/sec: 482.884\n",
      "INFO:tensorflow:loss = 0.01399735, step = 15101 (0.206 sec)\n",
      "INFO:tensorflow:global_step/sec: 473.638\n",
      "INFO:tensorflow:loss = 0.024882462, step = 15201 (0.210 sec)\n",
      "INFO:tensorflow:global_step/sec: 493.16\n",
      "INFO:tensorflow:loss = 0.024065059, step = 15301 (0.204 sec)\n",
      "INFO:tensorflow:global_step/sec: 471.448\n",
      "INFO:tensorflow:loss = 0.010977234, step = 15401 (0.212 sec)\n",
      "INFO:tensorflow:global_step/sec: 479.813\n",
      "INFO:tensorflow:loss = 0.020695545, step = 15501 (0.208 sec)\n",
      "INFO:tensorflow:global_step/sec: 478.489\n",
      "INFO:tensorflow:loss = 0.023899324, step = 15601 (0.209 sec)\n",
      "INFO:tensorflow:global_step/sec: 483.478\n",
      "INFO:tensorflow:loss = 0.026056474, step = 15701 (0.206 sec)\n",
      "INFO:tensorflow:global_step/sec: 475.828\n",
      "INFO:tensorflow:loss = 0.036305584, step = 15801 (0.210 sec)\n",
      "INFO:tensorflow:global_step/sec: 470.729\n",
      "INFO:tensorflow:loss = 0.030130608, step = 15901 (0.212 sec)\n",
      "INFO:tensorflow:global_step/sec: 455.606\n",
      "INFO:tensorflow:loss = 0.025261547, step = 16001 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 468.616\n",
      "INFO:tensorflow:loss = 0.05639106, step = 16101 (0.214 sec)\n",
      "INFO:tensorflow:global_step/sec: 478.546\n",
      "INFO:tensorflow:loss = 0.016278945, step = 16201 (0.209 sec)\n",
      "INFO:tensorflow:global_step/sec: 474.89\n",
      "INFO:tensorflow:loss = 0.03271088, step = 16301 (0.210 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.893\n",
      "INFO:tensorflow:loss = 0.050875228, step = 16401 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 475.225\n",
      "INFO:tensorflow:loss = 0.014902066, step = 16501 (0.211 sec)\n",
      "INFO:tensorflow:global_step/sec: 482.025\n",
      "INFO:tensorflow:loss = 0.020203961, step = 16601 (0.204 sec)\n",
      "INFO:tensorflow:global_step/sec: 462.138\n",
      "INFO:tensorflow:loss = 0.026703361, step = 16701 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 484.285\n",
      "INFO:tensorflow:loss = 0.028909443, step = 16801 (0.207 sec)\n",
      "INFO:tensorflow:global_step/sec: 471.125\n",
      "INFO:tensorflow:loss = 0.017584058, step = 16901 (0.212 sec)\n",
      "INFO:tensorflow:global_step/sec: 473.455\n",
      "INFO:tensorflow:loss = 0.026344351, step = 17001 (0.210 sec)\n",
      "INFO:tensorflow:global_step/sec: 469.589\n",
      "INFO:tensorflow:loss = 0.023616724, step = 17101 (0.211 sec)\n",
      "INFO:tensorflow:global_step/sec: 472.199\n",
      "INFO:tensorflow:loss = 0.014025886, step = 17201 (0.212 sec)\n",
      "INFO:tensorflow:global_step/sec: 452.289\n",
      "INFO:tensorflow:loss = 0.02290166, step = 17301 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.827\n",
      "INFO:tensorflow:loss = 0.014837083, step = 17401 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.78\n",
      "INFO:tensorflow:loss = 0.019864578, step = 17501 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 459.145\n",
      "INFO:tensorflow:loss = 0.02459327, step = 17601 (0.215 sec)\n",
      "INFO:tensorflow:global_step/sec: 465.938\n",
      "INFO:tensorflow:loss = 0.016008744, step = 17701 (0.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 449.789\n",
      "INFO:tensorflow:loss = 0.016970221, step = 17801 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 474.93\n",
      "INFO:tensorflow:loss = 0.015661228, step = 17901 (0.213 sec)\n",
      "INFO:tensorflow:global_step/sec: 455.402\n",
      "INFO:tensorflow:loss = 0.026945343, step = 18001 (0.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 458.911\n",
      "INFO:tensorflow:loss = 0.028911578, step = 18101 (0.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.657\n",
      "INFO:tensorflow:loss = 0.021444883, step = 18201 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 463.406\n",
      "INFO:tensorflow:loss = 0.02873559, step = 18301 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.89\n",
      "INFO:tensorflow:loss = 0.030262839, step = 18401 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 448.17\n",
      "INFO:tensorflow:loss = 0.017491808, step = 18501 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 468.03\n",
      "INFO:tensorflow:loss = 0.018047858, step = 18601 (0.214 sec)\n",
      "INFO:tensorflow:global_step/sec: 469.378\n",
      "INFO:tensorflow:loss = 0.042409047, step = 18701 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.065\n",
      "INFO:tensorflow:loss = 0.0097924825, step = 18801 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 478.085\n",
      "INFO:tensorflow:loss = 0.026300514, step = 18901 (0.208 sec)\n",
      "INFO:tensorflow:global_step/sec: 468.904\n",
      "INFO:tensorflow:loss = 0.014050778, step = 19001 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 476.538\n",
      "INFO:tensorflow:loss = 0.0294879, step = 19101 (0.210 sec)\n",
      "INFO:tensorflow:global_step/sec: 458.055\n",
      "INFO:tensorflow:loss = 0.024090935, step = 19201 (0.215 sec)\n",
      "INFO:tensorflow:global_step/sec: 466.149\n",
      "INFO:tensorflow:loss = 0.019012475, step = 19301 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 467.019\n",
      "INFO:tensorflow:loss = 0.014907913, step = 19401 (0.213 sec)\n",
      "INFO:tensorflow:global_step/sec: 459.229\n",
      "INFO:tensorflow:loss = 0.014032199, step = 19501 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 459.98\n",
      "INFO:tensorflow:loss = 0.011739438, step = 19601 (0.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 460.186\n",
      "INFO:tensorflow:loss = 0.011423313, step = 19701 (0.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 464.282\n",
      "INFO:tensorflow:loss = 0.017741919, step = 19801 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.714\n",
      "INFO:tensorflow:loss = 0.04727843, step = 19901 (0.218 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 20000 into /tmp/tmplcezpthw/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-07:25:05\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-20000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'linear' dict for global step 20000: architecture/adanet/ensembles = b\"\\nW\\n6adanet/iteration_0/ensemble_linear/architecture/adanetB\\x13\\x08\\x07\\x12\\x00B\\r| b'linear' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.049421385, average_loss/adanet/subnetwork = 0.049421385, average_loss/adanet/uniform_average_ensemble = 0.049421385, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.062442064, loss/adanet/subnetwork = 0.062442064, loss/adanet/uniform_average_ensemble = 0.062442064, prediction/mean/adanet/adanet_weighted_ensemble = 3.105895, prediction/mean/adanet/subnetwork = 3.105895, prediction/mean/adanet/uniform_average_ensemble = 3.105895\n",
      "INFO:tensorflow:Saving candidate '1_layer_dnn' dict for global step 20000: architecture/adanet/ensembles = b\"\\na\\n;adanet/iteration_0/ensemble_1_layer_dnn/architecture/adanetB\\x18\\x08\\x07\\x12\\x00B\\x12| b'1_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.039825242, average_loss/adanet/subnetwork = 0.039825242, average_loss/adanet/uniform_average_ensemble = 0.039825242, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.053258948, loss/adanet/subnetwork = 0.053258948, loss/adanet/uniform_average_ensemble = 0.053258948, prediction/mean/adanet/adanet_weighted_ensemble = 3.157783, prediction/mean/adanet/subnetwork = 3.157783, prediction/mean/adanet/uniform_average_ensemble = 3.157783\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-19-07:25:07\n",
      "INFO:tensorflow:Saving dict for global step 20000: average_loss = 0.039825242, average_loss/adanet/adanet_weighted_ensemble = 0.039825242, average_loss/adanet/subnetwork = 0.039825242, average_loss/adanet/uniform_average_ensemble = 0.039825242, global_step = 20000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.053258948, loss/adanet/adanet_weighted_ensemble = 0.053258948, loss/adanet/subnetwork = 0.053258948, loss/adanet/uniform_average_ensemble = 0.053258948, prediction/mean = 3.157783, prediction/mean/adanet/adanet_weighted_ensemble = 3.157783, prediction/mean/adanet/subnetwork = 3.157783, prediction/mean/adanet/uniform_average_ensemble = 3.157783\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20000: /tmp/tmplcezpthw/model.ckpt-20000\n",
      "INFO:tensorflow:Loss for final step: 0.033959076.\n",
      "INFO:tensorflow:Starting ensemble evaluation for iteration 0\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-20000\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/adanet/core/estimator.py:707: start_queue_runners (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "To construct input pipelines, use the `tf.data` module.\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Encountered end of input after 14 evaluations\n",
      "INFO:tensorflow:Computed ensemble metrics: adanet_loss/linear = 0.035089, adanet_loss/1_layer_dnn = 0.020768\n",
      "INFO:tensorflow:Finished ensemble evaluation for iteration 0\n",
      "INFO:tensorflow:'1_layer_dnn' at index 1 is moving onto the next iteration\n",
      "INFO:tensorflow:Freezing best ensemble to /tmp/tmplcezpthw/frozen/ensemble-0.meta\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-20000\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Overwriting checkpoint with new graph for iteration 1 to /tmp/tmplcezpthw/model.ckpt-20000\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-20000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Finished training Adanet iteration 0\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 1\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/increment.ckpt-1\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 20000 into /tmp/tmplcezpthw/model.ckpt.\n",
      "INFO:tensorflow:loss = 0.02693103, step = 20001\n",
      "INFO:tensorflow:global_step/sec: 179.565\n",
      "INFO:tensorflow:loss = 0.026572384, step = 20101 (0.558 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.029\n",
      "INFO:tensorflow:loss = 0.020609124, step = 20201 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.575\n",
      "INFO:tensorflow:loss = 0.018674485, step = 20301 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.592\n",
      "INFO:tensorflow:loss = 0.027416985, step = 20401 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.404\n",
      "INFO:tensorflow:loss = 0.016454628, step = 20501 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 437.098\n",
      "INFO:tensorflow:loss = 0.018486543, step = 20601 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.443\n",
      "INFO:tensorflow:loss = 0.011359109, step = 20701 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.299\n",
      "INFO:tensorflow:loss = 0.018955758, step = 20801 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.842\n",
      "INFO:tensorflow:loss = 0.015375405, step = 20901 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.486\n",
      "INFO:tensorflow:loss = 0.027283823, step = 21001 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.128\n",
      "INFO:tensorflow:loss = 0.023671398, step = 21101 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 446.444\n",
      "INFO:tensorflow:loss = 0.024100425, step = 21201 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 444.742\n",
      "INFO:tensorflow:loss = 0.028526282, step = 21301 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 460.88\n",
      "INFO:tensorflow:loss = 0.01697007, step = 21401 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 452.74\n",
      "INFO:tensorflow:loss = 0.010007119, step = 21501 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.072\n",
      "INFO:tensorflow:loss = 0.016501425, step = 21601 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 443.16\n",
      "INFO:tensorflow:loss = 0.022809077, step = 21701 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.64\n",
      "INFO:tensorflow:loss = 0.017237894, step = 21801 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 437.435\n",
      "INFO:tensorflow:loss = 0.023686618, step = 21901 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.3\n",
      "INFO:tensorflow:loss = 0.011594927, step = 22001 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 446.33\n",
      "INFO:tensorflow:loss = 0.032918155, step = 22101 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.546\n",
      "INFO:tensorflow:loss = 0.01916439, step = 22201 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 434.99\n",
      "INFO:tensorflow:loss = 0.016445741, step = 22301 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 445.237\n",
      "INFO:tensorflow:loss = 0.015188885, step = 22401 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 441.891\n",
      "INFO:tensorflow:loss = 0.029174086, step = 22501 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 440.696\n",
      "INFO:tensorflow:loss = 0.017533563, step = 22601 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.264\n",
      "INFO:tensorflow:loss = 0.022508316, step = 22701 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.447\n",
      "INFO:tensorflow:loss = 0.016300883, step = 22801 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.923\n",
      "INFO:tensorflow:loss = 0.017507907, step = 22901 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 446.576\n",
      "INFO:tensorflow:loss = 0.018998567, step = 23001 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.827\n",
      "INFO:tensorflow:loss = 0.015129518, step = 23101 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.876\n",
      "INFO:tensorflow:loss = 0.017450623, step = 23201 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.297\n",
      "INFO:tensorflow:loss = 0.029813815, step = 23301 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.211\n",
      "INFO:tensorflow:loss = 0.021030609, step = 23401 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.87\n",
      "INFO:tensorflow:loss = 0.042801227, step = 23501 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.307\n",
      "INFO:tensorflow:loss = 0.014079982, step = 23601 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.426\n",
      "INFO:tensorflow:loss = 0.014900922, step = 23701 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.969\n",
      "INFO:tensorflow:loss = 0.01572105, step = 23801 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 444.143\n",
      "INFO:tensorflow:loss = 0.011828184, step = 23901 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 440.845\n",
      "INFO:tensorflow:loss = 0.02369193, step = 24001 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.505\n",
      "INFO:tensorflow:loss = 0.018661404, step = 24101 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 442.672\n",
      "INFO:tensorflow:loss = 0.011832748, step = 24201 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 437.206\n",
      "INFO:tensorflow:loss = 0.01432484, step = 24301 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 440.314\n",
      "INFO:tensorflow:loss = 0.013588084, step = 24401 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 400.051\n",
      "INFO:tensorflow:loss = 0.017962664, step = 24501 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.757\n",
      "INFO:tensorflow:loss = 0.017602166, step = 24601 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 400.431\n",
      "INFO:tensorflow:loss = 0.027130947, step = 24701 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.925\n",
      "INFO:tensorflow:loss = 0.014364008, step = 24801 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.784\n",
      "INFO:tensorflow:loss = 0.0194394, step = 24901 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.857\n",
      "INFO:tensorflow:loss = 0.011405462, step = 25001 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.949\n",
      "INFO:tensorflow:loss = 0.01493275, step = 25101 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 439.279\n",
      "INFO:tensorflow:loss = 0.0191109, step = 25201 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 444.008\n",
      "INFO:tensorflow:loss = 0.028142165, step = 25301 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 441.481\n",
      "INFO:tensorflow:loss = 0.0126148965, step = 25401 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.334\n",
      "INFO:tensorflow:loss = 0.0114948815, step = 25501 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.586\n",
      "INFO:tensorflow:loss = 0.02813935, step = 25601 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.458\n",
      "INFO:tensorflow:loss = 0.015769035, step = 25701 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.913\n",
      "INFO:tensorflow:loss = 0.015441446, step = 25801 (0.243 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 360.759\n",
      "INFO:tensorflow:loss = 0.015690364, step = 25901 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.939\n",
      "INFO:tensorflow:loss = 0.022406287, step = 26001 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.156\n",
      "INFO:tensorflow:loss = 0.025258576, step = 26101 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 440.884\n",
      "INFO:tensorflow:loss = 0.02043756, step = 26201 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 453.824\n",
      "INFO:tensorflow:loss = 0.014347989, step = 26301 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.719\n",
      "INFO:tensorflow:loss = 0.026024658, step = 26401 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.758\n",
      "INFO:tensorflow:loss = 0.0362145, step = 26501 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 437.915\n",
      "INFO:tensorflow:loss = 0.019093065, step = 26601 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 447.336\n",
      "INFO:tensorflow:loss = 0.013939275, step = 26701 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.339\n",
      "INFO:tensorflow:loss = 0.01612249, step = 26801 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 450.615\n",
      "INFO:tensorflow:loss = 0.0098338295, step = 26901 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 452.961\n",
      "INFO:tensorflow:loss = 0.0160049, step = 27001 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 436.96\n",
      "INFO:tensorflow:loss = 0.01555538, step = 27101 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 448.425\n",
      "INFO:tensorflow:loss = 0.012435328, step = 27201 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.922\n",
      "INFO:tensorflow:loss = 0.030735817, step = 27301 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.729\n",
      "INFO:tensorflow:loss = 0.016168587, step = 27401 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.232\n",
      "INFO:tensorflow:loss = 0.012757748, step = 27501 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.737\n",
      "INFO:tensorflow:loss = 0.011389617, step = 27601 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 449.656\n",
      "INFO:tensorflow:loss = 0.020302068, step = 27701 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 448.186\n",
      "INFO:tensorflow:loss = 0.018866796, step = 27801 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.439\n",
      "INFO:tensorflow:loss = 0.012324452, step = 27901 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.605\n",
      "INFO:tensorflow:loss = 0.017894987, step = 28001 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.325\n",
      "INFO:tensorflow:loss = 0.020177953, step = 28101 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.237\n",
      "INFO:tensorflow:loss = 0.029398937, step = 28201 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.326\n",
      "INFO:tensorflow:loss = 0.019755973, step = 28301 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.384\n",
      "INFO:tensorflow:loss = 0.017080834, step = 28401 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.675\n",
      "INFO:tensorflow:loss = 0.010702234, step = 28501 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.491\n",
      "INFO:tensorflow:loss = 0.021251952, step = 28601 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.471\n",
      "INFO:tensorflow:loss = 0.013997257, step = 28701 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 447.018\n",
      "INFO:tensorflow:loss = 0.024245232, step = 28801 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.442\n",
      "INFO:tensorflow:loss = 0.045956668, step = 28901 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 448.095\n",
      "INFO:tensorflow:loss = 0.026044952, step = 29001 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.899\n",
      "INFO:tensorflow:loss = 0.018272396, step = 29101 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.181\n",
      "INFO:tensorflow:loss = 0.020429991, step = 29201 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 434.219\n",
      "INFO:tensorflow:loss = 0.008494976, step = 29301 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.448\n",
      "INFO:tensorflow:loss = 0.011289919, step = 29401 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 446.563\n",
      "INFO:tensorflow:loss = 0.02565977, step = 29501 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 434.283\n",
      "INFO:tensorflow:loss = 0.021196162, step = 29601 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.515\n",
      "INFO:tensorflow:loss = 0.014568824, step = 29701 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 448.299\n",
      "INFO:tensorflow:loss = 0.013778458, step = 29801 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.233\n",
      "INFO:tensorflow:loss = 0.021226026, step = 29901 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 445.526\n",
      "INFO:tensorflow:loss = 0.025036654, step = 30001 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 444.286\n",
      "INFO:tensorflow:loss = 0.009509817, step = 30101 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.958\n",
      "INFO:tensorflow:loss = 0.025581904, step = 30201 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.502\n",
      "INFO:tensorflow:loss = 0.027983958, step = 30301 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.096\n",
      "INFO:tensorflow:loss = 0.014193746, step = 30401 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.503\n",
      "INFO:tensorflow:loss = 0.011845943, step = 30501 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.038\n",
      "INFO:tensorflow:loss = 0.0114249755, step = 30601 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 400.602\n",
      "INFO:tensorflow:loss = 0.010759584, step = 30701 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.567\n",
      "INFO:tensorflow:loss = 0.01967357, step = 30801 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.349\n",
      "INFO:tensorflow:loss = 0.019226965, step = 30901 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.256\n",
      "INFO:tensorflow:loss = 0.007754709, step = 31001 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.345\n",
      "INFO:tensorflow:loss = 0.01607125, step = 31101 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.435\n",
      "INFO:tensorflow:loss = 0.011475198, step = 31201 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.081\n",
      "INFO:tensorflow:loss = 0.020241389, step = 31301 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.313\n",
      "INFO:tensorflow:loss = 0.008072655, step = 31401 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.053\n",
      "INFO:tensorflow:loss = 0.035552397, step = 31501 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.782\n",
      "INFO:tensorflow:loss = 0.0103283115, step = 31601 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.429\n",
      "INFO:tensorflow:loss = 0.015665729, step = 31701 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.365\n",
      "INFO:tensorflow:loss = 0.0108935265, step = 31801 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.709\n",
      "INFO:tensorflow:loss = 0.019247973, step = 31901 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 442.242\n",
      "INFO:tensorflow:loss = 0.016100835, step = 32001 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.656\n",
      "INFO:tensorflow:loss = 0.010600159, step = 32101 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.026\n",
      "INFO:tensorflow:loss = 0.010114832, step = 32201 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 443.672\n",
      "INFO:tensorflow:loss = 0.015323485, step = 32301 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.732\n",
      "INFO:tensorflow:loss = 0.010089278, step = 32401 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.351\n",
      "INFO:tensorflow:loss = 0.016632436, step = 32501 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.28\n",
      "INFO:tensorflow:loss = 0.015050786, step = 32601 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.586\n",
      "INFO:tensorflow:loss = 0.01639548, step = 32701 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 449.523\n",
      "INFO:tensorflow:loss = 0.018000161, step = 32801 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.265\n",
      "INFO:tensorflow:loss = 0.010589867, step = 32901 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.994\n",
      "INFO:tensorflow:loss = 0.019329488, step = 33001 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.886\n",
      "INFO:tensorflow:loss = 0.01762234, step = 33101 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.626\n",
      "INFO:tensorflow:loss = 0.009814825, step = 33201 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.308\n",
      "INFO:tensorflow:loss = 0.007630166, step = 33301 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.99\n",
      "INFO:tensorflow:loss = 0.014831969, step = 33401 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 436.553\n",
      "INFO:tensorflow:loss = 0.018828869, step = 33501 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.878\n",
      "INFO:tensorflow:loss = 0.012678431, step = 33601 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.873\n",
      "INFO:tensorflow:loss = 0.009158572, step = 33701 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 439.072\n",
      "INFO:tensorflow:loss = 0.021373678, step = 33801 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.458\n",
      "INFO:tensorflow:loss = 0.013477125, step = 33901 (0.236 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 406.421\n",
      "INFO:tensorflow:loss = 0.014301172, step = 34001 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.751\n",
      "INFO:tensorflow:loss = 0.016645, step = 34101 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.683\n",
      "INFO:tensorflow:loss = 0.016692562, step = 34201 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.976\n",
      "INFO:tensorflow:loss = 0.012835465, step = 34301 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.637\n",
      "INFO:tensorflow:loss = 0.014207549, step = 34401 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.82\n",
      "INFO:tensorflow:loss = 0.011994822, step = 34501 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.151\n",
      "INFO:tensorflow:loss = 0.020384066, step = 34601 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.082\n",
      "INFO:tensorflow:loss = 0.021298494, step = 34701 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.358\n",
      "INFO:tensorflow:loss = 0.022692181, step = 34801 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 448.772\n",
      "INFO:tensorflow:loss = 0.019764349, step = 34901 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.347\n",
      "INFO:tensorflow:loss = 0.029701483, step = 35001 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 449.646\n",
      "INFO:tensorflow:loss = 0.011285698, step = 35101 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 461.349\n",
      "INFO:tensorflow:loss = 0.013540333, step = 35201 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 463.145\n",
      "INFO:tensorflow:loss = 0.021103036, step = 35301 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.939\n",
      "INFO:tensorflow:loss = 0.006668018, step = 35401 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 453.247\n",
      "INFO:tensorflow:loss = 0.018318964, step = 35501 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 441.539\n",
      "INFO:tensorflow:loss = 0.013936672, step = 35601 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 445.324\n",
      "INFO:tensorflow:loss = 0.015862815, step = 35701 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 445.624\n",
      "INFO:tensorflow:loss = 0.027665522, step = 35801 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 400.662\n",
      "INFO:tensorflow:loss = 0.017128337, step = 35901 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.928\n",
      "INFO:tensorflow:loss = 0.015123015, step = 36001 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 440.873\n",
      "INFO:tensorflow:loss = 0.016026527, step = 36101 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.724\n",
      "INFO:tensorflow:loss = 0.01359102, step = 36201 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.602\n",
      "INFO:tensorflow:loss = 0.013185386, step = 36301 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.635\n",
      "INFO:tensorflow:loss = 0.017378077, step = 36401 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 448.907\n",
      "INFO:tensorflow:loss = 0.013636961, step = 36501 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 452.107\n",
      "INFO:tensorflow:loss = 0.014122384, step = 36601 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 456.916\n",
      "INFO:tensorflow:loss = 0.01841722, step = 36701 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 444.699\n",
      "INFO:tensorflow:loss = 0.015603266, step = 36801 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 442.279\n",
      "INFO:tensorflow:loss = 0.014569301, step = 36901 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 440.952\n",
      "INFO:tensorflow:loss = 0.02057124, step = 37001 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 447.435\n",
      "INFO:tensorflow:loss = 0.017893624, step = 37101 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 442.631\n",
      "INFO:tensorflow:loss = 0.010438396, step = 37201 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.77\n",
      "INFO:tensorflow:loss = 0.013162806, step = 37301 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.783\n",
      "INFO:tensorflow:loss = 0.011234447, step = 37401 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 436.446\n",
      "INFO:tensorflow:loss = 0.017049339, step = 37501 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 446.402\n",
      "INFO:tensorflow:loss = 0.017678963, step = 37601 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.982\n",
      "INFO:tensorflow:loss = 0.012237284, step = 37701 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.352\n",
      "INFO:tensorflow:loss = 0.016508484, step = 37801 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 440.325\n",
      "INFO:tensorflow:loss = 0.011567511, step = 37901 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.366\n",
      "INFO:tensorflow:loss = 0.017311856, step = 38001 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.506\n",
      "INFO:tensorflow:loss = 0.019526534, step = 38101 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 437.901\n",
      "INFO:tensorflow:loss = 0.01627019, step = 38201 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 443.166\n",
      "INFO:tensorflow:loss = 0.023562815, step = 38301 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.718\n",
      "INFO:tensorflow:loss = 0.020888362, step = 38401 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 447.828\n",
      "INFO:tensorflow:loss = 0.011109218, step = 38501 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.921\n",
      "INFO:tensorflow:loss = 0.015867518, step = 38601 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.267\n",
      "INFO:tensorflow:loss = 0.013251802, step = 38701 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 441.387\n",
      "INFO:tensorflow:loss = 0.009647096, step = 38801 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 442.588\n",
      "INFO:tensorflow:loss = 0.020391658, step = 38901 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.356\n",
      "INFO:tensorflow:loss = 0.0139506925, step = 39001 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.346\n",
      "INFO:tensorflow:loss = 0.016299425, step = 39101 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 439.631\n",
      "INFO:tensorflow:loss = 0.015790988, step = 39201 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 443.861\n",
      "INFO:tensorflow:loss = 0.01670075, step = 39301 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 444.21\n",
      "INFO:tensorflow:loss = 0.011833405, step = 39401 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.551\n",
      "INFO:tensorflow:loss = 0.01398856, step = 39501 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.095\n",
      "INFO:tensorflow:loss = 0.009578118, step = 39601 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.676\n",
      "INFO:tensorflow:loss = 0.008461144, step = 39701 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 440.874\n",
      "INFO:tensorflow:loss = 0.012226744, step = 39801 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.164\n",
      "INFO:tensorflow:loss = 0.02608168, step = 39901 (0.234 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 40000 into /tmp/tmplcezpthw/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-07:26:05\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-40000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'previous_ensemble' dict for global step 40000: architecture/adanet/ensembles = b\"\\nR\\n,adanet/previous_ensemble/architecture/adanetB\\x18\\x08\\x07\\x12\\x00B\\x12| b'1_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.039825242, average_loss/adanet/subnetwork = 0.039825242, average_loss/adanet/uniform_average_ensemble = 0.039825242, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.053258948, loss/adanet/subnetwork = 0.053258948, loss/adanet/uniform_average_ensemble = 0.053258948, prediction/mean/adanet/adanet_weighted_ensemble = 3.157783, prediction/mean/adanet/subnetwork = 3.157783, prediction/mean/adanet/uniform_average_ensemble = 3.157783\n",
      "INFO:tensorflow:Saving candidate '1_layer_dnn' dict for global step 40000: architecture/adanet/ensembles = b\"\\nr\\n;adanet/iteration_1/ensemble_1_layer_dnn/architecture/adanetB)\\x08\\x07\\x12\\x00B#| b'1_layer_dnn' | b'1_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.041038644, average_loss/adanet/subnetwork = 0.04407932, average_loss/adanet/uniform_average_ensemble = 0.041038644, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.05667209, loss/adanet/subnetwork = 0.061847888, loss/adanet/uniform_average_ensemble = 0.05667209, prediction/mean/adanet/adanet_weighted_ensemble = 3.1574097, prediction/mean/adanet/subnetwork = 3.1570365, prediction/mean/adanet/uniform_average_ensemble = 3.1574097\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving candidate '2_layer_dnn' dict for global step 40000: architecture/adanet/ensembles = b\"\\nr\\n;adanet/iteration_1/ensemble_2_layer_dnn/architecture/adanetB)\\x08\\x07\\x12\\x00B#| b'1_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.03574329, average_loss/adanet/subnetwork = 0.03454547, average_loss/adanet/uniform_average_ensemble = 0.03574329, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.04744883, loss/adanet/subnetwork = 0.04466034, loss/adanet/uniform_average_ensemble = 0.04744883, prediction/mean/adanet/adanet_weighted_ensemble = 3.148061, prediction/mean/adanet/subnetwork = 3.1383395, prediction/mean/adanet/uniform_average_ensemble = 3.148061\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-19-07:26:07\n",
      "INFO:tensorflow:Saving dict for global step 40000: average_loss = 0.03574329, average_loss/adanet/adanet_weighted_ensemble = 0.03574329, average_loss/adanet/subnetwork = 0.03454547, average_loss/adanet/uniform_average_ensemble = 0.03574329, global_step = 40000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.04744883, loss/adanet/adanet_weighted_ensemble = 0.04744883, loss/adanet/subnetwork = 0.04466034, loss/adanet/uniform_average_ensemble = 0.04744883, prediction/mean = 3.148061, prediction/mean/adanet/adanet_weighted_ensemble = 3.148061, prediction/mean/adanet/subnetwork = 3.1383395, prediction/mean/adanet/uniform_average_ensemble = 3.148061\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 40000: /tmp/tmplcezpthw/model.ckpt-40000\n",
      "INFO:tensorflow:Loss for final step: 0.009633257.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Starting ensemble evaluation for iteration 1\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-40000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Encountered end of input after 14 evaluations\n",
      "INFO:tensorflow:Computed ensemble metrics: adanet_loss/previous_ensemble = 0.020768, adanet_loss/1_layer_dnn = 0.020797, adanet_loss/2_layer_dnn = 0.016144\n",
      "INFO:tensorflow:Finished ensemble evaluation for iteration 1\n",
      "INFO:tensorflow:'2_layer_dnn' at index 2 is moving onto the next iteration\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Freezing best ensemble to /tmp/tmplcezpthw/frozen/ensemble-1.meta\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-40000\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Overwriting checkpoint with new graph for iteration 2 to /tmp/tmplcezpthw/model.ckpt-40000\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-40000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Finished training Adanet iteration 1\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 2\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/increment.ckpt-2\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 40000 into /tmp/tmplcezpthw/model.ckpt.\n",
      "INFO:tensorflow:loss = 0.017966717, step = 40001\n",
      "INFO:tensorflow:global_step/sec: 152.553\n",
      "INFO:tensorflow:loss = 0.020754296, step = 40101 (0.657 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.366\n",
      "INFO:tensorflow:loss = 0.015971366, step = 40201 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.851\n",
      "INFO:tensorflow:loss = 0.015143998, step = 40301 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.588\n",
      "INFO:tensorflow:loss = 0.020427275, step = 40401 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.313\n",
      "INFO:tensorflow:loss = 0.012627184, step = 40501 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.791\n",
      "INFO:tensorflow:loss = 0.013906378, step = 40601 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.672\n",
      "INFO:tensorflow:loss = 0.009350193, step = 40701 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.588\n",
      "INFO:tensorflow:loss = 0.015340277, step = 40801 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.758\n",
      "INFO:tensorflow:loss = 0.016695559, step = 40901 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.725\n",
      "INFO:tensorflow:loss = 0.023774747, step = 41001 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.062\n",
      "INFO:tensorflow:loss = 0.018350415, step = 41101 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 428.576\n",
      "INFO:tensorflow:loss = 0.023366842, step = 41201 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.069\n",
      "INFO:tensorflow:loss = 0.015467918, step = 41301 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.719\n",
      "INFO:tensorflow:loss = 0.010024572, step = 41401 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.804\n",
      "INFO:tensorflow:loss = 0.008039603, step = 41501 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.639\n",
      "INFO:tensorflow:loss = 0.012505763, step = 41601 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.267\n",
      "INFO:tensorflow:loss = 0.018137662, step = 41701 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.967\n",
      "INFO:tensorflow:loss = 0.014656871, step = 41801 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.388\n",
      "INFO:tensorflow:loss = 0.017236438, step = 41901 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.476\n",
      "INFO:tensorflow:loss = 0.010387262, step = 42001 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.38\n",
      "INFO:tensorflow:loss = 0.02417074, step = 42101 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.273\n",
      "INFO:tensorflow:loss = 0.017437343, step = 42201 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.301\n",
      "INFO:tensorflow:loss = 0.015888128, step = 42301 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.494\n",
      "INFO:tensorflow:loss = 0.012565606, step = 42401 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.884\n",
      "INFO:tensorflow:loss = 0.019407123, step = 42501 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.957\n",
      "INFO:tensorflow:loss = 0.012058962, step = 42601 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.704\n",
      "INFO:tensorflow:loss = 0.018187158, step = 42701 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.408\n",
      "INFO:tensorflow:loss = 0.013279471, step = 42801 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.312\n",
      "INFO:tensorflow:loss = 0.012677515, step = 42901 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.652\n",
      "INFO:tensorflow:loss = 0.013873251, step = 43001 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.601\n",
      "INFO:tensorflow:loss = 0.012054434, step = 43101 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.509\n",
      "INFO:tensorflow:loss = 0.020650763, step = 43201 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 362.97\n",
      "INFO:tensorflow:loss = 0.023672678, step = 43301 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.489\n",
      "INFO:tensorflow:loss = 0.01507185, step = 43401 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.942\n",
      "INFO:tensorflow:loss = 0.026599478, step = 43501 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.918\n",
      "INFO:tensorflow:loss = 0.011140069, step = 43601 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.033\n",
      "INFO:tensorflow:loss = 0.0126081165, step = 43701 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.49\n",
      "INFO:tensorflow:loss = 0.012672622, step = 43801 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.012\n",
      "INFO:tensorflow:loss = 0.0102767125, step = 43901 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.776\n",
      "INFO:tensorflow:loss = 0.018365905, step = 44001 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.145\n",
      "INFO:tensorflow:loss = 0.015633155, step = 44101 (0.244 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 379.79\n",
      "INFO:tensorflow:loss = 0.010589315, step = 44201 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.266\n",
      "INFO:tensorflow:loss = 0.008432313, step = 44301 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 344.976\n",
      "INFO:tensorflow:loss = 0.009699082, step = 44401 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.395\n",
      "INFO:tensorflow:loss = 0.010581549, step = 44501 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.583\n",
      "INFO:tensorflow:loss = 0.01785604, step = 44601 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.673\n",
      "INFO:tensorflow:loss = 0.013925884, step = 44701 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.821\n",
      "INFO:tensorflow:loss = 0.009575358, step = 44801 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.694\n",
      "INFO:tensorflow:loss = 0.019161768, step = 44901 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 428.92\n",
      "INFO:tensorflow:loss = 0.0071926797, step = 45001 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.532\n",
      "INFO:tensorflow:loss = 0.013663843, step = 45101 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.144\n",
      "INFO:tensorflow:loss = 0.017501198, step = 45201 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.878\n",
      "INFO:tensorflow:loss = 0.024153441, step = 45301 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.823\n",
      "INFO:tensorflow:loss = 0.011360073, step = 45401 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.389\n",
      "INFO:tensorflow:loss = 0.010682343, step = 45501 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.145\n",
      "INFO:tensorflow:loss = 0.020878276, step = 45601 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.245\n",
      "INFO:tensorflow:loss = 0.01186494, step = 45701 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.131\n",
      "INFO:tensorflow:loss = 0.01365256, step = 45801 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.478\n",
      "INFO:tensorflow:loss = 0.011504488, step = 45901 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.777\n",
      "INFO:tensorflow:loss = 0.0116174705, step = 46001 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.589\n",
      "INFO:tensorflow:loss = 0.022421416, step = 46101 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.298\n",
      "INFO:tensorflow:loss = 0.013327652, step = 46201 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.446\n",
      "INFO:tensorflow:loss = 0.009530427, step = 46301 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.273\n",
      "INFO:tensorflow:loss = 0.014356431, step = 46401 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.268\n",
      "INFO:tensorflow:loss = 0.02056465, step = 46501 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.477\n",
      "INFO:tensorflow:loss = 0.014381343, step = 46601 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.708\n",
      "INFO:tensorflow:loss = 0.015509526, step = 46701 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.271\n",
      "INFO:tensorflow:loss = 0.01061032, step = 46801 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.338\n",
      "INFO:tensorflow:loss = 0.013967289, step = 46901 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.61\n",
      "INFO:tensorflow:loss = 0.013896075, step = 47001 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.024\n",
      "INFO:tensorflow:loss = 0.013479393, step = 47101 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.438\n",
      "INFO:tensorflow:loss = 0.011129299, step = 47201 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.124\n",
      "INFO:tensorflow:loss = 0.018232778, step = 47301 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.271\n",
      "INFO:tensorflow:loss = 0.010534618, step = 47401 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.535\n",
      "INFO:tensorflow:loss = 0.008086846, step = 47501 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.186\n",
      "INFO:tensorflow:loss = 0.012784084, step = 47601 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.555\n",
      "INFO:tensorflow:loss = 0.022876812, step = 47701 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 434.648\n",
      "INFO:tensorflow:loss = 0.020214058, step = 47801 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.355\n",
      "INFO:tensorflow:loss = 0.011090637, step = 47901 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.204\n",
      "INFO:tensorflow:loss = 0.017569564, step = 48001 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.786\n",
      "INFO:tensorflow:loss = 0.01638979, step = 48101 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.78\n",
      "INFO:tensorflow:loss = 0.023256404, step = 48201 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.685\n",
      "INFO:tensorflow:loss = 0.01592594, step = 48301 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.882\n",
      "INFO:tensorflow:loss = 0.009586243, step = 48401 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.618\n",
      "INFO:tensorflow:loss = 0.006889733, step = 48501 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.436\n",
      "INFO:tensorflow:loss = 0.013114476, step = 48601 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.757\n",
      "INFO:tensorflow:loss = 0.012986295, step = 48701 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.347\n",
      "INFO:tensorflow:loss = 0.021419987, step = 48801 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.194\n",
      "INFO:tensorflow:loss = 0.039193366, step = 48901 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.685\n",
      "INFO:tensorflow:loss = 0.020841535, step = 49001 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.947\n",
      "INFO:tensorflow:loss = 0.014801757, step = 49101 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.196\n",
      "INFO:tensorflow:loss = 0.015221857, step = 49201 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.133\n",
      "INFO:tensorflow:loss = 0.0074894335, step = 49301 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.646\n",
      "INFO:tensorflow:loss = 0.009126705, step = 49401 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.879\n",
      "INFO:tensorflow:loss = 0.015288128, step = 49501 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.677\n",
      "INFO:tensorflow:loss = 0.013389628, step = 49601 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.356\n",
      "INFO:tensorflow:loss = 0.010871809, step = 49701 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.301\n",
      "INFO:tensorflow:loss = 0.010826562, step = 49801 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.99\n",
      "INFO:tensorflow:loss = 0.018271951, step = 49901 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.407\n",
      "INFO:tensorflow:loss = 0.018832907, step = 50001 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.409\n",
      "INFO:tensorflow:loss = 0.0075002844, step = 50101 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.265\n",
      "INFO:tensorflow:loss = 0.015696935, step = 50201 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.386\n",
      "INFO:tensorflow:loss = 0.022299072, step = 50301 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.369\n",
      "INFO:tensorflow:loss = 0.010444637, step = 50401 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 466.378\n",
      "INFO:tensorflow:loss = 0.007962931, step = 50501 (0.212 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.941\n",
      "INFO:tensorflow:loss = 0.010460144, step = 50601 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.793\n",
      "INFO:tensorflow:loss = 0.011612343, step = 50701 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.869\n",
      "INFO:tensorflow:loss = 0.019141298, step = 50801 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.845\n",
      "INFO:tensorflow:loss = 0.016415292, step = 50901 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.099\n",
      "INFO:tensorflow:loss = 0.0067565916, step = 51001 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.183\n",
      "INFO:tensorflow:loss = 0.017235782, step = 51101 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.816\n",
      "INFO:tensorflow:loss = 0.009165549, step = 51201 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.024\n",
      "INFO:tensorflow:loss = 0.013115405, step = 51301 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.156\n",
      "INFO:tensorflow:loss = 0.006884975, step = 51401 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.933\n",
      "INFO:tensorflow:loss = 0.029697599, step = 51501 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.956\n",
      "INFO:tensorflow:loss = 0.008664636, step = 51601 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.641\n",
      "INFO:tensorflow:loss = 0.013338229, step = 51701 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.189\n",
      "INFO:tensorflow:loss = 0.008891218, step = 51801 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.627\n",
      "INFO:tensorflow:loss = 0.011505337, step = 51901 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.594\n",
      "INFO:tensorflow:loss = 0.018077953, step = 52001 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.921\n",
      "INFO:tensorflow:loss = 0.0111584775, step = 52101 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.32\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.010002465, step = 52201 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.137\n",
      "INFO:tensorflow:loss = 0.013323818, step = 52301 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.415\n",
      "INFO:tensorflow:loss = 0.009106889, step = 52401 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 436.398\n",
      "INFO:tensorflow:loss = 0.01559157, step = 52501 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.383\n",
      "INFO:tensorflow:loss = 0.009623263, step = 52601 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.939\n",
      "INFO:tensorflow:loss = 0.014402029, step = 52701 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.51\n",
      "INFO:tensorflow:loss = 0.018671855, step = 52801 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.029\n",
      "INFO:tensorflow:loss = 0.010615724, step = 52901 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.32\n",
      "INFO:tensorflow:loss = 0.022925321, step = 53001 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 441.391\n",
      "INFO:tensorflow:loss = 0.0099629145, step = 53101 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 434.863\n",
      "INFO:tensorflow:loss = 0.0072393436, step = 53201 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.248\n",
      "INFO:tensorflow:loss = 0.0069563612, step = 53301 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.126\n",
      "INFO:tensorflow:loss = 0.010672631, step = 53401 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.557\n",
      "INFO:tensorflow:loss = 0.013554306, step = 53501 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.554\n",
      "INFO:tensorflow:loss = 0.012460485, step = 53601 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.672\n",
      "INFO:tensorflow:loss = 0.008272471, step = 53701 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.53\n",
      "INFO:tensorflow:loss = 0.014449131, step = 53801 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.809\n",
      "INFO:tensorflow:loss = 0.009777293, step = 53901 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 439.47\n",
      "INFO:tensorflow:loss = 0.0074636, step = 54001 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.32\n",
      "INFO:tensorflow:loss = 0.013860963, step = 54101 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.632\n",
      "INFO:tensorflow:loss = 0.012820887, step = 54201 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.506\n",
      "INFO:tensorflow:loss = 0.008376101, step = 54301 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.42\n",
      "INFO:tensorflow:loss = 0.010489011, step = 54401 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.957\n",
      "INFO:tensorflow:loss = 0.010256257, step = 54501 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.898\n",
      "INFO:tensorflow:loss = 0.014355536, step = 54601 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.819\n",
      "INFO:tensorflow:loss = 0.018218504, step = 54701 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.171\n",
      "INFO:tensorflow:loss = 0.018401045, step = 54801 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.387\n",
      "INFO:tensorflow:loss = 0.010837175, step = 54901 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.449\n",
      "INFO:tensorflow:loss = 0.023127666, step = 55001 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.249\n",
      "INFO:tensorflow:loss = 0.00845889, step = 55101 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.219\n",
      "INFO:tensorflow:loss = 0.01010492, step = 55201 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 434.135\n",
      "INFO:tensorflow:loss = 0.016563151, step = 55301 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.236\n",
      "INFO:tensorflow:loss = 0.007889459, step = 55401 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.894\n",
      "INFO:tensorflow:loss = 0.015438937, step = 55501 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.106\n",
      "INFO:tensorflow:loss = 0.0112075135, step = 55601 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.292\n",
      "INFO:tensorflow:loss = 0.008602387, step = 55701 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.049\n",
      "INFO:tensorflow:loss = 0.019720443, step = 55801 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.999\n",
      "INFO:tensorflow:loss = 0.0107068755, step = 55901 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.716\n",
      "INFO:tensorflow:loss = 0.013610104, step = 56001 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.324\n",
      "INFO:tensorflow:loss = 0.011953295, step = 56101 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.051\n",
      "INFO:tensorflow:loss = 0.007036713, step = 56201 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.253\n",
      "INFO:tensorflow:loss = 0.007729063, step = 56301 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 434.567\n",
      "INFO:tensorflow:loss = 0.009673175, step = 56401 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.88\n",
      "INFO:tensorflow:loss = 0.008710217, step = 56501 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.313\n",
      "INFO:tensorflow:loss = 0.009871947, step = 56601 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.106\n",
      "INFO:tensorflow:loss = 0.01577001, step = 56701 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.14\n",
      "INFO:tensorflow:loss = 0.008722901, step = 56801 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.247\n",
      "INFO:tensorflow:loss = 0.0067295507, step = 56901 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.887\n",
      "INFO:tensorflow:loss = 0.015591727, step = 57001 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.091\n",
      "INFO:tensorflow:loss = 0.016812723, step = 57101 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.365\n",
      "INFO:tensorflow:loss = 0.0071815904, step = 57201 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.372\n",
      "INFO:tensorflow:loss = 0.008234596, step = 57301 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.883\n",
      "INFO:tensorflow:loss = 0.006778944, step = 57401 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.61\n",
      "INFO:tensorflow:loss = 0.016462786, step = 57501 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.407\n",
      "INFO:tensorflow:loss = 0.011442966, step = 57601 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.563\n",
      "INFO:tensorflow:loss = 0.010314176, step = 57701 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.593\n",
      "INFO:tensorflow:loss = 0.014682214, step = 57801 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.243\n",
      "INFO:tensorflow:loss = 0.008448712, step = 57901 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.085\n",
      "INFO:tensorflow:loss = 0.013199998, step = 58001 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.764\n",
      "INFO:tensorflow:loss = 0.014379088, step = 58101 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 428.857\n",
      "INFO:tensorflow:loss = 0.01093298, step = 58201 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.141\n",
      "INFO:tensorflow:loss = 0.01882251, step = 58301 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.242\n",
      "INFO:tensorflow:loss = 0.014925126, step = 58401 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.056\n",
      "INFO:tensorflow:loss = 0.014487972, step = 58501 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.399\n",
      "INFO:tensorflow:loss = 0.013404521, step = 58601 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.344\n",
      "INFO:tensorflow:loss = 0.009785076, step = 58701 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.433\n",
      "INFO:tensorflow:loss = 0.009689671, step = 58801 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.964\n",
      "INFO:tensorflow:loss = 0.01731895, step = 58901 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.964\n",
      "INFO:tensorflow:loss = 0.010827826, step = 59001 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.47\n",
      "INFO:tensorflow:loss = 0.012085641, step = 59101 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.42\n",
      "INFO:tensorflow:loss = 0.00816013, step = 59201 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.829\n",
      "INFO:tensorflow:loss = 0.010327052, step = 59301 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.756\n",
      "INFO:tensorflow:loss = 0.006225149, step = 59401 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.018\n",
      "INFO:tensorflow:loss = 0.011975176, step = 59501 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.831\n",
      "INFO:tensorflow:loss = 0.007859128, step = 59601 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.042\n",
      "INFO:tensorflow:loss = 0.006618174, step = 59701 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.001\n",
      "INFO:tensorflow:loss = 0.013110094, step = 59801 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.093\n",
      "INFO:tensorflow:loss = 0.01723581, step = 59901 (0.246 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 60000 into /tmp/tmplcezpthw/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-07:27:10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-60000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'previous_ensemble' dict for global step 60000: architecture/adanet/ensembles = b\"\\nc\\n,adanet/previous_ensemble/architecture/adanetB)\\x08\\x07\\x12\\x00B#| b'1_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.03574329, average_loss/adanet/subnetwork = 0.03454547, average_loss/adanet/uniform_average_ensemble = 0.03574329, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.04744883, loss/adanet/subnetwork = 0.04466034, loss/adanet/uniform_average_ensemble = 0.04744883, prediction/mean/adanet/adanet_weighted_ensemble = 3.148061, prediction/mean/adanet/subnetwork = 3.1383395, prediction/mean/adanet/uniform_average_ensemble = 3.148061\n",
      "INFO:tensorflow:Saving candidate '2_layer_dnn' dict for global step 60000: architecture/adanet/ensembles = b\"\\n\\x83\\x01\\n;adanet/iteration_2/ensemble_2_layer_dnn/architecture/adanetB:\\x08\\x07\\x12\\x00B4| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.03383368, average_loss/adanet/subnetwork = 0.035940513, average_loss/adanet/uniform_average_ensemble = 0.033833675, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.04227618, loss/adanet/subnetwork = 0.038228974, loss/adanet/uniform_average_ensemble = 0.042276178, prediction/mean/adanet/adanet_weighted_ensemble = 3.14097, prediction/mean/adanet/subnetwork = 3.1267877, prediction/mean/adanet/uniform_average_ensemble = 3.14097\n",
      "INFO:tensorflow:Saving candidate '3_layer_dnn' dict for global step 60000: architecture/adanet/ensembles = b\"\\n\\x83\\x01\\n;adanet/iteration_2/ensemble_3_layer_dnn/architecture/adanetB:\\x08\\x07\\x12\\x00B4| b'1_layer_dnn' | b'2_layer_dnn' | b'3_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.03499535, average_loss/adanet/subnetwork = 0.03961562, average_loss/adanet/uniform_average_ensemble = 0.034995347, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.049417138, loss/adanet/subnetwork = 0.060717084, loss/adanet/uniform_average_ensemble = 0.04941713, prediction/mean/adanet/adanet_weighted_ensemble = 3.1505394, prediction/mean/adanet/subnetwork = 3.1554945, prediction/mean/adanet/uniform_average_ensemble = 3.1505394\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-19-07:27:12\n",
      "INFO:tensorflow:Saving dict for global step 60000: average_loss = 0.03499535, average_loss/adanet/adanet_weighted_ensemble = 0.03499535, average_loss/adanet/subnetwork = 0.03961562, average_loss/adanet/uniform_average_ensemble = 0.034995347, global_step = 60000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.049417138, loss/adanet/adanet_weighted_ensemble = 0.049417138, loss/adanet/subnetwork = 0.060717084, loss/adanet/uniform_average_ensemble = 0.04941713, prediction/mean = 3.1505394, prediction/mean/adanet/adanet_weighted_ensemble = 3.1505394, prediction/mean/adanet/subnetwork = 3.1554945, prediction/mean/adanet/uniform_average_ensemble = 3.1505394\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 60000: /tmp/tmplcezpthw/model.ckpt-60000\n",
      "INFO:tensorflow:Loss for final step: 0.0068436847.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Starting ensemble evaluation for iteration 2\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-60000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Encountered end of input after 14 evaluations\n",
      "INFO:tensorflow:Computed ensemble metrics: adanet_loss/previous_ensemble = 0.016144, adanet_loss/2_layer_dnn = 0.012531, adanet_loss/3_layer_dnn = 0.013370\n",
      "INFO:tensorflow:Finished ensemble evaluation for iteration 2\n",
      "INFO:tensorflow:'2_layer_dnn' at index 1 is moving onto the next iteration\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Freezing best ensemble to /tmp/tmplcezpthw/frozen/ensemble-2.meta\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-60000\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Overwriting checkpoint with new graph for iteration 3 to /tmp/tmplcezpthw/model.ckpt-60000\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-60000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Finished training Adanet iteration 2\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 3\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/increment.ckpt-3\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 60000 into /tmp/tmplcezpthw/model.ckpt.\n",
      "INFO:tensorflow:loss = 0.014618196, step = 60001\n",
      "INFO:tensorflow:global_step/sec: 137.361\n",
      "INFO:tensorflow:loss = 0.015324675, step = 60101 (0.732 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.137\n",
      "INFO:tensorflow:loss = 0.013147956, step = 60201 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.547\n",
      "INFO:tensorflow:loss = 0.010710466, step = 60301 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.909\n",
      "INFO:tensorflow:loss = 0.0149217155, step = 60401 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.994\n",
      "INFO:tensorflow:loss = 0.009752747, step = 60501 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.508\n",
      "INFO:tensorflow:loss = 0.010015659, step = 60601 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.486\n",
      "INFO:tensorflow:loss = 0.0082929935, step = 60701 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.669\n",
      "INFO:tensorflow:loss = 0.012442582, step = 60801 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.512\n",
      "INFO:tensorflow:loss = 0.014609805, step = 60901 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.515\n",
      "INFO:tensorflow:loss = 0.01769008, step = 61001 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.697\n",
      "INFO:tensorflow:loss = 0.01460558, step = 61101 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.325\n",
      "INFO:tensorflow:loss = 0.017269634, step = 61201 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.709\n",
      "INFO:tensorflow:loss = 0.009953711, step = 61301 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.396\n",
      "INFO:tensorflow:loss = 0.008323548, step = 61401 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.626\n",
      "INFO:tensorflow:loss = 0.0069469735, step = 61501 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.957\n",
      "INFO:tensorflow:loss = 0.009425541, step = 61601 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 369.311\n",
      "INFO:tensorflow:loss = 0.015549474, step = 61701 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.188\n",
      "INFO:tensorflow:loss = 0.012195961, step = 61801 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 363.08\n",
      "INFO:tensorflow:loss = 0.014084639, step = 61901 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.819\n",
      "INFO:tensorflow:loss = 0.008874404, step = 62001 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.251\n",
      "INFO:tensorflow:loss = 0.019104276, step = 62101 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.874\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.014049293, step = 62201 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.624\n",
      "INFO:tensorflow:loss = 0.012692098, step = 62301 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.775\n",
      "INFO:tensorflow:loss = 0.009777548, step = 62401 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.404\n",
      "INFO:tensorflow:loss = 0.015690956, step = 62501 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.937\n",
      "INFO:tensorflow:loss = 0.008675326, step = 62601 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.342\n",
      "INFO:tensorflow:loss = 0.013752129, step = 62701 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.059\n",
      "INFO:tensorflow:loss = 0.010698263, step = 62801 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 436.797\n",
      "INFO:tensorflow:loss = 0.010378689, step = 62901 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.163\n",
      "INFO:tensorflow:loss = 0.009665085, step = 63001 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.868\n",
      "INFO:tensorflow:loss = 0.0077294107, step = 63101 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 434.492\n",
      "INFO:tensorflow:loss = 0.017108265, step = 63201 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.967\n",
      "INFO:tensorflow:loss = 0.020117972, step = 63301 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.178\n",
      "INFO:tensorflow:loss = 0.014202065, step = 63401 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.148\n",
      "INFO:tensorflow:loss = 0.018432248, step = 63501 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.486\n",
      "INFO:tensorflow:loss = 0.008274863, step = 63601 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.9\n",
      "INFO:tensorflow:loss = 0.009492518, step = 63701 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.385\n",
      "INFO:tensorflow:loss = 0.011550526, step = 63801 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.756\n",
      "INFO:tensorflow:loss = 0.006735836, step = 63901 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.163\n",
      "INFO:tensorflow:loss = 0.013601805, step = 64001 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 428.681\n",
      "INFO:tensorflow:loss = 0.014022928, step = 64101 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.721\n",
      "INFO:tensorflow:loss = 0.009238082, step = 64201 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.98\n",
      "INFO:tensorflow:loss = 0.0060990183, step = 64301 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.758\n",
      "INFO:tensorflow:loss = 0.007990805, step = 64401 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.569\n",
      "INFO:tensorflow:loss = 0.012560089, step = 64501 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.622\n",
      "INFO:tensorflow:loss = 0.019384747, step = 64601 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.645\n",
      "INFO:tensorflow:loss = 0.008437725, step = 64701 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.209\n",
      "INFO:tensorflow:loss = 0.007267407, step = 64801 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.153\n",
      "INFO:tensorflow:loss = 0.014230946, step = 64901 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.409\n",
      "INFO:tensorflow:loss = 0.0057494575, step = 65001 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.449\n",
      "INFO:tensorflow:loss = 0.009977046, step = 65101 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.945\n",
      "INFO:tensorflow:loss = 0.012609685, step = 65201 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.364\n",
      "INFO:tensorflow:loss = 0.017385384, step = 65301 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.464\n",
      "INFO:tensorflow:loss = 0.010247884, step = 65401 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.576\n",
      "INFO:tensorflow:loss = 0.007686475, step = 65501 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.769\n",
      "INFO:tensorflow:loss = 0.023107788, step = 65601 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.587\n",
      "INFO:tensorflow:loss = 0.009419886, step = 65701 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.552\n",
      "INFO:tensorflow:loss = 0.008118824, step = 65801 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.611\n",
      "INFO:tensorflow:loss = 0.00832498, step = 65901 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.984\n",
      "INFO:tensorflow:loss = 0.008988107, step = 66001 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.144\n",
      "INFO:tensorflow:loss = 0.014885001, step = 66101 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.537\n",
      "INFO:tensorflow:loss = 0.011876632, step = 66201 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.991\n",
      "INFO:tensorflow:loss = 0.0065168478, step = 66301 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.208\n",
      "INFO:tensorflow:loss = 0.009724436, step = 66401 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.192\n",
      "INFO:tensorflow:loss = 0.015590355, step = 66501 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.483\n",
      "INFO:tensorflow:loss = 0.009362937, step = 66601 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.658\n",
      "INFO:tensorflow:loss = 0.008811436, step = 66701 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.913\n",
      "INFO:tensorflow:loss = 0.0062438073, step = 66801 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.022\n",
      "INFO:tensorflow:loss = 0.00825912, step = 66901 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.004\n",
      "INFO:tensorflow:loss = 0.008796803, step = 67001 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.642\n",
      "INFO:tensorflow:loss = 0.0096104015, step = 67101 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.06\n",
      "INFO:tensorflow:loss = 0.010513611, step = 67201 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.757\n",
      "INFO:tensorflow:loss = 0.0143983355, step = 67301 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 417\n",
      "INFO:tensorflow:loss = 0.009308563, step = 67401 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.633\n",
      "INFO:tensorflow:loss = 0.005975537, step = 67501 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.325\n",
      "INFO:tensorflow:loss = 0.011849377, step = 67601 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.015\n",
      "INFO:tensorflow:loss = 0.019730046, step = 67701 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.944\n",
      "INFO:tensorflow:loss = 0.014363306, step = 67801 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.665\n",
      "INFO:tensorflow:loss = 0.008542209, step = 67901 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.152\n",
      "INFO:tensorflow:loss = 0.0142958965, step = 68001 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.364\n",
      "INFO:tensorflow:loss = 0.019585459, step = 68101 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.489\n",
      "INFO:tensorflow:loss = 0.021717029, step = 68201 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 472.88\n",
      "INFO:tensorflow:loss = 0.013836529, step = 68301 (0.211 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.3\n",
      "INFO:tensorflow:loss = 0.010187588, step = 68401 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.074\n",
      "INFO:tensorflow:loss = 0.006819522, step = 68501 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.476\n",
      "INFO:tensorflow:loss = 0.0106657315, step = 68601 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.29\n",
      "INFO:tensorflow:loss = 0.010790708, step = 68701 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.117\n",
      "INFO:tensorflow:loss = 0.01868429, step = 68801 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.269\n",
      "INFO:tensorflow:loss = 0.028418373, step = 68901 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.878\n",
      "INFO:tensorflow:loss = 0.021071881, step = 69001 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.79\n",
      "INFO:tensorflow:loss = 0.012398725, step = 69101 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.678\n",
      "INFO:tensorflow:loss = 0.01205198, step = 69201 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.162\n",
      "INFO:tensorflow:loss = 0.006516939, step = 69301 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.741\n",
      "INFO:tensorflow:loss = 0.008661759, step = 69401 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.224\n",
      "INFO:tensorflow:loss = 0.011256352, step = 69501 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.327\n",
      "INFO:tensorflow:loss = 0.011116728, step = 69601 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.495\n",
      "INFO:tensorflow:loss = 0.009571152, step = 69701 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.363\n",
      "INFO:tensorflow:loss = 0.008698425, step = 69801 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.698\n",
      "INFO:tensorflow:loss = 0.014010752, step = 69901 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.085\n",
      "INFO:tensorflow:loss = 0.013709282, step = 70001 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.488\n",
      "INFO:tensorflow:loss = 0.0069980896, step = 70101 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.064\n",
      "INFO:tensorflow:loss = 0.011916493, step = 70201 (0.232 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 424.886\n",
      "INFO:tensorflow:loss = 0.01797392, step = 70301 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.407\n",
      "INFO:tensorflow:loss = 0.011372922, step = 70401 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 436.412\n",
      "INFO:tensorflow:loss = 0.005085878, step = 70501 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.976\n",
      "INFO:tensorflow:loss = 0.008556084, step = 70601 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.757\n",
      "INFO:tensorflow:loss = 0.009044853, step = 70701 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.659\n",
      "INFO:tensorflow:loss = 0.0126294475, step = 70801 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.657\n",
      "INFO:tensorflow:loss = 0.013419818, step = 70901 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.989\n",
      "INFO:tensorflow:loss = 0.0065733306, step = 71001 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.139\n",
      "INFO:tensorflow:loss = 0.011789247, step = 71101 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.635\n",
      "INFO:tensorflow:loss = 0.008844627, step = 71201 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.776\n",
      "INFO:tensorflow:loss = 0.010758241, step = 71301 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 428.782\n",
      "INFO:tensorflow:loss = 0.0066980226, step = 71401 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.934\n",
      "INFO:tensorflow:loss = 0.019456618, step = 71501 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.362\n",
      "INFO:tensorflow:loss = 0.009102147, step = 71601 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.539\n",
      "INFO:tensorflow:loss = 0.01298802, step = 71701 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.587\n",
      "INFO:tensorflow:loss = 0.008180043, step = 71801 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.833\n",
      "INFO:tensorflow:loss = 0.008207346, step = 71901 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.228\n",
      "INFO:tensorflow:loss = 0.011458617, step = 72001 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.547\n",
      "INFO:tensorflow:loss = 0.008495214, step = 72101 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.974\n",
      "INFO:tensorflow:loss = 0.009371083, step = 72201 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.245\n",
      "INFO:tensorflow:loss = 0.010389559, step = 72301 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.571\n",
      "INFO:tensorflow:loss = 0.008151503, step = 72401 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.316\n",
      "INFO:tensorflow:loss = 0.011045817, step = 72501 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.854\n",
      "INFO:tensorflow:loss = 0.006550163, step = 72601 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.99\n",
      "INFO:tensorflow:loss = 0.013558472, step = 72701 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.645\n",
      "INFO:tensorflow:loss = 0.016302168, step = 72801 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.583\n",
      "INFO:tensorflow:loss = 0.009635454, step = 72901 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.819\n",
      "INFO:tensorflow:loss = 0.013733709, step = 73001 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.573\n",
      "INFO:tensorflow:loss = 0.010683964, step = 73101 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.285\n",
      "INFO:tensorflow:loss = 0.0057868017, step = 73201 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.382\n",
      "INFO:tensorflow:loss = 0.0071841152, step = 73301 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.667\n",
      "INFO:tensorflow:loss = 0.010826091, step = 73401 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.758\n",
      "INFO:tensorflow:loss = 0.016430188, step = 73501 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.767\n",
      "INFO:tensorflow:loss = 0.008017408, step = 73601 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.116\n",
      "INFO:tensorflow:loss = 0.006444059, step = 73701 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.802\n",
      "INFO:tensorflow:loss = 0.010060328, step = 73801 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.117\n",
      "INFO:tensorflow:loss = 0.007508277, step = 73901 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.49\n",
      "INFO:tensorflow:loss = 0.006029764, step = 74001 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.547\n",
      "INFO:tensorflow:loss = 0.010934742, step = 74101 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.557\n",
      "INFO:tensorflow:loss = 0.010610297, step = 74201 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.058\n",
      "INFO:tensorflow:loss = 0.007095433, step = 74301 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.98\n",
      "INFO:tensorflow:loss = 0.010214947, step = 74401 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.528\n",
      "INFO:tensorflow:loss = 0.0067400355, step = 74501 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.137\n",
      "INFO:tensorflow:loss = 0.011095614, step = 74601 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.738\n",
      "INFO:tensorflow:loss = 0.014567835, step = 74701 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.863\n",
      "INFO:tensorflow:loss = 0.015405769, step = 74801 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.566\n",
      "INFO:tensorflow:loss = 0.007940804, step = 74901 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.108\n",
      "INFO:tensorflow:loss = 0.01965886, step = 75001 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.814\n",
      "INFO:tensorflow:loss = 0.005841125, step = 75101 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.11\n",
      "INFO:tensorflow:loss = 0.00767474, step = 75201 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.787\n",
      "INFO:tensorflow:loss = 0.01429298, step = 75301 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.948\n",
      "INFO:tensorflow:loss = 0.0060202274, step = 75401 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.576\n",
      "INFO:tensorflow:loss = 0.012017151, step = 75501 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.597\n",
      "INFO:tensorflow:loss = 0.008363055, step = 75601 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.265\n",
      "INFO:tensorflow:loss = 0.0070346887, step = 75701 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.696\n",
      "INFO:tensorflow:loss = 0.015342252, step = 75801 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.931\n",
      "INFO:tensorflow:loss = 0.0069325063, step = 75901 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.488\n",
      "INFO:tensorflow:loss = 0.01099792, step = 76001 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.309\n",
      "INFO:tensorflow:loss = 0.008994274, step = 76101 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.022\n",
      "INFO:tensorflow:loss = 0.008077464, step = 76201 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.462\n",
      "INFO:tensorflow:loss = 0.0057298057, step = 76301 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.509\n",
      "INFO:tensorflow:loss = 0.011365313, step = 76401 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.402\n",
      "INFO:tensorflow:loss = 0.008590138, step = 76501 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.652\n",
      "INFO:tensorflow:loss = 0.0074969917, step = 76601 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.304\n",
      "INFO:tensorflow:loss = 0.012659554, step = 76701 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.34\n",
      "INFO:tensorflow:loss = 0.007802845, step = 76801 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.953\n",
      "INFO:tensorflow:loss = 0.007350701, step = 76901 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.445\n",
      "INFO:tensorflow:loss = 0.012026157, step = 77001 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.43\n",
      "INFO:tensorflow:loss = 0.014631044, step = 77101 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.263\n",
      "INFO:tensorflow:loss = 0.005720512, step = 77201 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.151\n",
      "INFO:tensorflow:loss = 0.0069215083, step = 77301 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.901\n",
      "INFO:tensorflow:loss = 0.005737162, step = 77401 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.216\n",
      "INFO:tensorflow:loss = 0.01014894, step = 77501 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.704\n",
      "INFO:tensorflow:loss = 0.010485262, step = 77601 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.134\n",
      "INFO:tensorflow:loss = 0.0061357417, step = 77701 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.948\n",
      "INFO:tensorflow:loss = 0.0129522085, step = 77801 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.412\n",
      "INFO:tensorflow:loss = 0.005956566, step = 77901 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.95\n",
      "INFO:tensorflow:loss = 0.0138694085, step = 78001 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.339\n",
      "INFO:tensorflow:loss = 0.012074534, step = 78101 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.463\n",
      "INFO:tensorflow:loss = 0.012735812, step = 78201 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.564\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.013359522, step = 78301 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.34\n",
      "INFO:tensorflow:loss = 0.011084627, step = 78401 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.246\n",
      "INFO:tensorflow:loss = 0.010920718, step = 78501 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.927\n",
      "INFO:tensorflow:loss = 0.010683222, step = 78601 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.348\n",
      "INFO:tensorflow:loss = 0.008939888, step = 78701 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.25\n",
      "INFO:tensorflow:loss = 0.008002801, step = 78801 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.499\n",
      "INFO:tensorflow:loss = 0.01741761, step = 78901 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.949\n",
      "INFO:tensorflow:loss = 0.009349164, step = 79001 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.631\n",
      "INFO:tensorflow:loss = 0.011067858, step = 79101 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.309\n",
      "INFO:tensorflow:loss = 0.009572687, step = 79201 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.237\n",
      "INFO:tensorflow:loss = 0.010873193, step = 79301 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.072\n",
      "INFO:tensorflow:loss = 0.005899853, step = 79401 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.283\n",
      "INFO:tensorflow:loss = 0.0078644985, step = 79501 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.318\n",
      "INFO:tensorflow:loss = 0.00965251, step = 79601 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.309\n",
      "INFO:tensorflow:loss = 0.0060071563, step = 79701 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.593\n",
      "INFO:tensorflow:loss = 0.0082059335, step = 79801 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.528\n",
      "INFO:tensorflow:loss = 0.015458113, step = 79901 (0.237 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 80000 into /tmp/tmplcezpthw/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-07:28:15\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-80000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'previous_ensemble' dict for global step 80000: architecture/adanet/ensembles = b\"\\nt\\n,adanet/previous_ensemble/architecture/adanetB:\\x08\\x07\\x12\\x00B4| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.03383368, average_loss/adanet/subnetwork = 0.035940513, average_loss/adanet/uniform_average_ensemble = 0.033833675, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.04227618, loss/adanet/subnetwork = 0.038228974, loss/adanet/uniform_average_ensemble = 0.042276174, prediction/mean/adanet/adanet_weighted_ensemble = 3.14097, prediction/mean/adanet/subnetwork = 3.1267877, prediction/mean/adanet/uniform_average_ensemble = 3.14097\n",
      "INFO:tensorflow:Saving candidate '2_layer_dnn' dict for global step 80000: architecture/adanet/ensembles = b\"\\n\\x94\\x01\\n;adanet/iteration_3/ensemble_2_layer_dnn/architecture/adanetBK\\x08\\x07\\x12\\x00BE| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.03166156, average_loss/adanet/subnetwork = 0.029604359, average_loss/adanet/uniform_average_ensemble = 0.03166156, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.04027512, loss/adanet/subnetwork = 0.03914282, loss/adanet/uniform_average_ensemble = 0.04027511, prediction/mean/adanet/adanet_weighted_ensemble = 3.1422162, prediction/mean/adanet/subnetwork = 3.1459546, prediction/mean/adanet/uniform_average_ensemble = 3.1422162\n",
      "INFO:tensorflow:Saving candidate '3_layer_dnn' dict for global step 80000: architecture/adanet/ensembles = b\"\\n\\x94\\x01\\n;adanet/iteration_3/ensemble_3_layer_dnn/architecture/adanetBK\\x08\\x07\\x12\\x00BE| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' | b'3_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.030794745, average_loss/adanet/subnetwork = 0.028511057, average_loss/adanet/uniform_average_ensemble = 0.030794747, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.03752663, loss/adanet/subnetwork = 0.030393401, loss/adanet/uniform_average_ensemble = 0.037526634, prediction/mean/adanet/adanet_weighted_ensemble = 3.1357996, prediction/mean/adanet/subnetwork = 3.1202893, prediction/mean/adanet/uniform_average_ensemble = 3.1357996\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-19-07:28:17\n",
      "INFO:tensorflow:Saving dict for global step 80000: average_loss = 0.030794745, average_loss/adanet/adanet_weighted_ensemble = 0.030794745, average_loss/adanet/subnetwork = 0.028511057, average_loss/adanet/uniform_average_ensemble = 0.030794747, global_step = 80000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.03752663, loss/adanet/adanet_weighted_ensemble = 0.03752663, loss/adanet/subnetwork = 0.030393401, loss/adanet/uniform_average_ensemble = 0.037526634, prediction/mean = 3.1357996, prediction/mean/adanet/adanet_weighted_ensemble = 3.1357996, prediction/mean/adanet/subnetwork = 3.1202893, prediction/mean/adanet/uniform_average_ensemble = 3.1357996\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 80000: /tmp/tmplcezpthw/model.ckpt-80000\n",
      "INFO:tensorflow:Loss for final step: 0.006525429.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Starting ensemble evaluation for iteration 3\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-80000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Encountered end of input after 14 evaluations\n",
      "INFO:tensorflow:Computed ensemble metrics: adanet_loss/previous_ensemble = 0.012531, adanet_loss/2_layer_dnn = 0.011483, adanet_loss/3_layer_dnn = 0.010555\n",
      "INFO:tensorflow:Finished ensemble evaluation for iteration 3\n",
      "INFO:tensorflow:'3_layer_dnn' at index 2 is moving onto the next iteration\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Freezing best ensemble to /tmp/tmplcezpthw/frozen/ensemble-3.meta\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-80000\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-3.meta with features: ['x'].\n",
      "INFO:tensorflow:Overwriting checkpoint with new graph for iteration 4 to /tmp/tmplcezpthw/model.ckpt-80000\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-80000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Finished training Adanet iteration 3\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 4\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-3.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/increment.ckpt-4\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 80000 into /tmp/tmplcezpthw/model.ckpt.\n",
      "INFO:tensorflow:loss = 0.012771627, step = 80001\n",
      "INFO:tensorflow:global_step/sec: 131.484\n",
      "INFO:tensorflow:loss = 0.0121352915, step = 80101 (0.762 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.686\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.012014208, step = 80201 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.276\n",
      "INFO:tensorflow:loss = 0.008635065, step = 80301 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.59\n",
      "INFO:tensorflow:loss = 0.012590489, step = 80401 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.17\n",
      "INFO:tensorflow:loss = 0.008656274, step = 80501 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.518\n",
      "INFO:tensorflow:loss = 0.0081290435, step = 80601 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.457\n",
      "INFO:tensorflow:loss = 0.007441818, step = 80701 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.09\n",
      "INFO:tensorflow:loss = 0.011507606, step = 80801 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 330.144\n",
      "INFO:tensorflow:loss = 0.012954185, step = 80901 (0.306 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.795\n",
      "INFO:tensorflow:loss = 0.014048508, step = 81001 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.829\n",
      "INFO:tensorflow:loss = 0.013463169, step = 81101 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.308\n",
      "INFO:tensorflow:loss = 0.013465128, step = 81201 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.731\n",
      "INFO:tensorflow:loss = 0.0067526144, step = 81301 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.732\n",
      "INFO:tensorflow:loss = 0.0065478464, step = 81401 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.271\n",
      "INFO:tensorflow:loss = 0.0066968743, step = 81501 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 351.615\n",
      "INFO:tensorflow:loss = 0.007793371, step = 81601 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.443\n",
      "INFO:tensorflow:loss = 0.014154049, step = 81701 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.346\n",
      "INFO:tensorflow:loss = 0.010598782, step = 81801 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.529\n",
      "INFO:tensorflow:loss = 0.01202817, step = 81901 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.688\n",
      "INFO:tensorflow:loss = 0.008131183, step = 82001 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.704\n",
      "INFO:tensorflow:loss = 0.017173555, step = 82101 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.175\n",
      "INFO:tensorflow:loss = 0.011480885, step = 82201 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.898\n",
      "INFO:tensorflow:loss = 0.0102535095, step = 82301 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.972\n",
      "INFO:tensorflow:loss = 0.0076961704, step = 82401 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.908\n",
      "INFO:tensorflow:loss = 0.013443191, step = 82501 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.545\n",
      "INFO:tensorflow:loss = 0.0067555346, step = 82601 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 296.522\n",
      "INFO:tensorflow:loss = 0.011653241, step = 82701 (0.337 sec)\n",
      "INFO:tensorflow:global_step/sec: 272.737\n",
      "INFO:tensorflow:loss = 0.009919509, step = 82801 (0.366 sec)\n",
      "INFO:tensorflow:global_step/sec: 298.383\n",
      "INFO:tensorflow:loss = 0.008859744, step = 82901 (0.336 sec)\n",
      "INFO:tensorflow:global_step/sec: 303.817\n",
      "INFO:tensorflow:loss = 0.009619437, step = 83001 (0.329 sec)\n",
      "INFO:tensorflow:global_step/sec: 294.966\n",
      "INFO:tensorflow:loss = 0.008154873, step = 83101 (0.339 sec)\n",
      "INFO:tensorflow:global_step/sec: 298.459\n",
      "INFO:tensorflow:loss = 0.014442224, step = 83201 (0.337 sec)\n",
      "INFO:tensorflow:global_step/sec: 305.453\n",
      "INFO:tensorflow:loss = 0.016844384, step = 83301 (0.330 sec)\n",
      "INFO:tensorflow:global_step/sec: 297.047\n",
      "INFO:tensorflow:loss = 0.011380184, step = 83401 (0.331 sec)\n",
      "INFO:tensorflow:global_step/sec: 291.525\n",
      "INFO:tensorflow:loss = 0.018231083, step = 83501 (0.347 sec)\n",
      "INFO:tensorflow:global_step/sec: 299.82\n",
      "INFO:tensorflow:loss = 0.0076518646, step = 83601 (0.333 sec)\n",
      "INFO:tensorflow:global_step/sec: 310.479\n",
      "INFO:tensorflow:loss = 0.01101666, step = 83701 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 299.431\n",
      "INFO:tensorflow:loss = 0.008646242, step = 83801 (0.333 sec)\n",
      "INFO:tensorflow:global_step/sec: 284.666\n",
      "INFO:tensorflow:loss = 0.0053476538, step = 83901 (0.351 sec)\n",
      "INFO:tensorflow:global_step/sec: 301.069\n",
      "INFO:tensorflow:loss = 0.0126082385, step = 84001 (0.333 sec)\n",
      "INFO:tensorflow:global_step/sec: 306.547\n",
      "INFO:tensorflow:loss = 0.011207275, step = 84101 (0.326 sec)\n",
      "INFO:tensorflow:global_step/sec: 313.775\n",
      "INFO:tensorflow:loss = 0.008020308, step = 84201 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.418\n",
      "INFO:tensorflow:loss = 0.0051603187, step = 84301 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.288\n",
      "INFO:tensorflow:loss = 0.0075262506, step = 84401 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.062\n",
      "INFO:tensorflow:loss = 0.010481458, step = 84501 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.82\n",
      "INFO:tensorflow:loss = 0.012546435, step = 84601 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.199\n",
      "INFO:tensorflow:loss = 0.005997817, step = 84701 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.942\n",
      "INFO:tensorflow:loss = 0.008798814, step = 84801 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 304.141\n",
      "INFO:tensorflow:loss = 0.011663855, step = 84901 (0.326 sec)\n",
      "INFO:tensorflow:global_step/sec: 314.492\n",
      "INFO:tensorflow:loss = 0.004963225, step = 85001 (0.319 sec)\n",
      "INFO:tensorflow:global_step/sec: 345.206\n",
      "INFO:tensorflow:loss = 0.0074084736, step = 85101 (0.288 sec)\n",
      "INFO:tensorflow:global_step/sec: 305.08\n",
      "INFO:tensorflow:loss = 0.009785203, step = 85201 (0.327 sec)\n",
      "INFO:tensorflow:global_step/sec: 337.08\n",
      "INFO:tensorflow:loss = 0.01590272, step = 85301 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.678\n",
      "INFO:tensorflow:loss = 0.008546837, step = 85401 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.936\n",
      "INFO:tensorflow:loss = 0.006370773, step = 85501 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.528\n",
      "INFO:tensorflow:loss = 0.01786882, step = 85601 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 363.933\n",
      "INFO:tensorflow:loss = 0.008412208, step = 85701 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.192\n",
      "INFO:tensorflow:loss = 0.007872367, step = 85801 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 320.525\n",
      "INFO:tensorflow:loss = 0.008893929, step = 85901 (0.307 sec)\n",
      "INFO:tensorflow:global_step/sec: 340.774\n",
      "INFO:tensorflow:loss = 0.011444823, step = 86001 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 311.637\n",
      "INFO:tensorflow:loss = 0.013627991, step = 86101 (0.324 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.396\n",
      "INFO:tensorflow:loss = 0.011433875, step = 86201 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 332.342\n",
      "INFO:tensorflow:loss = 0.0057342714, step = 86301 (0.301 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.021\n",
      "INFO:tensorflow:loss = 0.009024318, step = 86401 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 350.185\n",
      "INFO:tensorflow:loss = 0.012929679, step = 86501 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.716\n",
      "INFO:tensorflow:loss = 0.0084665585, step = 86601 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.219\n",
      "INFO:tensorflow:loss = 0.011061366, step = 86701 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 334.988\n",
      "INFO:tensorflow:loss = 0.0077197826, step = 86801 (0.299 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.108\n",
      "INFO:tensorflow:loss = 0.012015966, step = 86901 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.039\n",
      "INFO:tensorflow:loss = 0.008734614, step = 87001 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.38\n",
      "INFO:tensorflow:loss = 0.009587226, step = 87101 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 207.283\n",
      "INFO:tensorflow:loss = 0.011050206, step = 87201 (0.483 sec)\n",
      "INFO:tensorflow:global_step/sec: 283.215\n",
      "INFO:tensorflow:loss = 0.011074185, step = 87301 (0.353 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.653\n",
      "INFO:tensorflow:loss = 0.009702394, step = 87401 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 320.372\n",
      "INFO:tensorflow:loss = 0.0047701485, step = 87501 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 295.381\n",
      "INFO:tensorflow:loss = 0.008785698, step = 87601 (0.339 sec)\n",
      "INFO:tensorflow:global_step/sec: 272.285\n",
      "INFO:tensorflow:loss = 0.017635599, step = 87701 (0.368 sec)\n",
      "INFO:tensorflow:global_step/sec: 206.481\n",
      "INFO:tensorflow:loss = 0.011502991, step = 87801 (0.483 sec)\n",
      "INFO:tensorflow:global_step/sec: 196.518\n",
      "INFO:tensorflow:loss = 0.008678509, step = 87901 (0.509 sec)\n",
      "INFO:tensorflow:global_step/sec: 233.566\n",
      "INFO:tensorflow:loss = 0.014851244, step = 88001 (0.428 sec)\n",
      "INFO:tensorflow:global_step/sec: 125.448\n",
      "INFO:tensorflow:loss = 0.013566, step = 88101 (0.805 sec)\n",
      "INFO:tensorflow:global_step/sec: 266.258\n",
      "INFO:tensorflow:loss = 0.01460981, step = 88201 (0.368 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 437.248\n",
      "INFO:tensorflow:loss = 0.013460955, step = 88301 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.017\n",
      "INFO:tensorflow:loss = 0.007128556, step = 88401 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.569\n",
      "INFO:tensorflow:loss = 0.005910028, step = 88501 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.059\n",
      "INFO:tensorflow:loss = 0.009571411, step = 88601 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 284.202\n",
      "INFO:tensorflow:loss = 0.009896141, step = 88701 (0.350 sec)\n",
      "INFO:tensorflow:global_step/sec: 266.092\n",
      "INFO:tensorflow:loss = 0.017198602, step = 88801 (0.376 sec)\n",
      "INFO:tensorflow:global_step/sec: 351.454\n",
      "INFO:tensorflow:loss = 0.022983724, step = 88901 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.425\n",
      "INFO:tensorflow:loss = 0.019097468, step = 89001 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.878\n",
      "INFO:tensorflow:loss = 0.011339415, step = 89101 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.846\n",
      "INFO:tensorflow:loss = 0.010781945, step = 89201 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.145\n",
      "INFO:tensorflow:loss = 0.0057053496, step = 89301 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.257\n",
      "INFO:tensorflow:loss = 0.009208331, step = 89401 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 324.886\n",
      "INFO:tensorflow:loss = 0.011385071, step = 89501 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.078\n",
      "INFO:tensorflow:loss = 0.012900406, step = 89601 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.563\n",
      "INFO:tensorflow:loss = 0.008409393, step = 89701 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 339.439\n",
      "INFO:tensorflow:loss = 0.0077062985, step = 89801 (0.294 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.714\n",
      "INFO:tensorflow:loss = 0.015839955, step = 89901 (0.311 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.022\n",
      "INFO:tensorflow:loss = 0.012045793, step = 90001 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.596\n",
      "INFO:tensorflow:loss = 0.006062067, step = 90101 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 314.417\n",
      "INFO:tensorflow:loss = 0.013103823, step = 90201 (0.323 sec)\n",
      "INFO:tensorflow:global_step/sec: 233.006\n",
      "INFO:tensorflow:loss = 0.011567364, step = 90301 (0.425 sec)\n",
      "INFO:tensorflow:global_step/sec: 197.539\n",
      "INFO:tensorflow:loss = 0.007015214, step = 90401 (0.505 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.223\n",
      "INFO:tensorflow:loss = 0.004756853, step = 90501 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.108\n",
      "INFO:tensorflow:loss = 0.008641821, step = 90601 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.594\n",
      "INFO:tensorflow:loss = 0.008739402, step = 90701 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.792\n",
      "INFO:tensorflow:loss = 0.014424091, step = 90801 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.824\n",
      "INFO:tensorflow:loss = 0.015975047, step = 90901 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 434.936\n",
      "INFO:tensorflow:loss = 0.007067974, step = 91001 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.821\n",
      "INFO:tensorflow:loss = 0.011657877, step = 91101 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.533\n",
      "INFO:tensorflow:loss = 0.0055408156, step = 91201 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.804\n",
      "INFO:tensorflow:loss = 0.010151473, step = 91301 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 437.322\n",
      "INFO:tensorflow:loss = 0.005707156, step = 91401 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.797\n",
      "INFO:tensorflow:loss = 0.018164214, step = 91501 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.7\n",
      "INFO:tensorflow:loss = 0.008018797, step = 91601 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.935\n",
      "INFO:tensorflow:loss = 0.011219912, step = 91701 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.789\n",
      "INFO:tensorflow:loss = 0.0064485804, step = 91801 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.513\n",
      "INFO:tensorflow:loss = 0.0062573696, step = 91901 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.114\n",
      "INFO:tensorflow:loss = 0.009774809, step = 92001 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 327.982\n",
      "INFO:tensorflow:loss = 0.007911012, step = 92101 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.627\n",
      "INFO:tensorflow:loss = 0.007353039, step = 92201 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 243.592\n",
      "INFO:tensorflow:loss = 0.008872479, step = 92301 (0.409 sec)\n",
      "INFO:tensorflow:global_step/sec: 226.922\n",
      "INFO:tensorflow:loss = 0.00765727, step = 92401 (0.441 sec)\n",
      "INFO:tensorflow:global_step/sec: 276.897\n",
      "INFO:tensorflow:loss = 0.009893896, step = 92501 (0.365 sec)\n",
      "INFO:tensorflow:global_step/sec: 304.303\n",
      "INFO:tensorflow:loss = 0.0064428737, step = 92601 (0.324 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.905\n",
      "INFO:tensorflow:loss = 0.012109645, step = 92701 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.054\n",
      "INFO:tensorflow:loss = 0.015083933, step = 92801 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 275.409\n",
      "INFO:tensorflow:loss = 0.009505656, step = 92901 (0.363 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.192\n",
      "INFO:tensorflow:loss = 0.015061153, step = 93001 (0.306 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.702\n",
      "INFO:tensorflow:loss = 0.009282329, step = 93101 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.088\n",
      "INFO:tensorflow:loss = 0.005166143, step = 93201 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.598\n",
      "INFO:tensorflow:loss = 0.0067952527, step = 93301 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 346.11\n",
      "INFO:tensorflow:loss = 0.008600542, step = 93401 (0.288 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.23\n",
      "INFO:tensorflow:loss = 0.010867932, step = 93501 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 285.132\n",
      "INFO:tensorflow:loss = 0.0077117365, step = 93601 (0.351 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.691\n",
      "INFO:tensorflow:loss = 0.006293457, step = 93701 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 265.788\n",
      "INFO:tensorflow:loss = 0.009726633, step = 93801 (0.381 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.379\n",
      "INFO:tensorflow:loss = 0.0081541585, step = 93901 (0.294 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.444\n",
      "INFO:tensorflow:loss = 0.0067407116, step = 94001 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.823\n",
      "INFO:tensorflow:loss = 0.011675996, step = 94101 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 327.191\n",
      "INFO:tensorflow:loss = 0.010407218, step = 94201 (0.305 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.932\n",
      "INFO:tensorflow:loss = 0.0072696796, step = 94301 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.405\n",
      "INFO:tensorflow:loss = 0.007630905, step = 94401 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.235\n",
      "INFO:tensorflow:loss = 0.007388452, step = 94501 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 369.113\n",
      "INFO:tensorflow:loss = 0.009430501, step = 94601 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 312.899\n",
      "INFO:tensorflow:loss = 0.012620936, step = 94701 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.78\n",
      "INFO:tensorflow:loss = 0.015241542, step = 94801 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.508\n",
      "INFO:tensorflow:loss = 0.007987112, step = 94901 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 362.022\n",
      "INFO:tensorflow:loss = 0.01785532, step = 95001 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.717\n",
      "INFO:tensorflow:loss = 0.0064375233, step = 95101 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.248\n",
      "INFO:tensorflow:loss = 0.007950843, step = 95201 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.958\n",
      "INFO:tensorflow:loss = 0.014186453, step = 95301 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.027\n",
      "INFO:tensorflow:loss = 0.0055443337, step = 95401 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.461\n",
      "INFO:tensorflow:loss = 0.0114728045, step = 95501 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.195\n",
      "INFO:tensorflow:loss = 0.008126725, step = 95601 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.172\n",
      "INFO:tensorflow:loss = 0.0058695627, step = 95701 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.554\n",
      "INFO:tensorflow:loss = 0.015734559, step = 95801 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.609\n",
      "INFO:tensorflow:loss = 0.0072773313, step = 95901 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.264\n",
      "INFO:tensorflow:loss = 0.008003841, step = 96001 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.336\n",
      "INFO:tensorflow:loss = 0.008413629, step = 96101 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.912\n",
      "INFO:tensorflow:loss = 0.006204883, step = 96201 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 285.179\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.005483456, step = 96301 (0.353 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.645\n",
      "INFO:tensorflow:loss = 0.010100526, step = 96401 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.007\n",
      "INFO:tensorflow:loss = 0.011245234, step = 96501 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.114\n",
      "INFO:tensorflow:loss = 0.006841069, step = 96601 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.838\n",
      "INFO:tensorflow:loss = 0.01420257, step = 96701 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 332.931\n",
      "INFO:tensorflow:loss = 0.008054491, step = 96801 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.436\n",
      "INFO:tensorflow:loss = 0.006289351, step = 96901 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 369.889\n",
      "INFO:tensorflow:loss = 0.012853891, step = 97001 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.088\n",
      "INFO:tensorflow:loss = 0.013619079, step = 97101 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.18\n",
      "INFO:tensorflow:loss = 0.00617637, step = 97201 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.376\n",
      "INFO:tensorflow:loss = 0.006251939, step = 97301 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.498\n",
      "INFO:tensorflow:loss = 0.0056112036, step = 97401 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.966\n",
      "INFO:tensorflow:loss = 0.01399757, step = 97501 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.345\n",
      "INFO:tensorflow:loss = 0.008695771, step = 97601 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.321\n",
      "INFO:tensorflow:loss = 0.0059478916, step = 97701 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.645\n",
      "INFO:tensorflow:loss = 0.0134305395, step = 97801 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.186\n",
      "INFO:tensorflow:loss = 0.0063350755, step = 97901 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.106\n",
      "INFO:tensorflow:loss = 0.011608165, step = 98001 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.835\n",
      "INFO:tensorflow:loss = 0.010383707, step = 98101 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.891\n",
      "INFO:tensorflow:loss = 0.008898836, step = 98201 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 434.788\n",
      "INFO:tensorflow:loss = 0.012249054, step = 98301 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.238\n",
      "INFO:tensorflow:loss = 0.010068682, step = 98401 (0.300 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.951\n",
      "INFO:tensorflow:loss = 0.010171878, step = 98501 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.147\n",
      "INFO:tensorflow:loss = 0.009882946, step = 98601 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.122\n",
      "INFO:tensorflow:loss = 0.0062706796, step = 98701 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.136\n",
      "INFO:tensorflow:loss = 0.007751474, step = 98801 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.674\n",
      "INFO:tensorflow:loss = 0.013043297, step = 98901 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.346\n",
      "INFO:tensorflow:loss = 0.009482058, step = 99001 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.897\n",
      "INFO:tensorflow:loss = 0.010178535, step = 99101 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.21\n",
      "INFO:tensorflow:loss = 0.006907771, step = 99201 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.752\n",
      "INFO:tensorflow:loss = 0.008608325, step = 99301 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.152\n",
      "INFO:tensorflow:loss = 0.004759037, step = 99401 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.277\n",
      "INFO:tensorflow:loss = 0.0062178248, step = 99501 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.483\n",
      "INFO:tensorflow:loss = 0.008499165, step = 99601 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.033\n",
      "INFO:tensorflow:loss = 0.0066619543, step = 99701 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.205\n",
      "INFO:tensorflow:loss = 0.01100979, step = 99801 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.778\n",
      "INFO:tensorflow:loss = 0.014243005, step = 99901 (0.267 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 100000 into /tmp/tmplcezpthw/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmplcezpthw/frozen/ensemble-3.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-07:29:31\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmplcezpthw/model.ckpt-100000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'previous_ensemble' dict for global step 100000: architecture/adanet/ensembles = b\"\\n\\x85\\x01\\n,adanet/previous_ensemble/architecture/adanetBK\\x08\\x07\\x12\\x00BE| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' | b'3_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.030794745, average_loss/adanet/subnetwork = 0.028511057, average_loss/adanet/uniform_average_ensemble = 0.030794745, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.03752663, loss/adanet/subnetwork = 0.030393401, loss/adanet/uniform_average_ensemble = 0.03752663, prediction/mean/adanet/adanet_weighted_ensemble = 3.1357996, prediction/mean/adanet/subnetwork = 3.1202893, prediction/mean/adanet/uniform_average_ensemble = 3.1357996\n",
      "INFO:tensorflow:Saving candidate '3_layer_dnn' dict for global step 100000: architecture/adanet/ensembles = b\"\\n\\xa5\\x01\\n;adanet/iteration_4/ensemble_3_layer_dnn/architecture/adanetB\\\\\\x08\\x07\\x12\\x00BV| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' | b'3_layer_dnn' | b'3_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.031948246, average_loss/adanet/subnetwork = 0.041267823, average_loss/adanet/uniform_average_ensemble = 0.031948246, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.038237095, loss/adanet/subnetwork = 0.04582113, loss/adanet/uniform_average_ensemble = 0.038237102, prediction/mean/adanet/adanet_weighted_ensemble = 3.13776, prediction/mean/adanet/subnetwork = 3.1456003, prediction/mean/adanet/uniform_average_ensemble = 3.13776\n",
      "INFO:tensorflow:Saving candidate '4_layer_dnn' dict for global step 100000: architecture/adanet/ensembles = b\"\\n\\xa5\\x01\\n;adanet/iteration_4/ensemble_4_layer_dnn/architecture/adanetB\\\\\\x08\\x07\\x12\\x00BV| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' | b'3_layer_dnn' | b'4_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.031013792, average_loss/adanet/subnetwork = 0.036653724, average_loss/adanet/uniform_average_ensemble = 0.031013792, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.03993301, loss/adanet/subnetwork = 0.056759786, loss/adanet/uniform_average_ensemble = 0.03993302, prediction/mean/adanet/adanet_weighted_ensemble = 3.1297462, prediction/mean/adanet/subnetwork = 3.1055298, prediction/mean/adanet/uniform_average_ensemble = 3.1297457\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-19-07:29:33\n",
      "INFO:tensorflow:Saving dict for global step 100000: average_loss = 0.031948246, average_loss/adanet/adanet_weighted_ensemble = 0.031948246, average_loss/adanet/subnetwork = 0.041267823, average_loss/adanet/uniform_average_ensemble = 0.031948246, global_step = 100000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.038237095, loss/adanet/adanet_weighted_ensemble = 0.038237095, loss/adanet/subnetwork = 0.04582113, loss/adanet/uniform_average_ensemble = 0.038237102, prediction/mean = 3.13776, prediction/mean/adanet/adanet_weighted_ensemble = 3.13776, prediction/mean/adanet/subnetwork = 3.1456003, prediction/mean/adanet/uniform_average_ensemble = 3.13776\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 100000: /tmp/tmplcezpthw/model.ckpt-100000\n",
      "INFO:tensorflow:Loss for final step: 0.0051082023.\n",
      "Loss: 0.031948246\n",
      "Architecture: b\"| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' | b'3_layer_dnn' | b'3_layer_dnn' |\"\n"
     ]
    }
   ],
   "source": [
    "#@title AdaNet parameters\n",
    "LEARNING_RATE = 0.001  #@param {type:\"number\"}\n",
    "TRAIN_STEPS = 100000  #@param {type:\"integer\"}\n",
    "BATCH_SIZE = 32  #@param {type:\"integer\"}\n",
    "\n",
    "LEARN_MIXTURE_WEIGHTS = False  #@param {type:\"boolean\"}\n",
    "ADANET_LAMBDA = 0  #@param {type:\"number\"}\n",
    "BOOSTING_ITERATIONS = 5  #@param {type:\"integer\"}\n",
    "\n",
    "\n",
    "def train_and_evaluate(learn_mixture_weights=LEARN_MIXTURE_WEIGHTS,\n",
    "                       adanet_lambda=ADANET_LAMBDA):\n",
    "  \"\"\"Trains an `adanet.Estimator` to predict housing prices.\"\"\"\n",
    "\n",
    "  estimator = adanet.Estimator(\n",
    "      # Since we are predicting housing prices, we'll use a regression\n",
    "      # head that optimizes for MSE.\n",
    "      head=tf.contrib.estimator.regression_head(\n",
    "          loss_reduction=tf.losses.Reduction.SUM_OVER_BATCH_SIZE),\n",
    "\n",
    "      # Define the generator, which defines our search space of subnetworks\n",
    "      # to train as candidates to add to the final AdaNet model.\n",
    "      subnetwork_generator=SimpleDNNGenerator(\n",
    "          optimizer=tf.train.RMSPropOptimizer(learning_rate=LEARNING_RATE),\n",
    "          learn_mixture_weights=learn_mixture_weights,\n",
    "          seed=RANDOM_SEED),\n",
    "\n",
    "      # Lambda is a the strength of complexity regularization. A larger\n",
    "      # value will penalize more complex subnetworks.\n",
    "      adanet_lambda=adanet_lambda,\n",
    "\n",
    "      # The number of train steps per iteration.\n",
    "      max_iteration_steps=TRAIN_STEPS // BOOSTING_ITERATIONS,\n",
    "\n",
    "      # The evaluator will evaluate the model on the full training set to\n",
    "      # compute the overall AdaNet loss (train loss + complexity\n",
    "      # regularization) to select the best candidate to include in the\n",
    "      # final AdaNet model.\n",
    "      evaluator=adanet.Evaluator(\n",
    "          input_fn=input_fn(\"train\", training=False, batch_size=BATCH_SIZE)),\n",
    "\n",
    "      # Configuration for Estimators.\n",
    "      config=tf.estimator.RunConfig(\n",
    "          save_checkpoints_steps=50000,\n",
    "          save_summary_steps=50000,\n",
    "          tf_random_seed=RANDOM_SEED))\n",
    "\n",
    "  # Train and evaluate using using the tf.estimator tooling.\n",
    "  train_spec = tf.estimator.TrainSpec(\n",
    "      input_fn=input_fn(\"train\", training=True, batch_size=BATCH_SIZE),\n",
    "      max_steps=TRAIN_STEPS)\n",
    "  eval_spec = tf.estimator.EvalSpec(\n",
    "      input_fn=input_fn(\"test\", training=False, batch_size=BATCH_SIZE),\n",
    "      steps=None)\n",
    "  return tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)\n",
    "\n",
    "\n",
    "def ensemble_architecture(result):\n",
    "  \"\"\"Extracts the ensemble architecture from evaluation results.\"\"\"\n",
    "\n",
    "  architecture = result[\"architecture/adanet/ensembles\"]\n",
    "  # The architecture is a serialized Summary proto for TensorBoard.\n",
    "  summary_proto = tf.summary.Summary.FromString(architecture)\n",
    "  return summary_proto.value[0].tensor.string_val[0]\n",
    "\n",
    "\n",
    "results, _ = train_and_evaluate()\n",
    "print(\"Loss:\", results[\"average_loss\"])\n",
    "print(\"Architecture:\", ensemble_architecture(results))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "A0vBGKeTj0Kf"
   },
   "source": [
    "These hyperparameters preduce a model that achieves **0.0348** MSE on the test\n",
    "set. Notice that the ensemble is composed of 5 subnetworks, each one a hidden\n",
    "layer deeper than the previous. The most complex subnetwork is made of 5 hidden\n",
    "layers.\n",
    "\n",
    "Since `SimpleDNNGenerator` produces subnetworks of varying complexity, and our\n",
    "model gives each one an equal weight, AdaNet selected the subnetwork that most\n",
    "lowered the ensemble's training loss at each iteration, likely the one with the\n",
    "most hidden layers, since it has the most capacity, and we aren't penalizing\n",
    "more complex subnetworks (yet).\n",
    "\n",
    "Next, instead of assigning equal weight to each subnetwork, let's learn the\n",
    "mixture weights as a convex optimization problem using SGD:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "height": 71,
     "test": {
      "output": "ignore",
      "timeout": 900
     }
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 234614,
     "status": "ok",
     "timestamp": 1539111509785,
     "user": {
      "displayName": "",
      "photoUrl": "",
      "userId": ""
     },
     "user_tz": 240
    },
    "id": "RXfJ6cb_KO35",
    "outputId": "d8e8e404-ed03-4d41-ce46-fd82ea75eff1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpsbdccn23\n",
      "INFO:tensorflow:Using config: {'_save_checkpoints_secs': None, '_experimental_distribute': None, '_service': None, '_task_id': 0, '_is_chief': True, '_master': '', '_evaluation_master': '', '_train_distribute': None, '_model_dir': '/tmp/tmpsbdccn23', '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f8029968a90>, '_keep_checkpoint_every_n_hours': 10000, '_global_id_in_cluster': 0, '_keep_checkpoint_max': 5, '_save_checkpoints_steps': 50000, '_tf_random_seed': 42, '_session_config': allow_soft_placement: true\n",
      "graph_options {\n",
      "  rewrite_options {\n",
      "    meta_optimizer_iterations: ONE\n",
      "  }\n",
      "}\n",
      ", '_protocol': None, '_device_fn': None, '_save_summary_steps': 50000, '_num_ps_replicas': 0, '_eval_distribute': None, '_num_worker_replicas': 1, '_log_step_count_steps': 100, '_task_type': 'worker'}\n",
      "INFO:tensorflow:Running training and evaluation locally (non-distributed).\n",
      "INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps 50000 or save_checkpoints_secs None.\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 0\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpsbdccn23/model.ckpt.\n",
      "INFO:tensorflow:loss = 21.773132, step = 1\n",
      "INFO:tensorflow:global_step/sec: 151.659\n",
      "INFO:tensorflow:loss = 0.6285208, step = 101 (0.661 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.914\n",
      "INFO:tensorflow:loss = 0.568697, step = 201 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.447\n",
      "INFO:tensorflow:loss = 0.07774219, step = 301 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 298.158\n",
      "INFO:tensorflow:loss = 0.08270247, step = 401 (0.332 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.096\n",
      "INFO:tensorflow:loss = 0.08153409, step = 501 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.588\n",
      "INFO:tensorflow:loss = 0.05655239, step = 601 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.393\n",
      "INFO:tensorflow:loss = 0.025883064, step = 701 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.02\n",
      "INFO:tensorflow:loss = 0.030127691, step = 801 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.488\n",
      "INFO:tensorflow:loss = 0.03756215, step = 901 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.863\n",
      "INFO:tensorflow:loss = 0.06788642, step = 1001 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.318\n",
      "INFO:tensorflow:loss = 0.036306262, step = 1101 (0.310 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.289\n",
      "INFO:tensorflow:loss = 0.05074877, step = 1201 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 321.58\n",
      "INFO:tensorflow:loss = 0.10058461, step = 1301 (0.311 sec)\n",
      "INFO:tensorflow:global_step/sec: 300.699\n",
      "INFO:tensorflow:loss = 0.026643617, step = 1401 (0.334 sec)\n",
      "INFO:tensorflow:global_step/sec: 318.013\n",
      "INFO:tensorflow:loss = 0.020885482, step = 1501 (0.313 sec)\n",
      "INFO:tensorflow:global_step/sec: 323.705\n",
      "INFO:tensorflow:loss = 0.03239681, step = 1601 (0.315 sec)\n",
      "INFO:tensorflow:global_step/sec: 328.631\n",
      "INFO:tensorflow:loss = 0.04160305, step = 1701 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.201\n",
      "INFO:tensorflow:loss = 0.0352926, step = 1801 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.005\n",
      "INFO:tensorflow:loss = 0.044745784, step = 1901 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.216\n",
      "INFO:tensorflow:loss = 0.02993768, step = 2001 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.851\n",
      "INFO:tensorflow:loss = 0.047246575, step = 2101 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 290.003\n",
      "INFO:tensorflow:loss = 0.024866767, step = 2201 (0.346 sec)\n",
      "INFO:tensorflow:global_step/sec: 306.232\n",
      "INFO:tensorflow:loss = 0.025053538, step = 2301 (0.332 sec)\n",
      "INFO:tensorflow:global_step/sec: 319.194\n",
      "INFO:tensorflow:loss = 0.022536863, step = 2401 (0.315 sec)\n",
      "INFO:tensorflow:global_step/sec: 327.319\n",
      "INFO:tensorflow:loss = 0.04780043, step = 2501 (0.299 sec)\n",
      "INFO:tensorflow:global_step/sec: 330.195\n",
      "INFO:tensorflow:loss = 0.032027524, step = 2601 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.554\n",
      "INFO:tensorflow:loss = 0.033754565, step = 2701 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.456\n",
      "INFO:tensorflow:loss = 0.014495807, step = 2801 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.815\n",
      "INFO:tensorflow:loss = 0.031205792, step = 2901 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.675\n",
      "INFO:tensorflow:loss = 0.026793242, step = 3001 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 445.07\n",
      "INFO:tensorflow:loss = 0.02696861, step = 3101 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.002\n",
      "INFO:tensorflow:loss = 0.027100282, step = 3201 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 452.535\n",
      "INFO:tensorflow:loss = 0.03591666, step = 3301 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.136\n",
      "INFO:tensorflow:loss = 0.025515229, step = 3401 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.819\n",
      "INFO:tensorflow:loss = 0.049373504, step = 3501 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 441.761\n",
      "INFO:tensorflow:loss = 0.024171133, step = 3601 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.165\n",
      "INFO:tensorflow:loss = 0.017237274, step = 3701 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 442.471\n",
      "INFO:tensorflow:loss = 0.020128746, step = 3801 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 443.692\n",
      "INFO:tensorflow:loss = 0.021598278, step = 3901 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.398\n",
      "INFO:tensorflow:loss = 0.03772788, step = 4001 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 453.543\n",
      "INFO:tensorflow:loss = 0.040997066, step = 4101 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 447.837\n",
      "INFO:tensorflow:loss = 0.021314848, step = 4201 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 449.319\n",
      "INFO:tensorflow:loss = 0.03397343, step = 4301 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 291.817\n",
      "INFO:tensorflow:loss = 0.03742571, step = 4401 (0.343 sec)\n",
      "INFO:tensorflow:global_step/sec: 349.156\n",
      "INFO:tensorflow:loss = 0.04003142, step = 4501 (0.287 sec)\n",
      "INFO:tensorflow:global_step/sec: 444.919\n",
      "INFO:tensorflow:loss = 0.037306767, step = 4601 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 324.799\n",
      "INFO:tensorflow:loss = 0.050043724, step = 4701 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.035\n",
      "INFO:tensorflow:loss = 0.04509888, step = 4801 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.386\n",
      "INFO:tensorflow:loss = 0.023579072, step = 4901 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.009\n",
      "INFO:tensorflow:loss = 0.014783351, step = 5001 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 465.426\n",
      "INFO:tensorflow:loss = 0.021115372, step = 5101 (0.214 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.114\n",
      "INFO:tensorflow:loss = 0.02869285, step = 5201 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 446.474\n",
      "INFO:tensorflow:loss = 0.044227358, step = 5301 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 442.508\n",
      "INFO:tensorflow:loss = 0.015665509, step = 5401 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 439.36\n",
      "INFO:tensorflow:loss = 0.017735064, step = 5501 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 452.882\n",
      "INFO:tensorflow:loss = 0.026888551, step = 5601 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 450.627\n",
      "INFO:tensorflow:loss = 0.025225505, step = 5701 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 455.843\n",
      "INFO:tensorflow:loss = 0.032536294, step = 5801 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 453.967\n",
      "INFO:tensorflow:loss = 0.014429852, step = 5901 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 446.021\n",
      "INFO:tensorflow:loss = 0.020685814, step = 6001 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 447.631\n",
      "INFO:tensorflow:loss = 0.035909995, step = 6101 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 454.564\n",
      "INFO:tensorflow:loss = 0.053759962, step = 6201 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.801\n",
      "INFO:tensorflow:loss = 0.02680358, step = 6301 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.766\n",
      "INFO:tensorflow:loss = 0.035358958, step = 6401 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.454\n",
      "INFO:tensorflow:loss = 0.04194645, step = 6501 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 324.174\n",
      "INFO:tensorflow:loss = 0.025395717, step = 6601 (0.309 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 281.343\n",
      "INFO:tensorflow:loss = 0.020708136, step = 6701 (0.350 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.588\n",
      "INFO:tensorflow:loss = 0.020778513, step = 6801 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 318.526\n",
      "INFO:tensorflow:loss = 0.016706605, step = 6901 (0.313 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.289\n",
      "INFO:tensorflow:loss = 0.026352338, step = 7001 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 195.744\n",
      "INFO:tensorflow:loss = 0.02617799, step = 7101 (0.508 sec)\n",
      "INFO:tensorflow:global_step/sec: 221.335\n",
      "INFO:tensorflow:loss = 0.017944645, step = 7201 (0.449 sec)\n",
      "INFO:tensorflow:global_step/sec: 337.755\n",
      "INFO:tensorflow:loss = 0.04815356, step = 7301 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.037\n",
      "INFO:tensorflow:loss = 0.025712956, step = 7401 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 330.685\n",
      "INFO:tensorflow:loss = 0.025668332, step = 7501 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 284.466\n",
      "INFO:tensorflow:loss = 0.01265048, step = 7601 (0.352 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.699\n",
      "INFO:tensorflow:loss = 0.022912, step = 7701 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 292.865\n",
      "INFO:tensorflow:loss = 0.019265048, step = 7801 (0.347 sec)\n",
      "INFO:tensorflow:global_step/sec: 316.832\n",
      "INFO:tensorflow:loss = 0.017410079, step = 7901 (0.310 sec)\n",
      "INFO:tensorflow:global_step/sec: 316.283\n",
      "INFO:tensorflow:loss = 0.024891995, step = 8001 (0.319 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.299\n",
      "INFO:tensorflow:loss = 0.021594066, step = 8101 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.749\n",
      "INFO:tensorflow:loss = 0.040524982, step = 8201 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.697\n",
      "INFO:tensorflow:loss = 0.023627196, step = 8301 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 458.265\n",
      "INFO:tensorflow:loss = 0.047175795, step = 8401 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 440.262\n",
      "INFO:tensorflow:loss = 0.020435514, step = 8501 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.995\n",
      "INFO:tensorflow:loss = 0.028949004, step = 8601 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.975\n",
      "INFO:tensorflow:loss = 0.022818578, step = 8701 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.266\n",
      "INFO:tensorflow:loss = 0.035533242, step = 8801 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.668\n",
      "INFO:tensorflow:loss = 0.054676503, step = 8901 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.338\n",
      "INFO:tensorflow:loss = 0.042836588, step = 9001 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.352\n",
      "INFO:tensorflow:loss = 0.020139698, step = 9101 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.011\n",
      "INFO:tensorflow:loss = 0.035128064, step = 9201 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 285.239\n",
      "INFO:tensorflow:loss = 0.014006899, step = 9301 (0.348 sec)\n",
      "INFO:tensorflow:global_step/sec: 276.22\n",
      "INFO:tensorflow:loss = 0.017652035, step = 9401 (0.364 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.221\n",
      "INFO:tensorflow:loss = 0.036492873, step = 9501 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.216\n",
      "INFO:tensorflow:loss = 0.032858968, step = 9601 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 340.446\n",
      "INFO:tensorflow:loss = 0.01183607, step = 9701 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.786\n",
      "INFO:tensorflow:loss = 0.021484602, step = 9801 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.491\n",
      "INFO:tensorflow:loss = 0.022089211, step = 9901 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.769\n",
      "INFO:tensorflow:loss = 0.029635977, step = 10001 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.734\n",
      "INFO:tensorflow:loss = 0.021254197, step = 10101 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 324.609\n",
      "INFO:tensorflow:loss = 0.04409772, step = 10201 (0.309 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.939\n",
      "INFO:tensorflow:loss = 0.03241565, step = 10301 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 314.298\n",
      "INFO:tensorflow:loss = 0.034868922, step = 10401 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.678\n",
      "INFO:tensorflow:loss = 0.03933882, step = 10501 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.201\n",
      "INFO:tensorflow:loss = 0.025350904, step = 10601 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.66\n",
      "INFO:tensorflow:loss = 0.012285138, step = 10701 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.542\n",
      "INFO:tensorflow:loss = 0.021395218, step = 10801 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.402\n",
      "INFO:tensorflow:loss = 0.027021939, step = 10901 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.055\n",
      "INFO:tensorflow:loss = 0.010032699, step = 11001 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 347.133\n",
      "INFO:tensorflow:loss = 0.028143033, step = 11101 (0.288 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.235\n",
      "INFO:tensorflow:loss = 0.012899801, step = 11201 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 458.722\n",
      "INFO:tensorflow:loss = 0.041966215, step = 11301 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 451.553\n",
      "INFO:tensorflow:loss = 0.018754564, step = 11401 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.239\n",
      "INFO:tensorflow:loss = 0.061010823, step = 11501 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.054\n",
      "INFO:tensorflow:loss = 0.014163884, step = 11601 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.244\n",
      "INFO:tensorflow:loss = 0.0238958, step = 11701 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.105\n",
      "INFO:tensorflow:loss = 0.01313608, step = 11801 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 355.109\n",
      "INFO:tensorflow:loss = 0.035015076, step = 11901 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.513\n",
      "INFO:tensorflow:loss = 0.025251625, step = 12001 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.553\n",
      "INFO:tensorflow:loss = 0.014988018, step = 12101 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.608\n",
      "INFO:tensorflow:loss = 0.010151859, step = 12201 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 344.769\n",
      "INFO:tensorflow:loss = 0.018342845, step = 12301 (0.290 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.301\n",
      "INFO:tensorflow:loss = 0.016532978, step = 12401 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.181\n",
      "INFO:tensorflow:loss = 0.023617888, step = 12501 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.51\n",
      "INFO:tensorflow:loss = 0.021013854, step = 12601 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.833\n",
      "INFO:tensorflow:loss = 0.019008305, step = 12701 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 463.087\n",
      "INFO:tensorflow:loss = 0.02030512, step = 12801 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.138\n",
      "INFO:tensorflow:loss = 0.026197102, step = 12901 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.009\n",
      "INFO:tensorflow:loss = 0.02379616, step = 13001 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 456.559\n",
      "INFO:tensorflow:loss = 0.029717997, step = 13101 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.366\n",
      "INFO:tensorflow:loss = 0.031371996, step = 13201 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 115.952\n",
      "INFO:tensorflow:loss = 0.015019484, step = 13301 (0.883 sec)\n",
      "INFO:tensorflow:global_step/sec: 281.093\n",
      "INFO:tensorflow:loss = 0.023579653, step = 13401 (0.337 sec)\n",
      "INFO:tensorflow:global_step/sec: 171.266\n",
      "INFO:tensorflow:loss = 0.02437912, step = 13501 (0.589 sec)\n",
      "INFO:tensorflow:global_step/sec: 260.224\n",
      "INFO:tensorflow:loss = 0.016651586, step = 13601 (0.379 sec)\n",
      "INFO:tensorflow:global_step/sec: 305.966\n",
      "INFO:tensorflow:loss = 0.014731653, step = 13701 (0.332 sec)\n",
      "INFO:tensorflow:global_step/sec: 339.617\n",
      "INFO:tensorflow:loss = 0.0352959, step = 13801 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 265.397\n",
      "INFO:tensorflow:loss = 0.021889884, step = 13901 (0.373 sec)\n",
      "INFO:tensorflow:global_step/sec: 334.262\n",
      "INFO:tensorflow:loss = 0.015826559, step = 14001 (0.300 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.809\n",
      "INFO:tensorflow:loss = 0.015816119, step = 14101 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 238.815\n",
      "INFO:tensorflow:loss = 0.016548071, step = 14201 (0.422 sec)\n",
      "INFO:tensorflow:global_step/sec: 264.662\n",
      "INFO:tensorflow:loss = 0.019683827, step = 14301 (0.383 sec)\n",
      "INFO:tensorflow:global_step/sec: 225.474\n",
      "INFO:tensorflow:loss = 0.012930198, step = 14401 (0.438 sec)\n",
      "INFO:tensorflow:global_step/sec: 201.01\n",
      "INFO:tensorflow:loss = 0.02105195, step = 14501 (0.497 sec)\n",
      "INFO:tensorflow:global_step/sec: 195.528\n",
      "INFO:tensorflow:loss = 0.028252073, step = 14601 (0.511 sec)\n",
      "INFO:tensorflow:global_step/sec: 201.718\n",
      "INFO:tensorflow:loss = 0.024852075, step = 14701 (0.494 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 165.709\n",
      "INFO:tensorflow:loss = 0.027445987, step = 14801 (0.603 sec)\n",
      "INFO:tensorflow:global_step/sec: 207.162\n",
      "INFO:tensorflow:loss = 0.04248728, step = 14901 (0.485 sec)\n",
      "INFO:tensorflow:global_step/sec: 170.31\n",
      "INFO:tensorflow:loss = 0.026710147, step = 15001 (0.590 sec)\n",
      "INFO:tensorflow:global_step/sec: 205.071\n",
      "INFO:tensorflow:loss = 0.014114473, step = 15101 (0.483 sec)\n",
      "INFO:tensorflow:global_step/sec: 238.418\n",
      "INFO:tensorflow:loss = 0.02495965, step = 15201 (0.420 sec)\n",
      "INFO:tensorflow:global_step/sec: 203.796\n",
      "INFO:tensorflow:loss = 0.024134407, step = 15301 (0.492 sec)\n",
      "INFO:tensorflow:global_step/sec: 268.184\n",
      "INFO:tensorflow:loss = 0.010985184, step = 15401 (0.380 sec)\n",
      "INFO:tensorflow:global_step/sec: 216.299\n",
      "INFO:tensorflow:loss = 0.020854292, step = 15501 (0.453 sec)\n",
      "INFO:tensorflow:global_step/sec: 166.083\n",
      "INFO:tensorflow:loss = 0.024032414, step = 15601 (0.606 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.255\n",
      "INFO:tensorflow:loss = 0.026188534, step = 15701 (0.310 sec)\n",
      "INFO:tensorflow:global_step/sec: 281.053\n",
      "INFO:tensorflow:loss = 0.03707458, step = 15801 (0.357 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.058\n",
      "INFO:tensorflow:loss = 0.030197568, step = 15901 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 450.355\n",
      "INFO:tensorflow:loss = 0.025264734, step = 16001 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 195.089\n",
      "INFO:tensorflow:loss = 0.05741349, step = 16101 (0.514 sec)\n",
      "INFO:tensorflow:global_step/sec: 321.249\n",
      "INFO:tensorflow:loss = 0.016645841, step = 16201 (0.310 sec)\n",
      "INFO:tensorflow:global_step/sec: 464.268\n",
      "INFO:tensorflow:loss = 0.032959394, step = 16301 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.474\n",
      "INFO:tensorflow:loss = 0.051571425, step = 16401 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.119\n",
      "INFO:tensorflow:loss = 0.0149952015, step = 16501 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 450.475\n",
      "INFO:tensorflow:loss = 0.020351136, step = 16601 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 444.693\n",
      "INFO:tensorflow:loss = 0.026955573, step = 16701 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 450.934\n",
      "INFO:tensorflow:loss = 0.029218871, step = 16801 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.734\n",
      "INFO:tensorflow:loss = 0.018013299, step = 16901 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.761\n",
      "INFO:tensorflow:loss = 0.026661586, step = 17001 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 428.081\n",
      "INFO:tensorflow:loss = 0.023716206, step = 17101 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 476.095\n",
      "INFO:tensorflow:loss = 0.0141041335, step = 17201 (0.210 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.496\n",
      "INFO:tensorflow:loss = 0.023004234, step = 17301 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.153\n",
      "INFO:tensorflow:loss = 0.014854547, step = 17401 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 199.873\n",
      "INFO:tensorflow:loss = 0.01979175, step = 17501 (0.500 sec)\n",
      "INFO:tensorflow:global_step/sec: 241.012\n",
      "INFO:tensorflow:loss = 0.02463748, step = 17601 (0.419 sec)\n",
      "INFO:tensorflow:global_step/sec: 272.143\n",
      "INFO:tensorflow:loss = 0.016009951, step = 17701 (0.363 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.144\n",
      "INFO:tensorflow:loss = 0.017083427, step = 17801 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.077\n",
      "INFO:tensorflow:loss = 0.015650112, step = 17901 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 316.194\n",
      "INFO:tensorflow:loss = 0.026930936, step = 18001 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 263.606\n",
      "INFO:tensorflow:loss = 0.029513063, step = 18101 (0.381 sec)\n",
      "INFO:tensorflow:global_step/sec: 247.061\n",
      "INFO:tensorflow:loss = 0.021598883, step = 18201 (0.402 sec)\n",
      "INFO:tensorflow:global_step/sec: 307.571\n",
      "INFO:tensorflow:loss = 0.02880636, step = 18301 (0.325 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.843\n",
      "INFO:tensorflow:loss = 0.030266954, step = 18401 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 440.884\n",
      "INFO:tensorflow:loss = 0.017472995, step = 18501 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.873\n",
      "INFO:tensorflow:loss = 0.018041471, step = 18601 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 281.66\n",
      "INFO:tensorflow:loss = 0.043840628, step = 18701 (0.352 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.345\n",
      "INFO:tensorflow:loss = 0.009787289, step = 18801 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.649\n",
      "INFO:tensorflow:loss = 0.026387483, step = 18901 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.762\n",
      "INFO:tensorflow:loss = 0.014039018, step = 19001 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.85\n",
      "INFO:tensorflow:loss = 0.02983357, step = 19101 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.486\n",
      "INFO:tensorflow:loss = 0.024136117, step = 19201 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.763\n",
      "INFO:tensorflow:loss = 0.019358052, step = 19301 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.257\n",
      "INFO:tensorflow:loss = 0.015025282, step = 19401 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.329\n",
      "INFO:tensorflow:loss = 0.014151661, step = 19501 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.523\n",
      "INFO:tensorflow:loss = 0.0117410775, step = 19601 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.669\n",
      "INFO:tensorflow:loss = 0.0114243785, step = 19701 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.451\n",
      "INFO:tensorflow:loss = 0.017932085, step = 19801 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.22\n",
      "INFO:tensorflow:loss = 0.048148207, step = 19901 (0.292 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 20000 into /tmp/tmpsbdccn23/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-07:33:01\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-20000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'linear' dict for global step 20000: architecture/adanet/ensembles = b\"\\nW\\n6adanet/iteration_0/ensemble_linear/architecture/adanetB\\x13\\x08\\x07\\x12\\x00B\\r| b'linear' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.049419954, average_loss/adanet/subnetwork = 0.049421385, average_loss/adanet/uniform_average_ensemble = 0.049421385, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.062510915, loss/adanet/subnetwork = 0.062442064, loss/adanet/uniform_average_ensemble = 0.062442064, prediction/mean/adanet/adanet_weighted_ensemble = 3.1072564, prediction/mean/adanet/subnetwork = 3.105895, prediction/mean/adanet/uniform_average_ensemble = 3.105895\n",
      "INFO:tensorflow:Saving candidate '1_layer_dnn' dict for global step 20000: architecture/adanet/ensembles = b\"\\na\\n;adanet/iteration_0/ensemble_1_layer_dnn/architecture/adanetB\\x18\\x08\\x07\\x12\\x00B\\x12| b'1_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.040044222, average_loss/adanet/subnetwork = 0.039825242, average_loss/adanet/uniform_average_ensemble = 0.039825242, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.05366586, loss/adanet/subnetwork = 0.053258948, loss/adanet/uniform_average_ensemble = 0.053258948, prediction/mean/adanet/adanet_weighted_ensemble = 3.1599553, prediction/mean/adanet/subnetwork = 3.157783, prediction/mean/adanet/uniform_average_ensemble = 3.157783\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-19-07:33:02\n",
      "INFO:tensorflow:Saving dict for global step 20000: average_loss = 0.040044222, average_loss/adanet/adanet_weighted_ensemble = 0.040044222, average_loss/adanet/subnetwork = 0.039825242, average_loss/adanet/uniform_average_ensemble = 0.039825242, global_step = 20000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.05366586, loss/adanet/adanet_weighted_ensemble = 0.05366586, loss/adanet/subnetwork = 0.053258948, loss/adanet/uniform_average_ensemble = 0.053258948, prediction/mean = 3.1599553, prediction/mean/adanet/adanet_weighted_ensemble = 3.1599553, prediction/mean/adanet/subnetwork = 3.157783, prediction/mean/adanet/uniform_average_ensemble = 3.157783\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20000: /tmp/tmpsbdccn23/model.ckpt-20000\n",
      "INFO:tensorflow:Loss for final step: 0.034430638.\n",
      "INFO:tensorflow:Starting ensemble evaluation for iteration 0\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-20000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Encountered end of input after 14 evaluations\n",
      "INFO:tensorflow:Computed ensemble metrics: adanet_loss/linear = 0.035082, adanet_loss/1_layer_dnn = 0.021031\n",
      "INFO:tensorflow:Finished ensemble evaluation for iteration 0\n",
      "INFO:tensorflow:'1_layer_dnn' at index 1 is moving onto the next iteration\n",
      "INFO:tensorflow:Freezing best ensemble to /tmp/tmpsbdccn23/frozen/ensemble-0.meta\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-20000\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Overwriting checkpoint with new graph for iteration 1 to /tmp/tmpsbdccn23/model.ckpt-20000\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-20000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Finished training Adanet iteration 0\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 1\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/increment.ckpt-1\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 20000 into /tmp/tmpsbdccn23/model.ckpt.\n",
      "INFO:tensorflow:loss = 0.0273384, step = 20001\n",
      "INFO:tensorflow:global_step/sec: 136.423\n",
      "INFO:tensorflow:loss = 0.026884198, step = 20101 (0.735 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.974\n",
      "INFO:tensorflow:loss = 0.020844536, step = 20201 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.155\n",
      "INFO:tensorflow:loss = 0.018991487, step = 20301 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.249\n",
      "INFO:tensorflow:loss = 0.02771065, step = 20401 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.271\n",
      "INFO:tensorflow:loss = 0.016612794, step = 20501 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.627\n",
      "INFO:tensorflow:loss = 0.018661585, step = 20601 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.981\n",
      "INFO:tensorflow:loss = 0.0115399975, step = 20701 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.383\n",
      "INFO:tensorflow:loss = 0.019317856, step = 20801 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.075\n",
      "INFO:tensorflow:loss = 0.015623036, step = 20901 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.016\n",
      "INFO:tensorflow:loss = 0.027647957, step = 21001 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.927\n",
      "INFO:tensorflow:loss = 0.024059676, step = 21101 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.833\n",
      "INFO:tensorflow:loss = 0.024439776, step = 21201 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.926\n",
      "INFO:tensorflow:loss = 0.028494542, step = 21301 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.279\n",
      "INFO:tensorflow:loss = 0.015246654, step = 21401 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.527\n",
      "INFO:tensorflow:loss = 0.010155028, step = 21501 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.669\n",
      "INFO:tensorflow:loss = 0.016802762, step = 21601 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.195\n",
      "INFO:tensorflow:loss = 0.023164492, step = 21701 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.598\n",
      "INFO:tensorflow:loss = 0.018999262, step = 21801 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.434\n",
      "INFO:tensorflow:loss = 0.024955176, step = 21901 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.292\n",
      "INFO:tensorflow:loss = 0.012451027, step = 22001 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.416\n",
      "INFO:tensorflow:loss = 0.030584168, step = 22101 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.766\n",
      "INFO:tensorflow:loss = 0.01878487, step = 22201 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.768\n",
      "INFO:tensorflow:loss = 0.01534641, step = 22301 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.313\n",
      "INFO:tensorflow:loss = 0.013484937, step = 22401 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.086\n",
      "INFO:tensorflow:loss = 0.025204843, step = 22501 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.23\n",
      "INFO:tensorflow:loss = 0.018113509, step = 22601 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.505\n",
      "INFO:tensorflow:loss = 0.021024233, step = 22701 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.843\n",
      "INFO:tensorflow:loss = 0.012189167, step = 22801 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.515\n",
      "INFO:tensorflow:loss = 0.018502358, step = 22901 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.736\n",
      "INFO:tensorflow:loss = 0.013431711, step = 23001 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.782\n",
      "INFO:tensorflow:loss = 0.010160147, step = 23101 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.158\n",
      "INFO:tensorflow:loss = 0.015025451, step = 23201 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.626\n",
      "INFO:tensorflow:loss = 0.02824898, step = 23301 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.671\n",
      "INFO:tensorflow:loss = 0.020847525, step = 23401 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.485\n",
      "INFO:tensorflow:loss = 0.035996947, step = 23501 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.203\n",
      "INFO:tensorflow:loss = 0.013584637, step = 23601 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.2\n",
      "INFO:tensorflow:loss = 0.012821469, step = 23701 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.827\n",
      "INFO:tensorflow:loss = 0.016141474, step = 23801 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.187\n",
      "INFO:tensorflow:loss = 0.012357437, step = 23901 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.377\n",
      "INFO:tensorflow:loss = 0.024248889, step = 24001 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.361\n",
      "INFO:tensorflow:loss = 0.016409412, step = 24101 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.753\n",
      "INFO:tensorflow:loss = 0.010683423, step = 24201 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.778\n",
      "INFO:tensorflow:loss = 0.01628223, step = 24301 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.965\n",
      "INFO:tensorflow:loss = 0.01637354, step = 24401 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.789\n",
      "INFO:tensorflow:loss = 0.01855785, step = 24501 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.704\n",
      "INFO:tensorflow:loss = 0.017513843, step = 24601 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.54\n",
      "INFO:tensorflow:loss = 0.028298842, step = 24701 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 260.297\n",
      "INFO:tensorflow:loss = 0.014887784, step = 24801 (0.382 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.024\n",
      "INFO:tensorflow:loss = 0.015499558, step = 24901 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.503\n",
      "INFO:tensorflow:loss = 0.009831697, step = 25001 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 362.411\n",
      "INFO:tensorflow:loss = 0.01123051, step = 25101 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.927\n",
      "INFO:tensorflow:loss = 0.0154354945, step = 25201 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.651\n",
      "INFO:tensorflow:loss = 0.02772168, step = 25301 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.988\n",
      "INFO:tensorflow:loss = 0.012476445, step = 25401 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 314.017\n",
      "INFO:tensorflow:loss = 0.010980725, step = 25501 (0.319 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.38\n",
      "INFO:tensorflow:loss = 0.023996085, step = 25601 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.905\n",
      "INFO:tensorflow:loss = 0.014931529, step = 25701 (0.299 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.679\n",
      "INFO:tensorflow:loss = 0.015225203, step = 25801 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.852\n",
      "INFO:tensorflow:loss = 0.012725477, step = 25901 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.227\n",
      "INFO:tensorflow:loss = 0.017309496, step = 26001 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.969\n",
      "INFO:tensorflow:loss = 0.022866834, step = 26101 (0.279 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 356.343\n",
      "INFO:tensorflow:loss = 0.021942817, step = 26201 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.273\n",
      "INFO:tensorflow:loss = 0.015928082, step = 26301 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.135\n",
      "INFO:tensorflow:loss = 0.028541323, step = 26401 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.565\n",
      "INFO:tensorflow:loss = 0.0317541, step = 26501 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.905\n",
      "INFO:tensorflow:loss = 0.01571558, step = 26601 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.229\n",
      "INFO:tensorflow:loss = 0.013409937, step = 26701 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.545\n",
      "INFO:tensorflow:loss = 0.0139035545, step = 26801 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.059\n",
      "INFO:tensorflow:loss = 0.008198794, step = 26901 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.527\n",
      "INFO:tensorflow:loss = 0.01690805, step = 27001 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.433\n",
      "INFO:tensorflow:loss = 0.011334573, step = 27101 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.664\n",
      "INFO:tensorflow:loss = 0.0121444445, step = 27201 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.024\n",
      "INFO:tensorflow:loss = 0.030798106, step = 27301 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.146\n",
      "INFO:tensorflow:loss = 0.01596788, step = 27401 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.36\n",
      "INFO:tensorflow:loss = 0.013270018, step = 27501 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.375\n",
      "INFO:tensorflow:loss = 0.010533983, step = 27601 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 434.682\n",
      "INFO:tensorflow:loss = 0.019549675, step = 27701 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.13\n",
      "INFO:tensorflow:loss = 0.017236963, step = 27801 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.225\n",
      "INFO:tensorflow:loss = 0.010631916, step = 27901 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.965\n",
      "INFO:tensorflow:loss = 0.018294312, step = 28001 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.841\n",
      "INFO:tensorflow:loss = 0.017294295, step = 28101 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.566\n",
      "INFO:tensorflow:loss = 0.024331944, step = 28201 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.278\n",
      "INFO:tensorflow:loss = 0.018043205, step = 28301 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 337.041\n",
      "INFO:tensorflow:loss = 0.019369828, step = 28401 (0.294 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.569\n",
      "INFO:tensorflow:loss = 0.011316432, step = 28501 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 310.895\n",
      "INFO:tensorflow:loss = 0.020100253, step = 28601 (0.322 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.626\n",
      "INFO:tensorflow:loss = 0.014176866, step = 28701 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.889\n",
      "INFO:tensorflow:loss = 0.024615161, step = 28801 (0.323 sec)\n",
      "INFO:tensorflow:global_step/sec: 309.443\n",
      "INFO:tensorflow:loss = 0.043754265, step = 28901 (0.317 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.903\n",
      "INFO:tensorflow:loss = 0.02670292, step = 29001 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.034\n",
      "INFO:tensorflow:loss = 0.015674828, step = 29101 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.026\n",
      "INFO:tensorflow:loss = 0.021145508, step = 29201 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.624\n",
      "INFO:tensorflow:loss = 0.009372098, step = 29301 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 443.378\n",
      "INFO:tensorflow:loss = 0.010765795, step = 29401 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.559\n",
      "INFO:tensorflow:loss = 0.027950631, step = 29501 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 310.062\n",
      "INFO:tensorflow:loss = 0.01830468, step = 29601 (0.327 sec)\n",
      "INFO:tensorflow:global_step/sec: 199.422\n",
      "INFO:tensorflow:loss = 0.010359384, step = 29701 (0.497 sec)\n",
      "INFO:tensorflow:global_step/sec: 271.036\n",
      "INFO:tensorflow:loss = 0.014122753, step = 29801 (0.369 sec)\n",
      "INFO:tensorflow:global_step/sec: 314.587\n",
      "INFO:tensorflow:loss = 0.019935787, step = 29901 (0.317 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.781\n",
      "INFO:tensorflow:loss = 0.02304878, step = 30001 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.314\n",
      "INFO:tensorflow:loss = 0.009860686, step = 30101 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.42\n",
      "INFO:tensorflow:loss = 0.024267951, step = 30201 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.443\n",
      "INFO:tensorflow:loss = 0.02381996, step = 30301 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.236\n",
      "INFO:tensorflow:loss = 0.01666069, step = 30401 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.019\n",
      "INFO:tensorflow:loss = 0.015865492, step = 30501 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.567\n",
      "INFO:tensorflow:loss = 0.010357593, step = 30601 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.399\n",
      "INFO:tensorflow:loss = 0.008685669, step = 30701 (0.299 sec)\n",
      "INFO:tensorflow:global_step/sec: 309.784\n",
      "INFO:tensorflow:loss = 0.0184724, step = 30801 (0.325 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.21\n",
      "INFO:tensorflow:loss = 0.021294821, step = 30901 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.338\n",
      "INFO:tensorflow:loss = 0.008166087, step = 31001 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.612\n",
      "INFO:tensorflow:loss = 0.017734066, step = 31101 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 439.613\n",
      "INFO:tensorflow:loss = 0.0104606915, step = 31201 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 363.688\n",
      "INFO:tensorflow:loss = 0.021444194, step = 31301 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 222.192\n",
      "INFO:tensorflow:loss = 0.008605653, step = 31401 (0.450 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.02\n",
      "INFO:tensorflow:loss = 0.02976946, step = 31501 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.086\n",
      "INFO:tensorflow:loss = 0.007972813, step = 31601 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.171\n",
      "INFO:tensorflow:loss = 0.01627874, step = 31701 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.429\n",
      "INFO:tensorflow:loss = 0.011192717, step = 31801 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 327.321\n",
      "INFO:tensorflow:loss = 0.02193925, step = 31901 (0.306 sec)\n",
      "INFO:tensorflow:global_step/sec: 299.496\n",
      "INFO:tensorflow:loss = 0.014254434, step = 32001 (0.334 sec)\n",
      "INFO:tensorflow:global_step/sec: 271.506\n",
      "INFO:tensorflow:loss = 0.0078085363, step = 32101 (0.371 sec)\n",
      "INFO:tensorflow:global_step/sec: 313.926\n",
      "INFO:tensorflow:loss = 0.008644865, step = 32201 (0.319 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.04\n",
      "INFO:tensorflow:loss = 0.015556526, step = 32301 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.567\n",
      "INFO:tensorflow:loss = 0.009916267, step = 32401 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 351.915\n",
      "INFO:tensorflow:loss = 0.01742769, step = 32501 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 355.231\n",
      "INFO:tensorflow:loss = 0.016402483, step = 32601 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 255.085\n",
      "INFO:tensorflow:loss = 0.016124513, step = 32701 (0.392 sec)\n",
      "INFO:tensorflow:global_step/sec: 207.236\n",
      "INFO:tensorflow:loss = 0.01674425, step = 32801 (0.484 sec)\n",
      "INFO:tensorflow:global_step/sec: 147.567\n",
      "INFO:tensorflow:loss = 0.01349519, step = 32901 (0.685 sec)\n",
      "INFO:tensorflow:global_step/sec: 208.221\n",
      "INFO:tensorflow:loss = 0.01785659, step = 33001 (0.472 sec)\n",
      "INFO:tensorflow:global_step/sec: 305.861\n",
      "INFO:tensorflow:loss = 0.014213873, step = 33101 (0.327 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.686\n",
      "INFO:tensorflow:loss = 0.011744956, step = 33201 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.786\n",
      "INFO:tensorflow:loss = 0.008218929, step = 33301 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 400.964\n",
      "INFO:tensorflow:loss = 0.017277919, step = 33401 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.57\n",
      "INFO:tensorflow:loss = 0.018505786, step = 33501 (0.300 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.142\n",
      "INFO:tensorflow:loss = 0.013220284, step = 33601 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.896\n",
      "INFO:tensorflow:loss = 0.008027487, step = 33701 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.732\n",
      "INFO:tensorflow:loss = 0.023950852, step = 33801 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.508\n",
      "INFO:tensorflow:loss = 0.013348222, step = 33901 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.808\n",
      "INFO:tensorflow:loss = 0.01093495, step = 34001 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.775\n",
      "INFO:tensorflow:loss = 0.014343632, step = 34101 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.685\n",
      "INFO:tensorflow:loss = 0.014524261, step = 34201 (0.253 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 368.627\n",
      "INFO:tensorflow:loss = 0.013047135, step = 34301 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.667\n",
      "INFO:tensorflow:loss = 0.011771475, step = 34401 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.095\n",
      "INFO:tensorflow:loss = 0.012609817, step = 34501 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 453.402\n",
      "INFO:tensorflow:loss = 0.021411832, step = 34601 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.492\n",
      "INFO:tensorflow:loss = 0.020396, step = 34701 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.78\n",
      "INFO:tensorflow:loss = 0.022956783, step = 34801 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 349.204\n",
      "INFO:tensorflow:loss = 0.022739397, step = 34901 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 303.931\n",
      "INFO:tensorflow:loss = 0.025612412, step = 35001 (0.332 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.001\n",
      "INFO:tensorflow:loss = 0.012663094, step = 35101 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.752\n",
      "INFO:tensorflow:loss = 0.013372776, step = 35201 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.397\n",
      "INFO:tensorflow:loss = 0.022115503, step = 35301 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.729\n",
      "INFO:tensorflow:loss = 0.00788129, step = 35401 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 321.495\n",
      "INFO:tensorflow:loss = 0.019508958, step = 35501 (0.312 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.431\n",
      "INFO:tensorflow:loss = 0.017536202, step = 35601 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.22\n",
      "INFO:tensorflow:loss = 0.020626929, step = 35701 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.062\n",
      "INFO:tensorflow:loss = 0.022569638, step = 35801 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 304.841\n",
      "INFO:tensorflow:loss = 0.021247275, step = 35901 (0.333 sec)\n",
      "INFO:tensorflow:global_step/sec: 346.18\n",
      "INFO:tensorflow:loss = 0.015491639, step = 36001 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 344.597\n",
      "INFO:tensorflow:loss = 0.015590073, step = 36101 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 349.238\n",
      "INFO:tensorflow:loss = 0.009045677, step = 36201 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.275\n",
      "INFO:tensorflow:loss = 0.01685698, step = 36301 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.257\n",
      "INFO:tensorflow:loss = 0.019577648, step = 36401 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.628\n",
      "INFO:tensorflow:loss = 0.010640087, step = 36501 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 447.235\n",
      "INFO:tensorflow:loss = 0.01636697, step = 36601 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.277\n",
      "INFO:tensorflow:loss = 0.014891412, step = 36701 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.542\n",
      "INFO:tensorflow:loss = 0.013473487, step = 36801 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.892\n",
      "INFO:tensorflow:loss = 0.01046416, step = 36901 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.526\n",
      "INFO:tensorflow:loss = 0.021578131, step = 37001 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.394\n",
      "INFO:tensorflow:loss = 0.019680705, step = 37101 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 428.259\n",
      "INFO:tensorflow:loss = 0.010610458, step = 37201 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.392\n",
      "INFO:tensorflow:loss = 0.016316906, step = 37301 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 369.289\n",
      "INFO:tensorflow:loss = 0.009061607, step = 37401 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.041\n",
      "INFO:tensorflow:loss = 0.01523049, step = 37501 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.295\n",
      "INFO:tensorflow:loss = 0.016769737, step = 37601 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.109\n",
      "INFO:tensorflow:loss = 0.013719063, step = 37701 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.077\n",
      "INFO:tensorflow:loss = 0.013885902, step = 37801 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 313.497\n",
      "INFO:tensorflow:loss = 0.013337968, step = 37901 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.943\n",
      "INFO:tensorflow:loss = 0.01748215, step = 38001 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.152\n",
      "INFO:tensorflow:loss = 0.01752853, step = 38101 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.4\n",
      "INFO:tensorflow:loss = 0.018100314, step = 38201 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.629\n",
      "INFO:tensorflow:loss = 0.023500139, step = 38301 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.425\n",
      "INFO:tensorflow:loss = 0.024213504, step = 38401 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 340.207\n",
      "INFO:tensorflow:loss = 0.011179834, step = 38501 (0.294 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.073\n",
      "INFO:tensorflow:loss = 0.015121329, step = 38601 (0.317 sec)\n",
      "INFO:tensorflow:global_step/sec: 287.632\n",
      "INFO:tensorflow:loss = 0.016238084, step = 38701 (0.349 sec)\n",
      "INFO:tensorflow:global_step/sec: 238.941\n",
      "INFO:tensorflow:loss = 0.0121732075, step = 38801 (0.420 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.9\n",
      "INFO:tensorflow:loss = 0.018280178, step = 38901 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.166\n",
      "INFO:tensorflow:loss = 0.012122821, step = 39001 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 349.063\n",
      "INFO:tensorflow:loss = 0.018404687, step = 39101 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 369.14\n",
      "INFO:tensorflow:loss = 0.015240444, step = 39201 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.656\n",
      "INFO:tensorflow:loss = 0.013075953, step = 39301 (0.294 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.095\n",
      "INFO:tensorflow:loss = 0.01575252, step = 39401 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.316\n",
      "INFO:tensorflow:loss = 0.011827726, step = 39501 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.414\n",
      "INFO:tensorflow:loss = 0.011090815, step = 39601 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 347.163\n",
      "INFO:tensorflow:loss = 0.008469071, step = 39701 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.79\n",
      "INFO:tensorflow:loss = 0.013101498, step = 39801 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.024\n",
      "INFO:tensorflow:loss = 0.021683915, step = 39901 (0.272 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 40000 into /tmp/tmpsbdccn23/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-07:34:11\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-40000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'previous_ensemble' dict for global step 40000: architecture/adanet/ensembles = b\"\\nR\\n,adanet/previous_ensemble/architecture/adanetB\\x18\\x08\\x07\\x12\\x00B\\x12| b'1_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.040044222, average_loss/adanet/subnetwork = 0.039825242, average_loss/adanet/uniform_average_ensemble = 0.039825242, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.05366586, loss/adanet/subnetwork = 0.053258948, loss/adanet/uniform_average_ensemble = 0.053258948, prediction/mean/adanet/adanet_weighted_ensemble = 3.1599553, prediction/mean/adanet/subnetwork = 3.157783, prediction/mean/adanet/uniform_average_ensemble = 3.157783\n",
      "INFO:tensorflow:Saving candidate '1_layer_dnn' dict for global step 40000: architecture/adanet/ensembles = b\"\\nr\\n;adanet/iteration_1/ensemble_1_layer_dnn/architecture/adanetB)\\x08\\x07\\x12\\x00B#| b'1_layer_dnn' | b'1_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.03763339, average_loss/adanet/subnetwork = 0.04407932, average_loss/adanet/uniform_average_ensemble = 0.041038644, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.047553353, loss/adanet/subnetwork = 0.061847888, loss/adanet/uniform_average_ensemble = 0.05667209, prediction/mean/adanet/adanet_weighted_ensemble = 3.1071606, prediction/mean/adanet/subnetwork = 3.1570365, prediction/mean/adanet/uniform_average_ensemble = 3.1574097\n",
      "INFO:tensorflow:Saving candidate '2_layer_dnn' dict for global step 40000: architecture/adanet/ensembles = b\"\\nr\\n;adanet/iteration_1/ensemble_2_layer_dnn/architecture/adanetB)\\x08\\x07\\x12\\x00B#| b'1_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.034577098, average_loss/adanet/subnetwork = 0.03454547, average_loss/adanet/uniform_average_ensemble = 0.03574329, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.044068314, loss/adanet/subnetwork = 0.04466034, loss/adanet/uniform_average_ensemble = 0.04744883, prediction/mean/adanet/adanet_weighted_ensemble = 3.1163297, prediction/mean/adanet/subnetwork = 3.1383395, prediction/mean/adanet/uniform_average_ensemble = 3.148061\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Finished evaluation at 2018-11-19-07:34:14\n",
      "INFO:tensorflow:Saving dict for global step 40000: average_loss = 0.034577098, average_loss/adanet/adanet_weighted_ensemble = 0.034577098, average_loss/adanet/subnetwork = 0.03454547, average_loss/adanet/uniform_average_ensemble = 0.03574329, global_step = 40000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.044068314, loss/adanet/adanet_weighted_ensemble = 0.044068314, loss/adanet/subnetwork = 0.04466034, loss/adanet/uniform_average_ensemble = 0.04744883, prediction/mean = 3.1163297, prediction/mean/adanet/adanet_weighted_ensemble = 3.1163297, prediction/mean/adanet/subnetwork = 3.1383395, prediction/mean/adanet/uniform_average_ensemble = 3.148061\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 40000: /tmp/tmpsbdccn23/model.ckpt-40000\n",
      "INFO:tensorflow:Loss for final step: 0.012424957.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Starting ensemble evaluation for iteration 1\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-40000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Encountered end of input after 14 evaluations\n",
      "INFO:tensorflow:Computed ensemble metrics: adanet_loss/previous_ensemble = 0.021031, adanet_loss/1_layer_dnn = 0.017057, adanet_loss/2_layer_dnn = 0.013947\n",
      "INFO:tensorflow:Finished ensemble evaluation for iteration 1\n",
      "INFO:tensorflow:'2_layer_dnn' at index 2 is moving onto the next iteration\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Freezing best ensemble to /tmp/tmpsbdccn23/frozen/ensemble-1.meta\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-40000\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Overwriting checkpoint with new graph for iteration 2 to /tmp/tmpsbdccn23/model.ckpt-40000\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-40000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Finished training Adanet iteration 1\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 2\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/increment.ckpt-2\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 40000 into /tmp/tmpsbdccn23/model.ckpt.\n",
      "INFO:tensorflow:loss = 0.014832886, step = 40001\n",
      "INFO:tensorflow:global_step/sec: 114.746\n",
      "INFO:tensorflow:loss = 0.01829448, step = 40101 (0.873 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.426\n",
      "INFO:tensorflow:loss = 0.014921835, step = 40201 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.529\n",
      "INFO:tensorflow:loss = 0.012808114, step = 40301 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.58\n",
      "INFO:tensorflow:loss = 0.017732274, step = 40401 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 440.235\n",
      "INFO:tensorflow:loss = 0.011289857, step = 40501 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.597\n",
      "INFO:tensorflow:loss = 0.012631366, step = 40601 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 436.879\n",
      "INFO:tensorflow:loss = 0.008941069, step = 40701 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.932\n",
      "INFO:tensorflow:loss = 0.01171897, step = 40801 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.934\n",
      "INFO:tensorflow:loss = 0.013929779, step = 40901 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.787\n",
      "INFO:tensorflow:loss = 0.020013342, step = 41001 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.7\n",
      "INFO:tensorflow:loss = 0.014787745, step = 41101 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.254\n",
      "INFO:tensorflow:loss = 0.019948637, step = 41201 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 439.762\n",
      "INFO:tensorflow:loss = 0.01731324, step = 41301 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.041\n",
      "INFO:tensorflow:loss = 0.0077695516, step = 41401 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.208\n",
      "INFO:tensorflow:loss = 0.007187483, step = 41501 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.278\n",
      "INFO:tensorflow:loss = 0.010535212, step = 41601 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.801\n",
      "INFO:tensorflow:loss = 0.014927811, step = 41701 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.527\n",
      "INFO:tensorflow:loss = 0.012406211, step = 41801 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 441.614\n",
      "INFO:tensorflow:loss = 0.016569352, step = 41901 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.523\n",
      "INFO:tensorflow:loss = 0.00920915, step = 42001 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.587\n",
      "INFO:tensorflow:loss = 0.021074543, step = 42101 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 347.571\n",
      "INFO:tensorflow:loss = 0.014027754, step = 42201 (0.288 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.979\n",
      "INFO:tensorflow:loss = 0.0130133405, step = 42301 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.903\n",
      "INFO:tensorflow:loss = 0.01040647, step = 42401 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.997\n",
      "INFO:tensorflow:loss = 0.016617848, step = 42501 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 362.433\n",
      "INFO:tensorflow:loss = 0.010316135, step = 42601 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.204\n",
      "INFO:tensorflow:loss = 0.014691562, step = 42701 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.902\n",
      "INFO:tensorflow:loss = 0.010456921, step = 42801 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 437.953\n",
      "INFO:tensorflow:loss = 0.010401211, step = 42901 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.753\n",
      "INFO:tensorflow:loss = 0.010302524, step = 43001 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.183\n",
      "INFO:tensorflow:loss = 0.007739976, step = 43101 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.242\n",
      "INFO:tensorflow:loss = 0.01565973, step = 43201 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.774\n",
      "INFO:tensorflow:loss = 0.022553444, step = 43301 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.631\n",
      "INFO:tensorflow:loss = 0.015012393, step = 43401 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.615\n",
      "INFO:tensorflow:loss = 0.02511124, step = 43501 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.343\n",
      "INFO:tensorflow:loss = 0.008861767, step = 43601 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.342\n",
      "INFO:tensorflow:loss = 0.012034379, step = 43701 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.36\n",
      "INFO:tensorflow:loss = 0.011844678, step = 43801 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 441.809\n",
      "INFO:tensorflow:loss = 0.008281194, step = 43901 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.099\n",
      "INFO:tensorflow:loss = 0.015554931, step = 44001 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.247\n",
      "INFO:tensorflow:loss = 0.013009555, step = 44101 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.993\n",
      "INFO:tensorflow:loss = 0.008968855, step = 44201 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.831\n",
      "INFO:tensorflow:loss = 0.00953502, step = 44301 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.526\n",
      "INFO:tensorflow:loss = 0.010429392, step = 44401 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.699\n",
      "INFO:tensorflow:loss = 0.010482366, step = 44501 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.101\n",
      "INFO:tensorflow:loss = 0.013949459, step = 44601 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.482\n",
      "INFO:tensorflow:loss = 0.012443367, step = 44701 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.853\n",
      "INFO:tensorflow:loss = 0.009748522, step = 44801 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.757\n",
      "INFO:tensorflow:loss = 0.01382673, step = 44901 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.936\n",
      "INFO:tensorflow:loss = 0.00706046, step = 45001 (0.282 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 335.553\n",
      "INFO:tensorflow:loss = 0.009013201, step = 45101 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.226\n",
      "INFO:tensorflow:loss = 0.012831824, step = 45201 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.237\n",
      "INFO:tensorflow:loss = 0.019404039, step = 45301 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.002\n",
      "INFO:tensorflow:loss = 0.010275149, step = 45401 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 345.575\n",
      "INFO:tensorflow:loss = 0.008305512, step = 45501 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.571\n",
      "INFO:tensorflow:loss = 0.016967151, step = 45601 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.589\n",
      "INFO:tensorflow:loss = 0.010462762, step = 45701 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 428.88\n",
      "INFO:tensorflow:loss = 0.010056531, step = 45801 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.849\n",
      "INFO:tensorflow:loss = 0.010550333, step = 45901 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 300.686\n",
      "INFO:tensorflow:loss = 0.010025362, step = 46001 (0.336 sec)\n",
      "INFO:tensorflow:global_step/sec: 345.68\n",
      "INFO:tensorflow:loss = 0.017599348, step = 46101 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.46\n",
      "INFO:tensorflow:loss = 0.013904706, step = 46201 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.543\n",
      "INFO:tensorflow:loss = 0.010277975, step = 46301 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.767\n",
      "INFO:tensorflow:loss = 0.014407231, step = 46401 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 450.248\n",
      "INFO:tensorflow:loss = 0.015635742, step = 46501 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.894\n",
      "INFO:tensorflow:loss = 0.010363281, step = 46601 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.733\n",
      "INFO:tensorflow:loss = 0.011994712, step = 46701 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.047\n",
      "INFO:tensorflow:loss = 0.011747437, step = 46801 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.316\n",
      "INFO:tensorflow:loss = 0.008638218, step = 46901 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.987\n",
      "INFO:tensorflow:loss = 0.012716239, step = 47001 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.683\n",
      "INFO:tensorflow:loss = 0.0075392555, step = 47101 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.552\n",
      "INFO:tensorflow:loss = 0.0109973885, step = 47201 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.18\n",
      "INFO:tensorflow:loss = 0.016964935, step = 47301 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.109\n",
      "INFO:tensorflow:loss = 0.010867781, step = 47401 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 444.788\n",
      "INFO:tensorflow:loss = 0.007910306, step = 47501 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.847\n",
      "INFO:tensorflow:loss = 0.011069623, step = 47601 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 433.046\n",
      "INFO:tensorflow:loss = 0.020190854, step = 47701 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.742\n",
      "INFO:tensorflow:loss = 0.016452719, step = 47801 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 449.751\n",
      "INFO:tensorflow:loss = 0.0073835654, step = 47901 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 328.635\n",
      "INFO:tensorflow:loss = 0.015220561, step = 48001 (0.305 sec)\n",
      "INFO:tensorflow:global_step/sec: 445.055\n",
      "INFO:tensorflow:loss = 0.016045125, step = 48101 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 337.266\n",
      "INFO:tensorflow:loss = 0.01820409, step = 48201 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.749\n",
      "INFO:tensorflow:loss = 0.0143665, step = 48301 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.269\n",
      "INFO:tensorflow:loss = 0.009493674, step = 48401 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 444.608\n",
      "INFO:tensorflow:loss = 0.007580799, step = 48501 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.427\n",
      "INFO:tensorflow:loss = 0.011235774, step = 48601 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.494\n",
      "INFO:tensorflow:loss = 0.01343517, step = 48701 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 369.536\n",
      "INFO:tensorflow:loss = 0.018923119, step = 48801 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.471\n",
      "INFO:tensorflow:loss = 0.03216851, step = 48901 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.7\n",
      "INFO:tensorflow:loss = 0.020711366, step = 49001 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.435\n",
      "INFO:tensorflow:loss = 0.013104171, step = 49101 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.462\n",
      "INFO:tensorflow:loss = 0.012799671, step = 49201 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.279\n",
      "INFO:tensorflow:loss = 0.007343926, step = 49301 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.971\n",
      "INFO:tensorflow:loss = 0.010295708, step = 49401 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.881\n",
      "INFO:tensorflow:loss = 0.017586607, step = 49501 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.638\n",
      "INFO:tensorflow:loss = 0.009988839, step = 49601 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.09\n",
      "INFO:tensorflow:loss = 0.007452753, step = 49701 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.431\n",
      "INFO:tensorflow:loss = 0.01277627, step = 49801 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.2\n",
      "INFO:tensorflow:loss = 0.012888042, step = 49901 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.315\n",
      "INFO:tensorflow:loss = 0.018000335, step = 50001 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.967\n",
      "INFO:tensorflow:loss = 0.0065920707, step = 50101 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.114\n",
      "INFO:tensorflow:loss = 0.014588508, step = 50201 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.428\n",
      "INFO:tensorflow:loss = 0.01564922, step = 50301 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.952\n",
      "INFO:tensorflow:loss = 0.010311159, step = 50401 (0.290 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.843\n",
      "INFO:tensorflow:loss = 0.012255011, step = 50501 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.854\n",
      "INFO:tensorflow:loss = 0.008234791, step = 50601 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.062\n",
      "INFO:tensorflow:loss = 0.0077810716, step = 50701 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.147\n",
      "INFO:tensorflow:loss = 0.015871625, step = 50801 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.122\n",
      "INFO:tensorflow:loss = 0.018112477, step = 50901 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.392\n",
      "INFO:tensorflow:loss = 0.008799363, step = 51001 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.657\n",
      "INFO:tensorflow:loss = 0.014347598, step = 51101 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 441.512\n",
      "INFO:tensorflow:loss = 0.0071265157, step = 51201 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.091\n",
      "INFO:tensorflow:loss = 0.0126987565, step = 51301 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.766\n",
      "INFO:tensorflow:loss = 0.0069261747, step = 51401 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.881\n",
      "INFO:tensorflow:loss = 0.023216171, step = 51501 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.683\n",
      "INFO:tensorflow:loss = 0.0061972993, step = 51601 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 339.69\n",
      "INFO:tensorflow:loss = 0.012542128, step = 51701 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.378\n",
      "INFO:tensorflow:loss = 0.0076074554, step = 51801 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.717\n",
      "INFO:tensorflow:loss = 0.011658053, step = 51901 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 439.042\n",
      "INFO:tensorflow:loss = 0.010644754, step = 52001 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.733\n",
      "INFO:tensorflow:loss = 0.0065492284, step = 52101 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.698\n",
      "INFO:tensorflow:loss = 0.007355309, step = 52201 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.281\n",
      "INFO:tensorflow:loss = 0.012876192, step = 52301 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.987\n",
      "INFO:tensorflow:loss = 0.0075557916, step = 52401 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.036\n",
      "INFO:tensorflow:loss = 0.014948637, step = 52501 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 439.749\n",
      "INFO:tensorflow:loss = 0.0104660895, step = 52601 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.725\n",
      "INFO:tensorflow:loss = 0.015582761, step = 52701 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 448.185\n",
      "INFO:tensorflow:loss = 0.013980483, step = 52801 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.558\n",
      "INFO:tensorflow:loss = 0.011834176, step = 52901 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.357\n",
      "INFO:tensorflow:loss = 0.01793776, step = 53001 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.491\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.0075753564, step = 53101 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 443.121\n",
      "INFO:tensorflow:loss = 0.008639535, step = 53201 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.811\n",
      "INFO:tensorflow:loss = 0.0059426376, step = 53301 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 436.38\n",
      "INFO:tensorflow:loss = 0.009628052, step = 53401 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 369.474\n",
      "INFO:tensorflow:loss = 0.009919619, step = 53501 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 442.452\n",
      "INFO:tensorflow:loss = 0.010228084, step = 53601 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.581\n",
      "INFO:tensorflow:loss = 0.0070839766, step = 53701 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.726\n",
      "INFO:tensorflow:loss = 0.013984772, step = 53801 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.864\n",
      "INFO:tensorflow:loss = 0.009667996, step = 53901 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.66\n",
      "INFO:tensorflow:loss = 0.0065857605, step = 54001 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.871\n",
      "INFO:tensorflow:loss = 0.012894244, step = 54101 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.335\n",
      "INFO:tensorflow:loss = 0.011329085, step = 54201 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.554\n",
      "INFO:tensorflow:loss = 0.008055725, step = 54301 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 440.383\n",
      "INFO:tensorflow:loss = 0.008337489, step = 54401 (0.228 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.33\n",
      "INFO:tensorflow:loss = 0.008150151, step = 54501 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.747\n",
      "INFO:tensorflow:loss = 0.01582564, step = 54601 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.535\n",
      "INFO:tensorflow:loss = 0.017359605, step = 54701 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.782\n",
      "INFO:tensorflow:loss = 0.018918417, step = 54801 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.343\n",
      "INFO:tensorflow:loss = 0.008644354, step = 54901 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.576\n",
      "INFO:tensorflow:loss = 0.019323938, step = 55001 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.331\n",
      "INFO:tensorflow:loss = 0.008325594, step = 55101 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 439.888\n",
      "INFO:tensorflow:loss = 0.010735119, step = 55201 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.475\n",
      "INFO:tensorflow:loss = 0.016199987, step = 55301 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.116\n",
      "INFO:tensorflow:loss = 0.00715934, step = 55401 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.429\n",
      "INFO:tensorflow:loss = 0.015292912, step = 55501 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.498\n",
      "INFO:tensorflow:loss = 0.015136609, step = 55601 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.196\n",
      "INFO:tensorflow:loss = 0.009289202, step = 55701 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.261\n",
      "INFO:tensorflow:loss = 0.01513825, step = 55801 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.064\n",
      "INFO:tensorflow:loss = 0.008155204, step = 55901 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.912\n",
      "INFO:tensorflow:loss = 0.008829491, step = 56001 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.092\n",
      "INFO:tensorflow:loss = 0.0074831126, step = 56101 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.66\n",
      "INFO:tensorflow:loss = 0.007206453, step = 56201 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.079\n",
      "INFO:tensorflow:loss = 0.009848906, step = 56301 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.193\n",
      "INFO:tensorflow:loss = 0.007322054, step = 56401 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.461\n",
      "INFO:tensorflow:loss = 0.006280196, step = 56501 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 363.11\n",
      "INFO:tensorflow:loss = 0.01000681, step = 56601 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.707\n",
      "INFO:tensorflow:loss = 0.009717852, step = 56701 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.063\n",
      "INFO:tensorflow:loss = 0.007942522, step = 56801 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.624\n",
      "INFO:tensorflow:loss = 0.0065999324, step = 56901 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.591\n",
      "INFO:tensorflow:loss = 0.015195541, step = 57001 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.092\n",
      "INFO:tensorflow:loss = 0.013276763, step = 57101 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 299.541\n",
      "INFO:tensorflow:loss = 0.006648481, step = 57201 (0.334 sec)\n",
      "INFO:tensorflow:global_step/sec: 441.535\n",
      "INFO:tensorflow:loss = 0.00914113, step = 57301 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.387\n",
      "INFO:tensorflow:loss = 0.0055382145, step = 57401 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.849\n",
      "INFO:tensorflow:loss = 0.011563813, step = 57501 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.306\n",
      "INFO:tensorflow:loss = 0.008670833, step = 57601 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.765\n",
      "INFO:tensorflow:loss = 0.007814769, step = 57701 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.282\n",
      "INFO:tensorflow:loss = 0.010945664, step = 57801 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.153\n",
      "INFO:tensorflow:loss = 0.008887624, step = 57901 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.971\n",
      "INFO:tensorflow:loss = 0.00947511, step = 58001 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.517\n",
      "INFO:tensorflow:loss = 0.013491194, step = 58101 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 351.358\n",
      "INFO:tensorflow:loss = 0.010712771, step = 58201 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 518.108\n",
      "INFO:tensorflow:loss = 0.01708028, step = 58301 (0.191 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.214\n",
      "INFO:tensorflow:loss = 0.012890382, step = 58401 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.392\n",
      "INFO:tensorflow:loss = 0.010849256, step = 58501 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.183\n",
      "INFO:tensorflow:loss = 0.010764739, step = 58601 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.713\n",
      "INFO:tensorflow:loss = 0.007417483, step = 58701 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 351.272\n",
      "INFO:tensorflow:loss = 0.008777965, step = 58801 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 428.121\n",
      "INFO:tensorflow:loss = 0.011306363, step = 58901 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.985\n",
      "INFO:tensorflow:loss = 0.009028367, step = 59001 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.822\n",
      "INFO:tensorflow:loss = 0.011517867, step = 59101 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.551\n",
      "INFO:tensorflow:loss = 0.008101302, step = 59201 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.289\n",
      "INFO:tensorflow:loss = 0.007651964, step = 59301 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.375\n",
      "INFO:tensorflow:loss = 0.0077045253, step = 59401 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.669\n",
      "INFO:tensorflow:loss = 0.008584835, step = 59501 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.43\n",
      "INFO:tensorflow:loss = 0.0082714, step = 59601 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.781\n",
      "INFO:tensorflow:loss = 0.0062246136, step = 59701 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.315\n",
      "INFO:tensorflow:loss = 0.009631314, step = 59801 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.813\n",
      "INFO:tensorflow:loss = 0.014084325, step = 59901 (0.238 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 60000 into /tmp/tmpsbdccn23/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-07:35:20\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-60000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'previous_ensemble' dict for global step 60000: architecture/adanet/ensembles = b\"\\nc\\n,adanet/previous_ensemble/architecture/adanetB)\\x08\\x07\\x12\\x00B#| b'1_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.034577098, average_loss/adanet/subnetwork = 0.03454547, average_loss/adanet/uniform_average_ensemble = 0.03574329, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.044068314, loss/adanet/subnetwork = 0.04466034, loss/adanet/uniform_average_ensemble = 0.04744883, prediction/mean/adanet/adanet_weighted_ensemble = 3.1163297, prediction/mean/adanet/subnetwork = 3.1383395, prediction/mean/adanet/uniform_average_ensemble = 3.148061\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving candidate '2_layer_dnn' dict for global step 60000: architecture/adanet/ensembles = b\"\\n\\x83\\x01\\n;adanet/iteration_2/ensemble_2_layer_dnn/architecture/adanetB:\\x08\\x07\\x12\\x00B4| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.032477677, average_loss/adanet/subnetwork = 0.035940513, average_loss/adanet/uniform_average_ensemble = 0.033833675, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.038081136, loss/adanet/subnetwork = 0.038228974, loss/adanet/uniform_average_ensemble = 0.042276174, prediction/mean/adanet/adanet_weighted_ensemble = 3.1002867, prediction/mean/adanet/subnetwork = 3.1267877, prediction/mean/adanet/uniform_average_ensemble = 3.14097\n",
      "INFO:tensorflow:Saving candidate '3_layer_dnn' dict for global step 60000: architecture/adanet/ensembles = b\"\\n\\x83\\x01\\n;adanet/iteration_2/ensemble_3_layer_dnn/architecture/adanetB:\\x08\\x07\\x12\\x00B4| b'1_layer_dnn' | b'2_layer_dnn' | b'3_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.032702476, average_loss/adanet/subnetwork = 0.03961562, average_loss/adanet/uniform_average_ensemble = 0.034995347, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.04397107, loss/adanet/subnetwork = 0.060717084, loss/adanet/uniform_average_ensemble = 0.04941713, prediction/mean/adanet/adanet_weighted_ensemble = 3.1151874, prediction/mean/adanet/subnetwork = 3.1554945, prediction/mean/adanet/uniform_average_ensemble = 3.1505394\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-19-07:35:22\n",
      "INFO:tensorflow:Saving dict for global step 60000: average_loss = 0.032702476, average_loss/adanet/adanet_weighted_ensemble = 0.032702476, average_loss/adanet/subnetwork = 0.03961562, average_loss/adanet/uniform_average_ensemble = 0.034995347, global_step = 60000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.04397107, loss/adanet/adanet_weighted_ensemble = 0.04397107, loss/adanet/subnetwork = 0.060717084, loss/adanet/uniform_average_ensemble = 0.04941713, prediction/mean = 3.1151874, prediction/mean/adanet/adanet_weighted_ensemble = 3.1151874, prediction/mean/adanet/subnetwork = 3.1554945, prediction/mean/adanet/uniform_average_ensemble = 3.1505394\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 60000: /tmp/tmpsbdccn23/model.ckpt-60000\n",
      "INFO:tensorflow:Loss for final step: 0.0070643816.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Starting ensemble evaluation for iteration 2\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-60000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Encountered end of input after 14 evaluations\n",
      "INFO:tensorflow:Computed ensemble metrics: adanet_loss/previous_ensemble = 0.013947, adanet_loss/2_layer_dnn = 0.009538, adanet_loss/3_layer_dnn = 0.009813\n",
      "INFO:tensorflow:Finished ensemble evaluation for iteration 2\n",
      "INFO:tensorflow:'2_layer_dnn' at index 1 is moving onto the next iteration\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Freezing best ensemble to /tmp/tmpsbdccn23/frozen/ensemble-2.meta\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-60000\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Overwriting checkpoint with new graph for iteration 3 to /tmp/tmpsbdccn23/model.ckpt-60000\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-60000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Finished training Adanet iteration 2\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 3\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/increment.ckpt-3\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 60000 into /tmp/tmpsbdccn23/model.ckpt.\n",
      "INFO:tensorflow:loss = 0.008954897, step = 60001\n",
      "INFO:tensorflow:global_step/sec: 92.828\n",
      "INFO:tensorflow:loss = 0.012422267, step = 60101 (1.079 sec)\n",
      "INFO:tensorflow:global_step/sec: 265.651\n",
      "INFO:tensorflow:loss = 0.011811845, step = 60201 (0.376 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.205\n",
      "INFO:tensorflow:loss = 0.0085447095, step = 60301 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 312.425\n",
      "INFO:tensorflow:loss = 0.010566365, step = 60401 (0.328 sec)\n",
      "INFO:tensorflow:global_step/sec: 325.778\n",
      "INFO:tensorflow:loss = 0.007907722, step = 60501 (0.299 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.635\n",
      "INFO:tensorflow:loss = 0.008076945, step = 60601 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.106\n",
      "INFO:tensorflow:loss = 0.008076125, step = 60701 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.559\n",
      "INFO:tensorflow:loss = 0.00854122, step = 60801 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 290.819\n",
      "INFO:tensorflow:loss = 0.013604099, step = 60901 (0.346 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.832\n",
      "INFO:tensorflow:loss = 0.013849139, step = 61001 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.863\n",
      "INFO:tensorflow:loss = 0.009620787, step = 61101 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.977\n",
      "INFO:tensorflow:loss = 0.014966475, step = 61201 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.46\n",
      "INFO:tensorflow:loss = 0.008546752, step = 61301 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.128\n",
      "INFO:tensorflow:loss = 0.0040911045, step = 61401 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 363.178\n",
      "INFO:tensorflow:loss = 0.0056746555, step = 61501 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.413\n",
      "INFO:tensorflow:loss = 0.0071111997, step = 61601 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.105\n",
      "INFO:tensorflow:loss = 0.010427438, step = 61701 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.461\n",
      "INFO:tensorflow:loss = 0.008231003, step = 61801 (0.303 sec)\n",
      "INFO:tensorflow:global_step/sec: 319.386\n",
      "INFO:tensorflow:loss = 0.011897422, step = 61901 (0.317 sec)\n",
      "INFO:tensorflow:global_step/sec: 307.151\n",
      "INFO:tensorflow:loss = 0.00823301, step = 62001 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.473\n",
      "INFO:tensorflow:loss = 0.014546139, step = 62101 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.053\n",
      "INFO:tensorflow:loss = 0.009515283, step = 62201 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.31\n",
      "INFO:tensorflow:loss = 0.010678613, step = 62301 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.045\n",
      "INFO:tensorflow:loss = 0.0074213683, step = 62401 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.74\n",
      "INFO:tensorflow:loss = 0.009495454, step = 62501 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.813\n",
      "INFO:tensorflow:loss = 0.005697895, step = 62601 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.169\n",
      "INFO:tensorflow:loss = 0.010163821, step = 62701 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.117\n",
      "INFO:tensorflow:loss = 0.0076353406, step = 62801 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.448\n",
      "INFO:tensorflow:loss = 0.0068241376, step = 62901 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.553\n",
      "INFO:tensorflow:loss = 0.00697405, step = 63001 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.608\n",
      "INFO:tensorflow:loss = 0.0059626615, step = 63101 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.933\n",
      "INFO:tensorflow:loss = 0.009790217, step = 63201 (0.263 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 344.91\n",
      "INFO:tensorflow:loss = 0.015341707, step = 63301 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.371\n",
      "INFO:tensorflow:loss = 0.011125417, step = 63401 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.791\n",
      "INFO:tensorflow:loss = 0.015757501, step = 63501 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.056\n",
      "INFO:tensorflow:loss = 0.0055276924, step = 63601 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.476\n",
      "INFO:tensorflow:loss = 0.0087307785, step = 63701 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.094\n",
      "INFO:tensorflow:loss = 0.007943563, step = 63801 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.001\n",
      "INFO:tensorflow:loss = 0.0052700844, step = 63901 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.688\n",
      "INFO:tensorflow:loss = 0.009482314, step = 64001 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.242\n",
      "INFO:tensorflow:loss = 0.008546544, step = 64101 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.529\n",
      "INFO:tensorflow:loss = 0.0058548674, step = 64201 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.634\n",
      "INFO:tensorflow:loss = 0.005681091, step = 64301 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.87\n",
      "INFO:tensorflow:loss = 0.0062552574, step = 64401 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.78\n",
      "INFO:tensorflow:loss = 0.007959742, step = 64501 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.676\n",
      "INFO:tensorflow:loss = 0.010111723, step = 64601 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.391\n",
      "INFO:tensorflow:loss = 0.0070217606, step = 64701 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.668\n",
      "INFO:tensorflow:loss = 0.006805526, step = 64801 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.283\n",
      "INFO:tensorflow:loss = 0.008696163, step = 64901 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.122\n",
      "INFO:tensorflow:loss = 0.0048346533, step = 65001 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.722\n",
      "INFO:tensorflow:loss = 0.005634658, step = 65101 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 451.6\n",
      "INFO:tensorflow:loss = 0.009131726, step = 65201 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 346.227\n",
      "INFO:tensorflow:loss = 0.011486806, step = 65301 (0.287 sec)\n",
      "INFO:tensorflow:global_step/sec: 447.51\n",
      "INFO:tensorflow:loss = 0.007819829, step = 65401 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.197\n",
      "INFO:tensorflow:loss = 0.0051940074, step = 65501 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.06\n",
      "INFO:tensorflow:loss = 0.01065613, step = 65601 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.752\n",
      "INFO:tensorflow:loss = 0.007604518, step = 65701 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.72\n",
      "INFO:tensorflow:loss = 0.0067437836, step = 65801 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.564\n",
      "INFO:tensorflow:loss = 0.008424299, step = 65901 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 279.009\n",
      "INFO:tensorflow:loss = 0.0073489696, step = 66001 (0.362 sec)\n",
      "INFO:tensorflow:global_step/sec: 291.516\n",
      "INFO:tensorflow:loss = 0.009656365, step = 66101 (0.341 sec)\n",
      "INFO:tensorflow:global_step/sec: 129.838\n",
      "INFO:tensorflow:loss = 0.010571654, step = 66201 (0.769 sec)\n",
      "INFO:tensorflow:global_step/sec: 239.589\n",
      "INFO:tensorflow:loss = 0.005911069, step = 66301 (0.417 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.689\n",
      "INFO:tensorflow:loss = 0.008480383, step = 66401 (0.317 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.83\n",
      "INFO:tensorflow:loss = 0.009532223, step = 66501 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 340.276\n",
      "INFO:tensorflow:loss = 0.0062528215, step = 66601 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 289.214\n",
      "INFO:tensorflow:loss = 0.0075903647, step = 66701 (0.347 sec)\n",
      "INFO:tensorflow:global_step/sec: 308.65\n",
      "INFO:tensorflow:loss = 0.006710857, step = 66801 (0.322 sec)\n",
      "INFO:tensorflow:global_step/sec: 295.625\n",
      "INFO:tensorflow:loss = 0.005974801, step = 66901 (0.337 sec)\n",
      "INFO:tensorflow:global_step/sec: 309.5\n",
      "INFO:tensorflow:loss = 0.0073279357, step = 67001 (0.324 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.482\n",
      "INFO:tensorflow:loss = 0.00564766, step = 67101 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.592\n",
      "INFO:tensorflow:loss = 0.008679296, step = 67201 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 274.691\n",
      "INFO:tensorflow:loss = 0.00754069, step = 67301 (0.367 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.224\n",
      "INFO:tensorflow:loss = 0.008784084, step = 67401 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.662\n",
      "INFO:tensorflow:loss = 0.0042150477, step = 67501 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.229\n",
      "INFO:tensorflow:loss = 0.008199089, step = 67601 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 227.338\n",
      "INFO:tensorflow:loss = 0.011958205, step = 67701 (0.434 sec)\n",
      "INFO:tensorflow:global_step/sec: 339.637\n",
      "INFO:tensorflow:loss = 0.010456848, step = 67801 (0.300 sec)\n",
      "INFO:tensorflow:global_step/sec: 355.882\n",
      "INFO:tensorflow:loss = 0.0057692006, step = 67901 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 318.019\n",
      "INFO:tensorflow:loss = 0.010300332, step = 68001 (0.317 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.432\n",
      "INFO:tensorflow:loss = 0.010580592, step = 68101 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.12\n",
      "INFO:tensorflow:loss = 0.012563731, step = 68201 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.23\n",
      "INFO:tensorflow:loss = 0.011234738, step = 68301 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.081\n",
      "INFO:tensorflow:loss = 0.0067612804, step = 68401 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 465.797\n",
      "INFO:tensorflow:loss = 0.0053947447, step = 68501 (0.214 sec)\n",
      "INFO:tensorflow:global_step/sec: 453.073\n",
      "INFO:tensorflow:loss = 0.005955888, step = 68601 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 452.81\n",
      "INFO:tensorflow:loss = 0.009174608, step = 68701 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.342\n",
      "INFO:tensorflow:loss = 0.011071189, step = 68801 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.485\n",
      "INFO:tensorflow:loss = 0.014622258, step = 68901 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 467.609\n",
      "INFO:tensorflow:loss = 0.01595101, step = 69001 (0.214 sec)\n",
      "INFO:tensorflow:global_step/sec: 440.184\n",
      "INFO:tensorflow:loss = 0.00855986, step = 69101 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.741\n",
      "INFO:tensorflow:loss = 0.007446575, step = 69201 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.076\n",
      "INFO:tensorflow:loss = 0.004101183, step = 69301 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 339.274\n",
      "INFO:tensorflow:loss = 0.0051287794, step = 69401 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.228\n",
      "INFO:tensorflow:loss = 0.008996535, step = 69501 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.888\n",
      "INFO:tensorflow:loss = 0.0059713535, step = 69601 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 450.702\n",
      "INFO:tensorflow:loss = 0.0046340106, step = 69701 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 461.543\n",
      "INFO:tensorflow:loss = 0.007652163, step = 69801 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 448.432\n",
      "INFO:tensorflow:loss = 0.008599006, step = 69901 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.014\n",
      "INFO:tensorflow:loss = 0.010495636, step = 70001 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 446.185\n",
      "INFO:tensorflow:loss = 0.004957189, step = 70101 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 456.512\n",
      "INFO:tensorflow:loss = 0.008525104, step = 70201 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 449.339\n",
      "INFO:tensorflow:loss = 0.008331495, step = 70301 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 448.78\n",
      "INFO:tensorflow:loss = 0.005508979, step = 70401 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.591\n",
      "INFO:tensorflow:loss = 0.005293389, step = 70501 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 453.014\n",
      "INFO:tensorflow:loss = 0.0064183557, step = 70601 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 461.905\n",
      "INFO:tensorflow:loss = 0.0062915036, step = 70701 (0.215 sec)\n",
      "INFO:tensorflow:global_step/sec: 459.012\n",
      "INFO:tensorflow:loss = 0.009505634, step = 70801 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.454\n",
      "INFO:tensorflow:loss = 0.009867096, step = 70901 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.518\n",
      "INFO:tensorflow:loss = 0.006648094, step = 71001 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 476.601\n",
      "INFO:tensorflow:loss = 0.009199454, step = 71101 (0.207 sec)\n",
      "INFO:tensorflow:global_step/sec: 400.533\n",
      "INFO:tensorflow:loss = 0.0040453756, step = 71201 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 460.339\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.0076406007, step = 71301 (0.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.033\n",
      "INFO:tensorflow:loss = 0.0035454957, step = 71401 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.817\n",
      "INFO:tensorflow:loss = 0.012230123, step = 71501 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.877\n",
      "INFO:tensorflow:loss = 0.0046327766, step = 71601 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.979\n",
      "INFO:tensorflow:loss = 0.009797904, step = 71701 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.299\n",
      "INFO:tensorflow:loss = 0.0057702614, step = 71801 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 434.612\n",
      "INFO:tensorflow:loss = 0.0057235556, step = 71901 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.485\n",
      "INFO:tensorflow:loss = 0.0064652134, step = 72001 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.645\n",
      "INFO:tensorflow:loss = 0.004923028, step = 72101 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.817\n",
      "INFO:tensorflow:loss = 0.0047397236, step = 72201 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.134\n",
      "INFO:tensorflow:loss = 0.0059193037, step = 72301 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.632\n",
      "INFO:tensorflow:loss = 0.0061919587, step = 72401 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.184\n",
      "INFO:tensorflow:loss = 0.007883918, step = 72501 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.57\n",
      "INFO:tensorflow:loss = 0.004685017, step = 72601 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 445.886\n",
      "INFO:tensorflow:loss = 0.008785181, step = 72701 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.41\n",
      "INFO:tensorflow:loss = 0.0116095375, step = 72801 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.365\n",
      "INFO:tensorflow:loss = 0.010542066, step = 72901 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.843\n",
      "INFO:tensorflow:loss = 0.010022514, step = 73001 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.313\n",
      "INFO:tensorflow:loss = 0.005617762, step = 73101 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 428.89\n",
      "INFO:tensorflow:loss = 0.0064762337, step = 73201 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.634\n",
      "INFO:tensorflow:loss = 0.005318011, step = 73301 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.844\n",
      "INFO:tensorflow:loss = 0.006772394, step = 73401 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.088\n",
      "INFO:tensorflow:loss = 0.007899692, step = 73501 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.211\n",
      "INFO:tensorflow:loss = 0.008283274, step = 73601 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 400.345\n",
      "INFO:tensorflow:loss = 0.0034044734, step = 73701 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 428.207\n",
      "INFO:tensorflow:loss = 0.0071456577, step = 73801 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 428.488\n",
      "INFO:tensorflow:loss = 0.004962736, step = 73901 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 437.858\n",
      "INFO:tensorflow:loss = 0.0046185316, step = 74001 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.271\n",
      "INFO:tensorflow:loss = 0.00762443, step = 74101 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.809\n",
      "INFO:tensorflow:loss = 0.006046824, step = 74201 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 436.202\n",
      "INFO:tensorflow:loss = 0.0076372833, step = 74301 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 441.895\n",
      "INFO:tensorflow:loss = 0.0073806546, step = 74401 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.039\n",
      "INFO:tensorflow:loss = 0.004416166, step = 74501 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.574\n",
      "INFO:tensorflow:loss = 0.0072134756, step = 74601 (0.231 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.381\n",
      "INFO:tensorflow:loss = 0.009367794, step = 74701 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.452\n",
      "INFO:tensorflow:loss = 0.014975514, step = 74801 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.45\n",
      "INFO:tensorflow:loss = 0.0055306666, step = 74901 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.953\n",
      "INFO:tensorflow:loss = 0.013139147, step = 75001 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 428.276\n",
      "INFO:tensorflow:loss = 0.005985034, step = 75101 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.84\n",
      "INFO:tensorflow:loss = 0.00876871, step = 75201 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 438.702\n",
      "INFO:tensorflow:loss = 0.010819644, step = 75301 (0.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.443\n",
      "INFO:tensorflow:loss = 0.006305651, step = 75401 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.613\n",
      "INFO:tensorflow:loss = 0.008520262, step = 75501 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 424\n",
      "INFO:tensorflow:loss = 0.010008522, step = 75601 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 436.661\n",
      "INFO:tensorflow:loss = 0.004215484, step = 75701 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.174\n",
      "INFO:tensorflow:loss = 0.009802019, step = 75801 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 435.696\n",
      "INFO:tensorflow:loss = 0.005014785, step = 75901 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.266\n",
      "INFO:tensorflow:loss = 0.00578269, step = 76001 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.726\n",
      "INFO:tensorflow:loss = 0.004567367, step = 76101 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.007\n",
      "INFO:tensorflow:loss = 0.0045864154, step = 76201 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.764\n",
      "INFO:tensorflow:loss = 0.0054661036, step = 76301 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.723\n",
      "INFO:tensorflow:loss = 0.0062434366, step = 76401 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.905\n",
      "INFO:tensorflow:loss = 0.0044673495, step = 76501 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.353\n",
      "INFO:tensorflow:loss = 0.0051483996, step = 76601 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 351.804\n",
      "INFO:tensorflow:loss = 0.006520223, step = 76701 (0.288 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.154\n",
      "INFO:tensorflow:loss = 0.0060899286, step = 76801 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.695\n",
      "INFO:tensorflow:loss = 0.0047183395, step = 76901 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.74\n",
      "INFO:tensorflow:loss = 0.007407232, step = 77001 (0.301 sec)\n",
      "INFO:tensorflow:global_step/sec: 332.601\n",
      "INFO:tensorflow:loss = 0.01136122, step = 77101 (0.301 sec)\n",
      "INFO:tensorflow:global_step/sec: 323.885\n",
      "INFO:tensorflow:loss = 0.0063055223, step = 77201 (0.309 sec)\n",
      "INFO:tensorflow:global_step/sec: 248.848\n",
      "INFO:tensorflow:loss = 0.005401225, step = 77301 (0.402 sec)\n",
      "INFO:tensorflow:global_step/sec: 299.695\n",
      "INFO:tensorflow:loss = 0.0042113457, step = 77401 (0.335 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.507\n",
      "INFO:tensorflow:loss = 0.007134037, step = 77501 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.281\n",
      "INFO:tensorflow:loss = 0.0068142368, step = 77601 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.385\n",
      "INFO:tensorflow:loss = 0.0041198777, step = 77701 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.037\n",
      "INFO:tensorflow:loss = 0.009277171, step = 77801 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.222\n",
      "INFO:tensorflow:loss = 0.0044798655, step = 77901 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.494\n",
      "INFO:tensorflow:loss = 0.009371477, step = 78001 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.796\n",
      "INFO:tensorflow:loss = 0.006991404, step = 78101 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.747\n",
      "INFO:tensorflow:loss = 0.008116752, step = 78201 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.272\n",
      "INFO:tensorflow:loss = 0.009652657, step = 78301 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.803\n",
      "INFO:tensorflow:loss = 0.005908731, step = 78401 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.119\n",
      "INFO:tensorflow:loss = 0.006526216, step = 78501 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 400.736\n",
      "INFO:tensorflow:loss = 0.0062486287, step = 78601 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.596\n",
      "INFO:tensorflow:loss = 0.0060153054, step = 78701 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.23\n",
      "INFO:tensorflow:loss = 0.004742941, step = 78801 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.924\n",
      "INFO:tensorflow:loss = 0.011788448, step = 78901 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.698\n",
      "INFO:tensorflow:loss = 0.0057389974, step = 79001 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.021\n",
      "INFO:tensorflow:loss = 0.006981428, step = 79101 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.133\n",
      "INFO:tensorflow:loss = 0.006011064, step = 79201 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.489\n",
      "INFO:tensorflow:loss = 0.006946276, step = 79301 (0.251 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 411.164\n",
      "INFO:tensorflow:loss = 0.007632213, step = 79401 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.003\n",
      "INFO:tensorflow:loss = 0.0043778038, step = 79501 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.753\n",
      "INFO:tensorflow:loss = 0.005868679, step = 79601 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.006\n",
      "INFO:tensorflow:loss = 0.0047539934, step = 79701 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.249\n",
      "INFO:tensorflow:loss = 0.005021378, step = 79801 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.278\n",
      "INFO:tensorflow:loss = 0.008445743, step = 79901 (0.258 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 80000 into /tmp/tmpsbdccn23/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-07:36:38\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-80000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'previous_ensemble' dict for global step 80000: architecture/adanet/ensembles = b\"\\nt\\n,adanet/previous_ensemble/architecture/adanetB:\\x08\\x07\\x12\\x00B4| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.032477677, average_loss/adanet/subnetwork = 0.035940513, average_loss/adanet/uniform_average_ensemble = 0.033833675, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.038081136, loss/adanet/subnetwork = 0.038228974, loss/adanet/uniform_average_ensemble = 0.042276174, prediction/mean/adanet/adanet_weighted_ensemble = 3.1002867, prediction/mean/adanet/subnetwork = 3.1267877, prediction/mean/adanet/uniform_average_ensemble = 3.14097\n",
      "INFO:tensorflow:Saving candidate '2_layer_dnn' dict for global step 80000: architecture/adanet/ensembles = b\"\\n\\x94\\x01\\n;adanet/iteration_3/ensemble_2_layer_dnn/architecture/adanetBK\\x08\\x07\\x12\\x00BE| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.034470588, average_loss/adanet/subnetwork = 0.029604359, average_loss/adanet/uniform_average_ensemble = 0.03166156, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.034929536, loss/adanet/subnetwork = 0.03914282, loss/adanet/uniform_average_ensemble = 0.04027512, prediction/mean/adanet/adanet_weighted_ensemble = 3.0717044, prediction/mean/adanet/subnetwork = 3.1459546, prediction/mean/adanet/uniform_average_ensemble = 3.1422162\n",
      "INFO:tensorflow:Saving candidate '3_layer_dnn' dict for global step 80000: architecture/adanet/ensembles = b\"\\n\\x94\\x01\\n;adanet/iteration_3/ensemble_3_layer_dnn/architecture/adanetBK\\x08\\x07\\x12\\x00BE| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' | b'3_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.03214211, average_loss/adanet/subnetwork = 0.028511057, average_loss/adanet/uniform_average_ensemble = 0.030794747, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.032537706, loss/adanet/subnetwork = 0.030393401, loss/adanet/uniform_average_ensemble = 0.037526637, prediction/mean/adanet/adanet_weighted_ensemble = 3.080623, prediction/mean/adanet/subnetwork = 3.1202893, prediction/mean/adanet/uniform_average_ensemble = 3.1357996\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-19-07:36:40\n",
      "INFO:tensorflow:Saving dict for global step 80000: average_loss = 0.03214211, average_loss/adanet/adanet_weighted_ensemble = 0.03214211, average_loss/adanet/subnetwork = 0.028511057, average_loss/adanet/uniform_average_ensemble = 0.030794747, global_step = 80000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.032537706, loss/adanet/adanet_weighted_ensemble = 0.032537706, loss/adanet/subnetwork = 0.030393401, loss/adanet/uniform_average_ensemble = 0.037526637, prediction/mean = 3.080623, prediction/mean/adanet/adanet_weighted_ensemble = 3.080623, prediction/mean/adanet/subnetwork = 3.1202893, prediction/mean/adanet/uniform_average_ensemble = 3.1357996\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 80000: /tmp/tmpsbdccn23/model.ckpt-80000\n",
      "INFO:tensorflow:Loss for final step: 0.0049718106.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Starting ensemble evaluation for iteration 3\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-80000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Encountered end of input after 14 evaluations\n",
      "INFO:tensorflow:Computed ensemble metrics: adanet_loss/previous_ensemble = 0.009538, adanet_loss/2_layer_dnn = 0.006759, adanet_loss/3_layer_dnn = 0.006060\n",
      "INFO:tensorflow:Finished ensemble evaluation for iteration 3\n",
      "INFO:tensorflow:'3_layer_dnn' at index 2 is moving onto the next iteration\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Freezing best ensemble to /tmp/tmpsbdccn23/frozen/ensemble-3.meta\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-80000\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-3.meta with features: ['x'].\n",
      "INFO:tensorflow:Overwriting checkpoint with new graph for iteration 4 to /tmp/tmpsbdccn23/model.ckpt-80000\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-80000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Finished training Adanet iteration 3\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 4\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-3.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/increment.ckpt-4\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 80000 into /tmp/tmpsbdccn23/model.ckpt.\n",
      "INFO:tensorflow:loss = 0.005968827, step = 80001\n",
      "INFO:tensorflow:global_step/sec: 99.0742\n",
      "INFO:tensorflow:loss = 0.0064767604, step = 80101 (1.011 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.565\n",
      "INFO:tensorflow:loss = 0.010301616, step = 80201 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.837\n",
      "INFO:tensorflow:loss = 0.005047547, step = 80301 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.585\n",
      "INFO:tensorflow:loss = 0.007517004, step = 80401 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.89\n",
      "INFO:tensorflow:loss = 0.0047707586, step = 80501 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.676\n",
      "INFO:tensorflow:loss = 0.005466242, step = 80601 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.678\n",
      "INFO:tensorflow:loss = 0.0065845056, step = 80701 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.914\n",
      "INFO:tensorflow:loss = 0.0063047465, step = 80801 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.144\n",
      "INFO:tensorflow:loss = 0.009454205, step = 80901 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.051\n",
      "INFO:tensorflow:loss = 0.0071954094, step = 81001 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.662\n",
      "INFO:tensorflow:loss = 0.0073958794, step = 81101 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.87\n",
      "INFO:tensorflow:loss = 0.007455849, step = 81201 (0.238 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 416.172\n",
      "INFO:tensorflow:loss = 0.0038198205, step = 81301 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.783\n",
      "INFO:tensorflow:loss = 0.002871193, step = 81401 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.426\n",
      "INFO:tensorflow:loss = 0.0057152724, step = 81501 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.228\n",
      "INFO:tensorflow:loss = 0.0043174643, step = 81601 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.499\n",
      "INFO:tensorflow:loss = 0.007462681, step = 81701 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.319\n",
      "INFO:tensorflow:loss = 0.006305181, step = 81801 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.071\n",
      "INFO:tensorflow:loss = 0.00810397, step = 81901 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.099\n",
      "INFO:tensorflow:loss = 0.0071171797, step = 82001 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.092\n",
      "INFO:tensorflow:loss = 0.010425732, step = 82101 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.505\n",
      "INFO:tensorflow:loss = 0.006138864, step = 82201 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.649\n",
      "INFO:tensorflow:loss = 0.0058132876, step = 82301 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 323.916\n",
      "INFO:tensorflow:loss = 0.0037873953, step = 82401 (0.312 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.712\n",
      "INFO:tensorflow:loss = 0.006950916, step = 82501 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.116\n",
      "INFO:tensorflow:loss = 0.0026223212, step = 82601 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.1\n",
      "INFO:tensorflow:loss = 0.006516129, step = 82701 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.497\n",
      "INFO:tensorflow:loss = 0.0060444055, step = 82801 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.855\n",
      "INFO:tensorflow:loss = 0.0053609535, step = 82901 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.694\n",
      "INFO:tensorflow:loss = 0.005755792, step = 83001 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.668\n",
      "INFO:tensorflow:loss = 0.005737916, step = 83101 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.898\n",
      "INFO:tensorflow:loss = 0.005597957, step = 83201 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.201\n",
      "INFO:tensorflow:loss = 0.008296445, step = 83301 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.465\n",
      "INFO:tensorflow:loss = 0.007999277, step = 83401 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.837\n",
      "INFO:tensorflow:loss = 0.00965465, step = 83501 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.903\n",
      "INFO:tensorflow:loss = 0.0043019895, step = 83601 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.02\n",
      "INFO:tensorflow:loss = 0.0080252625, step = 83701 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.164\n",
      "INFO:tensorflow:loss = 0.006977511, step = 83801 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 426.883\n",
      "INFO:tensorflow:loss = 0.0037052818, step = 83901 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.03\n",
      "INFO:tensorflow:loss = 0.006911964, step = 84001 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.653\n",
      "INFO:tensorflow:loss = 0.0065367995, step = 84101 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.965\n",
      "INFO:tensorflow:loss = 0.0038247383, step = 84201 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.914\n",
      "INFO:tensorflow:loss = 0.005363325, step = 84301 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.533\n",
      "INFO:tensorflow:loss = 0.005000092, step = 84401 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.174\n",
      "INFO:tensorflow:loss = 0.0057771797, step = 84501 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.403\n",
      "INFO:tensorflow:loss = 0.007785161, step = 84601 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 400.768\n",
      "INFO:tensorflow:loss = 0.005486235, step = 84701 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.797\n",
      "INFO:tensorflow:loss = 0.004908246, step = 84801 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.161\n",
      "INFO:tensorflow:loss = 0.005847712, step = 84901 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.736\n",
      "INFO:tensorflow:loss = 0.00392846, step = 85001 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.149\n",
      "INFO:tensorflow:loss = 0.0034701028, step = 85101 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.866\n",
      "INFO:tensorflow:loss = 0.006456704, step = 85201 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.083\n",
      "INFO:tensorflow:loss = 0.0077680405, step = 85301 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.993\n",
      "INFO:tensorflow:loss = 0.006451573, step = 85401 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.067\n",
      "INFO:tensorflow:loss = 0.0045454674, step = 85501 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.904\n",
      "INFO:tensorflow:loss = 0.007705566, step = 85601 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.93\n",
      "INFO:tensorflow:loss = 0.0056776307, step = 85701 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.587\n",
      "INFO:tensorflow:loss = 0.006023227, step = 85801 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.706\n",
      "INFO:tensorflow:loss = 0.0063513527, step = 85901 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.191\n",
      "INFO:tensorflow:loss = 0.006911626, step = 86001 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.278\n",
      "INFO:tensorflow:loss = 0.0060713426, step = 86101 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.658\n",
      "INFO:tensorflow:loss = 0.009185913, step = 86201 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.326\n",
      "INFO:tensorflow:loss = 0.00455994, step = 86301 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.194\n",
      "INFO:tensorflow:loss = 0.004431198, step = 86401 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.604\n",
      "INFO:tensorflow:loss = 0.007269846, step = 86501 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 320.704\n",
      "INFO:tensorflow:loss = 0.004442989, step = 86601 (0.312 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.185\n",
      "INFO:tensorflow:loss = 0.004992756, step = 86701 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.615\n",
      "INFO:tensorflow:loss = 0.0076543908, step = 86801 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.645\n",
      "INFO:tensorflow:loss = 0.0050187297, step = 86901 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.068\n",
      "INFO:tensorflow:loss = 0.0046609966, step = 87001 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 369.616\n",
      "INFO:tensorflow:loss = 0.0044163875, step = 87101 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.168\n",
      "INFO:tensorflow:loss = 0.007560266, step = 87201 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.969\n",
      "INFO:tensorflow:loss = 0.005357282, step = 87301 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.405\n",
      "INFO:tensorflow:loss = 0.0069226543, step = 87401 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.517\n",
      "INFO:tensorflow:loss = 0.0030825417, step = 87501 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.765\n",
      "INFO:tensorflow:loss = 0.0069364267, step = 87601 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.683\n",
      "INFO:tensorflow:loss = 0.00977301, step = 87701 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.146\n",
      "INFO:tensorflow:loss = 0.008857778, step = 87801 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.964\n",
      "INFO:tensorflow:loss = 0.00629481, step = 87901 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 277.609\n",
      "INFO:tensorflow:loss = 0.008666011, step = 88001 (0.359 sec)\n",
      "INFO:tensorflow:global_step/sec: 325.486\n",
      "INFO:tensorflow:loss = 0.008077523, step = 88101 (0.304 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.651\n",
      "INFO:tensorflow:loss = 0.0077744913, step = 88201 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.216\n",
      "INFO:tensorflow:loss = 0.009249054, step = 88301 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.307\n",
      "INFO:tensorflow:loss = 0.0044535142, step = 88401 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.247\n",
      "INFO:tensorflow:loss = 0.005009072, step = 88501 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.673\n",
      "INFO:tensorflow:loss = 0.004016795, step = 88601 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.024\n",
      "INFO:tensorflow:loss = 0.008505994, step = 88701 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.246\n",
      "INFO:tensorflow:loss = 0.006793923, step = 88801 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.289\n",
      "INFO:tensorflow:loss = 0.009669229, step = 88901 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.872\n",
      "INFO:tensorflow:loss = 0.015339302, step = 89001 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.305\n",
      "INFO:tensorflow:loss = 0.0065113106, step = 89101 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.831\n",
      "INFO:tensorflow:loss = 0.006077813, step = 89201 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.471\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.0045325924, step = 89301 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.166\n",
      "INFO:tensorflow:loss = 0.005176927, step = 89401 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.981\n",
      "INFO:tensorflow:loss = 0.0048831245, step = 89501 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.779\n",
      "INFO:tensorflow:loss = 0.005080943, step = 89601 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.321\n",
      "INFO:tensorflow:loss = 0.0035263128, step = 89701 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.429\n",
      "INFO:tensorflow:loss = 0.005478317, step = 89801 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.066\n",
      "INFO:tensorflow:loss = 0.006956035, step = 89901 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 362.109\n",
      "INFO:tensorflow:loss = 0.0063518365, step = 90001 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.001\n",
      "INFO:tensorflow:loss = 0.0044877417, step = 90101 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.714\n",
      "INFO:tensorflow:loss = 0.007858607, step = 90201 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.357\n",
      "INFO:tensorflow:loss = 0.00694497, step = 90301 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.51\n",
      "INFO:tensorflow:loss = 0.0038716933, step = 90401 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.052\n",
      "INFO:tensorflow:loss = 0.0037437351, step = 90501 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.947\n",
      "INFO:tensorflow:loss = 0.0058524073, step = 90601 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.164\n",
      "INFO:tensorflow:loss = 0.004938422, step = 90701 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.343\n",
      "INFO:tensorflow:loss = 0.007973131, step = 90801 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.635\n",
      "INFO:tensorflow:loss = 0.00639206, step = 90901 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.043\n",
      "INFO:tensorflow:loss = 0.0053040823, step = 91001 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.135\n",
      "INFO:tensorflow:loss = 0.007648998, step = 91101 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.075\n",
      "INFO:tensorflow:loss = 0.0030359528, step = 91201 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.662\n",
      "INFO:tensorflow:loss = 0.0068054684, step = 91301 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.07\n",
      "INFO:tensorflow:loss = 0.0032522846, step = 91401 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.771\n",
      "INFO:tensorflow:loss = 0.0074958466, step = 91501 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.981\n",
      "INFO:tensorflow:loss = 0.0038287179, step = 91601 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.664\n",
      "INFO:tensorflow:loss = 0.007917597, step = 91701 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.282\n",
      "INFO:tensorflow:loss = 0.0039814087, step = 91801 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.873\n",
      "INFO:tensorflow:loss = 0.003935887, step = 91901 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.193\n",
      "INFO:tensorflow:loss = 0.005519495, step = 92001 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.267\n",
      "INFO:tensorflow:loss = 0.0040905545, step = 92101 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.434\n",
      "INFO:tensorflow:loss = 0.0028350702, step = 92201 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.894\n",
      "INFO:tensorflow:loss = 0.004300507, step = 92301 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.323\n",
      "INFO:tensorflow:loss = 0.005657818, step = 92401 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 324.657\n",
      "INFO:tensorflow:loss = 0.00594386, step = 92501 (0.306 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.625\n",
      "INFO:tensorflow:loss = 0.005790655, step = 92601 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.286\n",
      "INFO:tensorflow:loss = 0.007684627, step = 92701 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.155\n",
      "INFO:tensorflow:loss = 0.010586033, step = 92801 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.585\n",
      "INFO:tensorflow:loss = 0.010163115, step = 92901 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 400.916\n",
      "INFO:tensorflow:loss = 0.009129146, step = 93001 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.027\n",
      "INFO:tensorflow:loss = 0.005577663, step = 93101 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.391\n",
      "INFO:tensorflow:loss = 0.0049700486, step = 93201 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.215\n",
      "INFO:tensorflow:loss = 0.005613825, step = 93301 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.294\n",
      "INFO:tensorflow:loss = 0.0049747387, step = 93401 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.58\n",
      "INFO:tensorflow:loss = 0.0050827097, step = 93501 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.499\n",
      "INFO:tensorflow:loss = 0.0077262563, step = 93601 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.579\n",
      "INFO:tensorflow:loss = 0.0030253306, step = 93701 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.897\n",
      "INFO:tensorflow:loss = 0.00661487, step = 93801 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.499\n",
      "INFO:tensorflow:loss = 0.004713323, step = 93901 (0.242 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.83\n",
      "INFO:tensorflow:loss = 0.0033538695, step = 94001 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.35\n",
      "INFO:tensorflow:loss = 0.007733818, step = 94101 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 400.16\n",
      "INFO:tensorflow:loss = 0.0055770525, step = 94201 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.271\n",
      "INFO:tensorflow:loss = 0.006473027, step = 94301 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.777\n",
      "INFO:tensorflow:loss = 0.0041343123, step = 94401 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.091\n",
      "INFO:tensorflow:loss = 0.004067247, step = 94501 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.124\n",
      "INFO:tensorflow:loss = 0.0044339644, step = 94601 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.815\n",
      "INFO:tensorflow:loss = 0.0072677145, step = 94701 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.996\n",
      "INFO:tensorflow:loss = 0.0120704435, step = 94801 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.298\n",
      "INFO:tensorflow:loss = 0.0046949405, step = 94901 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.967\n",
      "INFO:tensorflow:loss = 0.0115144625, step = 95001 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.875\n",
      "INFO:tensorflow:loss = 0.005885225, step = 95101 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.058\n",
      "INFO:tensorflow:loss = 0.008664591, step = 95201 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.881\n",
      "INFO:tensorflow:loss = 0.009544247, step = 95301 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.196\n",
      "INFO:tensorflow:loss = 0.005615027, step = 95401 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.573\n",
      "INFO:tensorflow:loss = 0.006134661, step = 95501 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.953\n",
      "INFO:tensorflow:loss = 0.008331908, step = 95601 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.582\n",
      "INFO:tensorflow:loss = 0.0023516277, step = 95701 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 363.741\n",
      "INFO:tensorflow:loss = 0.010425936, step = 95801 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.286\n",
      "INFO:tensorflow:loss = 0.0038912015, step = 95901 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.645\n",
      "INFO:tensorflow:loss = 0.003912572, step = 96001 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.652\n",
      "INFO:tensorflow:loss = 0.003866215, step = 96101 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.323\n",
      "INFO:tensorflow:loss = 0.0038413634, step = 96201 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 414.861\n",
      "INFO:tensorflow:loss = 0.005082471, step = 96301 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.607\n",
      "INFO:tensorflow:loss = 0.0068623424, step = 96401 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.715\n",
      "INFO:tensorflow:loss = 0.0044919383, step = 96501 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.625\n",
      "INFO:tensorflow:loss = 0.0055344556, step = 96601 (0.230 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.959\n",
      "INFO:tensorflow:loss = 0.0050028567, step = 96701 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.715\n",
      "INFO:tensorflow:loss = 0.005279351, step = 96801 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 421.292\n",
      "INFO:tensorflow:loss = 0.00425283, step = 96901 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.245\n",
      "INFO:tensorflow:loss = 0.005783615, step = 97001 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 417.896\n",
      "INFO:tensorflow:loss = 0.011019124, step = 97101 (0.239 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.398\n",
      "INFO:tensorflow:loss = 0.0031761369, step = 97201 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.265\n",
      "INFO:tensorflow:loss = 0.0053030048, step = 97301 (0.238 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 421.408\n",
      "INFO:tensorflow:loss = 0.002677832, step = 97401 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.995\n",
      "INFO:tensorflow:loss = 0.0071991016, step = 97501 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.571\n",
      "INFO:tensorflow:loss = 0.0049621644, step = 97601 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.601\n",
      "INFO:tensorflow:loss = 0.005003189, step = 97701 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 415.16\n",
      "INFO:tensorflow:loss = 0.008433397, step = 97801 (0.241 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.071\n",
      "INFO:tensorflow:loss = 0.004794498, step = 97901 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.041\n",
      "INFO:tensorflow:loss = 0.009021459, step = 98001 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.839\n",
      "INFO:tensorflow:loss = 0.004788899, step = 98101 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.398\n",
      "INFO:tensorflow:loss = 0.0073380973, step = 98201 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.001\n",
      "INFO:tensorflow:loss = 0.00816357, step = 98301 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 400.665\n",
      "INFO:tensorflow:loss = 0.0055529545, step = 98401 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.691\n",
      "INFO:tensorflow:loss = 0.005340519, step = 98501 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 407.212\n",
      "INFO:tensorflow:loss = 0.0058506094, step = 98601 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.757\n",
      "INFO:tensorflow:loss = 0.0052951295, step = 98701 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.894\n",
      "INFO:tensorflow:loss = 0.004655308, step = 98801 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.17\n",
      "INFO:tensorflow:loss = 0.010407442, step = 98901 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 413.529\n",
      "INFO:tensorflow:loss = 0.0045062825, step = 99001 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.54\n",
      "INFO:tensorflow:loss = 0.006256766, step = 99101 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 419.873\n",
      "INFO:tensorflow:loss = 0.0043131188, step = 99201 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.528\n",
      "INFO:tensorflow:loss = 0.0057967496, step = 99301 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.794\n",
      "INFO:tensorflow:loss = 0.004853651, step = 99401 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.357\n",
      "INFO:tensorflow:loss = 0.0043617273, step = 99501 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 484.889\n",
      "INFO:tensorflow:loss = 0.0048889033, step = 99601 (0.208 sec)\n",
      "INFO:tensorflow:global_step/sec: 416.99\n",
      "INFO:tensorflow:loss = 0.004474927, step = 99701 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.358\n",
      "INFO:tensorflow:loss = 0.005093452, step = 99801 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.738\n",
      "INFO:tensorflow:loss = 0.007578698, step = 99901 (0.268 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 100000 into /tmp/tmpsbdccn23/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpsbdccn23/frozen/ensemble-3.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-07:37:47\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpsbdccn23/model.ckpt-100000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'previous_ensemble' dict for global step 100000: architecture/adanet/ensembles = b\"\\n\\x85\\x01\\n,adanet/previous_ensemble/architecture/adanetBK\\x08\\x07\\x12\\x00BE| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' | b'3_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.03214211, average_loss/adanet/subnetwork = 0.028511057, average_loss/adanet/uniform_average_ensemble = 0.030794745, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.032537706, loss/adanet/subnetwork = 0.030393401, loss/adanet/uniform_average_ensemble = 0.03752663, prediction/mean/adanet/adanet_weighted_ensemble = 3.080623, prediction/mean/adanet/subnetwork = 3.1202893, prediction/mean/adanet/uniform_average_ensemble = 3.1357996\n",
      "INFO:tensorflow:Saving candidate '3_layer_dnn' dict for global step 100000: architecture/adanet/ensembles = b\"\\n\\xa5\\x01\\n;adanet/iteration_4/ensemble_3_layer_dnn/architecture/adanetB\\\\\\x08\\x07\\x12\\x00BV| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' | b'3_layer_dnn' | b'3_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.031512633, average_loss/adanet/subnetwork = 0.041267823, average_loss/adanet/uniform_average_ensemble = 0.031948242, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.030517163, loss/adanet/subnetwork = 0.04582113, loss/adanet/uniform_average_ensemble = 0.038237095, prediction/mean/adanet/adanet_weighted_ensemble = 3.0612369, prediction/mean/adanet/subnetwork = 3.1456003, prediction/mean/adanet/uniform_average_ensemble = 3.13776\n",
      "INFO:tensorflow:Saving candidate '4_layer_dnn' dict for global step 100000: architecture/adanet/ensembles = b\"\\n\\xa5\\x01\\n;adanet/iteration_4/ensemble_4_layer_dnn/architecture/adanetB\\\\\\x08\\x07\\x12\\x00BV| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' | b'3_layer_dnn' | b'4_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.031453762, average_loss/adanet/subnetwork = 0.036653724, average_loss/adanet/uniform_average_ensemble = 0.031013792, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.031099707, loss/adanet/subnetwork = 0.056759786, loss/adanet/uniform_average_ensemble = 0.03993302, prediction/mean/adanet/adanet_weighted_ensemble = 3.0610926, prediction/mean/adanet/subnetwork = 3.1055298, prediction/mean/adanet/uniform_average_ensemble = 3.1297457\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-19-07:37:49\n",
      "INFO:tensorflow:Saving dict for global step 100000: average_loss = 0.031453762, average_loss/adanet/adanet_weighted_ensemble = 0.031453762, average_loss/adanet/subnetwork = 0.036653724, average_loss/adanet/uniform_average_ensemble = 0.031013792, global_step = 100000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.031099707, loss/adanet/adanet_weighted_ensemble = 0.031099707, loss/adanet/subnetwork = 0.056759786, loss/adanet/uniform_average_ensemble = 0.03993302, prediction/mean = 3.0610926, prediction/mean/adanet/adanet_weighted_ensemble = 3.0610926, prediction/mean/adanet/subnetwork = 3.1055298, prediction/mean/adanet/uniform_average_ensemble = 3.1297457\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 100000: /tmp/tmpsbdccn23/model.ckpt-100000\n",
      "INFO:tensorflow:Loss for final step: 0.004615612.\n",
      "Loss: 0.031453762\n",
      "Uniform average loss: 0.031013792\n",
      "Architecture: b\"| b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' | b'3_layer_dnn' | b'4_layer_dnn' |\"\n"
     ]
    }
   ],
   "source": [
    "#@test {\"skip\": true}\n",
    "results, _ = train_and_evaluate(learn_mixture_weights=True)\n",
    "print(\"Loss:\", results[\"average_loss\"])\n",
    "print(\"Uniform average loss:\", results[\"average_loss/adanet/uniform_average_ensemble\"])\n",
    "print(\"Architecture:\", ensemble_architecture(results))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "BTUdJ_FWl84O"
   },
   "source": [
    "Learning the mixture weights produces a model with **0.0449** MSE, a bit worse\n",
    "than the uniform average model, which the `adanet.Estimator` always compute as a\n",
    "baseline. The mixture weights were learned without regularization, so they\n",
    "likely overfit to the training set.\n",
    "\n",
    "Observe that AdaNet learned the same ensemble composition as the previous run.\n",
    "Without complexity regularization, AdaNet will favor more complex subnetworks,\n",
    "which may have worse generalization despite improving the empirical error.\n",
    "\n",
    "Finally, let's apply some **complexity regularization** by using $\\lambda > 0$.\n",
    "Since this will penalize more complex subnetworks, AdaNet will select the\n",
    "candidate subnetwork that most improves the objective for its marginal\n",
    "complexity:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "height": 71,
     "test": {
      "output": "ignore",
      "timeout": 900
     }
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 211897,
     "status": "ok",
     "timestamp": 1539111721710,
     "user": {
      "displayName": "",
      "photoUrl": "",
      "userId": ""
     },
     "user_tz": 240
    },
    "id": "3g_C-dT7KaTy",
    "outputId": "f3c3012a-7906-4222-b603-db3ce849c33c",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpyxwongpm\n",
      "INFO:tensorflow:Using config: {'_save_checkpoints_secs': None, '_experimental_distribute': None, '_service': None, '_task_id': 0, '_is_chief': True, '_master': '', '_evaluation_master': '', '_train_distribute': None, '_model_dir': '/tmp/tmpyxwongpm', '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f802a6f6668>, '_keep_checkpoint_every_n_hours': 10000, '_global_id_in_cluster': 0, '_keep_checkpoint_max': 5, '_save_checkpoints_steps': 50000, '_tf_random_seed': 42, '_session_config': allow_soft_placement: true\n",
      "graph_options {\n",
      "  rewrite_options {\n",
      "    meta_optimizer_iterations: ONE\n",
      "  }\n",
      "}\n",
      ", '_protocol': None, '_device_fn': None, '_save_summary_steps': 50000, '_num_ps_replicas': 0, '_eval_distribute': None, '_num_worker_replicas': 1, '_log_step_count_steps': 100, '_task_type': 'worker'}\n",
      "INFO:tensorflow:Running training and evaluation locally (non-distributed).\n",
      "INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps 50000 or save_checkpoints_secs None.\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 0\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpyxwongpm/model.ckpt.\n",
      "INFO:tensorflow:loss = 21.773132, step = 1\n",
      "INFO:tensorflow:global_step/sec: 140.958\n",
      "INFO:tensorflow:loss = 0.62784123, step = 101 (0.711 sec)\n",
      "INFO:tensorflow:global_step/sec: 316.671\n",
      "INFO:tensorflow:loss = 0.56678665, step = 201 (0.315 sec)\n",
      "INFO:tensorflow:global_step/sec: 300.513\n",
      "INFO:tensorflow:loss = 0.078039765, step = 301 (0.333 sec)\n",
      "INFO:tensorflow:global_step/sec: 299.365\n",
      "INFO:tensorflow:loss = 0.086782694, step = 401 (0.334 sec)\n",
      "INFO:tensorflow:global_step/sec: 305.519\n",
      "INFO:tensorflow:loss = 0.08137445, step = 501 (0.327 sec)\n",
      "INFO:tensorflow:global_step/sec: 310.289\n",
      "INFO:tensorflow:loss = 0.056509923, step = 601 (0.325 sec)\n",
      "INFO:tensorflow:global_step/sec: 319.378\n",
      "INFO:tensorflow:loss = 0.025883604, step = 701 (0.313 sec)\n",
      "INFO:tensorflow:global_step/sec: 216.501\n",
      "INFO:tensorflow:loss = 0.030180356, step = 801 (0.462 sec)\n",
      "INFO:tensorflow:global_step/sec: 232.224\n",
      "INFO:tensorflow:loss = 0.037590638, step = 901 (0.429 sec)\n",
      "INFO:tensorflow:global_step/sec: 249.671\n",
      "INFO:tensorflow:loss = 0.06694432, step = 1001 (0.405 sec)\n",
      "INFO:tensorflow:global_step/sec: 237.714\n",
      "INFO:tensorflow:loss = 0.038478173, step = 1101 (0.416 sec)\n",
      "INFO:tensorflow:global_step/sec: 321.145\n",
      "INFO:tensorflow:loss = 0.04998316, step = 1201 (0.311 sec)\n",
      "INFO:tensorflow:global_step/sec: 242.151\n",
      "INFO:tensorflow:loss = 0.09006661, step = 1301 (0.417 sec)\n",
      "INFO:tensorflow:global_step/sec: 308.934\n",
      "INFO:tensorflow:loss = 0.026879994, step = 1401 (0.319 sec)\n",
      "INFO:tensorflow:global_step/sec: 255.401\n",
      "INFO:tensorflow:loss = 0.021093277, step = 1501 (0.393 sec)\n",
      "INFO:tensorflow:global_step/sec: 332.521\n",
      "INFO:tensorflow:loss = 0.03607753, step = 1601 (0.300 sec)\n",
      "INFO:tensorflow:global_step/sec: 312.926\n",
      "INFO:tensorflow:loss = 0.03416162, step = 1701 (0.322 sec)\n",
      "INFO:tensorflow:global_step/sec: 211.064\n",
      "INFO:tensorflow:loss = 0.04626117, step = 1801 (0.471 sec)\n",
      "INFO:tensorflow:global_step/sec: 281.592\n",
      "INFO:tensorflow:loss = 0.07378492, step = 1901 (0.356 sec)\n",
      "INFO:tensorflow:global_step/sec: 282.328\n",
      "INFO:tensorflow:loss = 0.049188316, step = 2001 (0.354 sec)\n",
      "INFO:tensorflow:global_step/sec: 308.875\n",
      "INFO:tensorflow:loss = 0.078179166, step = 2101 (0.323 sec)\n",
      "INFO:tensorflow:global_step/sec: 334.139\n",
      "INFO:tensorflow:loss = 0.03029899, step = 2201 (0.299 sec)\n",
      "INFO:tensorflow:global_step/sec: 294.106\n",
      "INFO:tensorflow:loss = 0.024719719, step = 2301 (0.341 sec)\n",
      "INFO:tensorflow:global_step/sec: 332.18\n",
      "INFO:tensorflow:loss = 0.024992699, step = 2401 (0.301 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.081\n",
      "INFO:tensorflow:loss = 0.04709203, step = 2501 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.409\n",
      "INFO:tensorflow:loss = 0.047214545, step = 2601 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.516\n",
      "INFO:tensorflow:loss = 0.038211394, step = 2701 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 345.828\n",
      "INFO:tensorflow:loss = 0.03274207, step = 2801 (0.294 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.417\n",
      "INFO:tensorflow:loss = 0.04549656, step = 2901 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.133\n",
      "INFO:tensorflow:loss = 0.035480063, step = 3001 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 344.663\n",
      "INFO:tensorflow:loss = 0.024679933, step = 3101 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.242\n",
      "INFO:tensorflow:loss = 0.041259166, step = 3201 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.471\n",
      "INFO:tensorflow:loss = 0.04356738, step = 3301 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.285\n",
      "INFO:tensorflow:loss = 0.034602944, step = 3401 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.285\n",
      "INFO:tensorflow:loss = 0.069668576, step = 3501 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.728\n",
      "INFO:tensorflow:loss = 0.034798123, step = 3601 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.306\n",
      "INFO:tensorflow:loss = 0.021452527, step = 3701 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 350.869\n",
      "INFO:tensorflow:loss = 0.02612273, step = 3801 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.128\n",
      "INFO:tensorflow:loss = 0.031272262, step = 3901 (0.299 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.451\n",
      "INFO:tensorflow:loss = 0.05301467, step = 4001 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.576\n",
      "INFO:tensorflow:loss = 0.02896322, step = 4101 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.845\n",
      "INFO:tensorflow:loss = 0.022142775, step = 4201 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.606\n",
      "INFO:tensorflow:loss = 0.02221645, step = 4301 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 306.676\n",
      "INFO:tensorflow:loss = 0.027055696, step = 4401 (0.323 sec)\n",
      "INFO:tensorflow:global_step/sec: 291.316\n",
      "INFO:tensorflow:loss = 0.050597515, step = 4501 (0.347 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.302\n",
      "INFO:tensorflow:loss = 0.02597157, step = 4601 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 326.918\n",
      "INFO:tensorflow:loss = 0.079174936, step = 4701 (0.303 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.635\n",
      "INFO:tensorflow:loss = 0.034027025, step = 4801 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.448\n",
      "INFO:tensorflow:loss = 0.033307478, step = 4901 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.233\n",
      "INFO:tensorflow:loss = 0.02684283, step = 5001 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.57\n",
      "INFO:tensorflow:loss = 0.039310887, step = 5101 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.382\n",
      "INFO:tensorflow:loss = 0.030656522, step = 5201 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 346.319\n",
      "INFO:tensorflow:loss = 0.078128755, step = 5301 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 321.706\n",
      "INFO:tensorflow:loss = 0.021291938, step = 5401 (0.315 sec)\n",
      "INFO:tensorflow:global_step/sec: 319.996\n",
      "INFO:tensorflow:loss = 0.032513306, step = 5501 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.397\n",
      "INFO:tensorflow:loss = 0.028400544, step = 5601 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.27\n",
      "INFO:tensorflow:loss = 0.034857225, step = 5701 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 316.78\n",
      "INFO:tensorflow:loss = 0.037171274, step = 5801 (0.314 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.394\n",
      "INFO:tensorflow:loss = 0.017138816, step = 5901 (0.290 sec)\n",
      "INFO:tensorflow:global_step/sec: 329.102\n",
      "INFO:tensorflow:loss = 0.030491471, step = 6001 (0.312 sec)\n",
      "INFO:tensorflow:global_step/sec: 349.063\n",
      "INFO:tensorflow:loss = 0.048120163, step = 6101 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 339.279\n",
      "INFO:tensorflow:loss = 0.044583093, step = 6201 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 339.525\n",
      "INFO:tensorflow:loss = 0.04749337, step = 6301 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 334.616\n",
      "INFO:tensorflow:loss = 0.07128422, step = 6401 (0.304 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.25\n",
      "INFO:tensorflow:loss = 0.05821591, step = 6501 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.526\n",
      "INFO:tensorflow:loss = 0.019353827, step = 6601 (0.298 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 359.77\n",
      "INFO:tensorflow:loss = 0.033137664, step = 6701 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.24\n",
      "INFO:tensorflow:loss = 0.021211509, step = 6801 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.291\n",
      "INFO:tensorflow:loss = 0.01806557, step = 6901 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.734\n",
      "INFO:tensorflow:loss = 0.03138727, step = 7001 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.72\n",
      "INFO:tensorflow:loss = 0.03288175, step = 7101 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.23\n",
      "INFO:tensorflow:loss = 0.015384585, step = 7201 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.461\n",
      "INFO:tensorflow:loss = 0.07852742, step = 7301 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.544\n",
      "INFO:tensorflow:loss = 0.037094302, step = 7401 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.76\n",
      "INFO:tensorflow:loss = 0.054601498, step = 7501 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 347.916\n",
      "INFO:tensorflow:loss = 0.020195387, step = 7601 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.736\n",
      "INFO:tensorflow:loss = 0.027472012, step = 7701 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.645\n",
      "INFO:tensorflow:loss = 0.032032155, step = 7801 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 362.047\n",
      "INFO:tensorflow:loss = 0.043274492, step = 7901 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.284\n",
      "INFO:tensorflow:loss = 0.03239342, step = 8001 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.743\n",
      "INFO:tensorflow:loss = 0.027077664, step = 8101 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.366\n",
      "INFO:tensorflow:loss = 0.036092184, step = 8201 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.434\n",
      "INFO:tensorflow:loss = 0.032753907, step = 8301 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 362.738\n",
      "INFO:tensorflow:loss = 0.051111944, step = 8401 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.412\n",
      "INFO:tensorflow:loss = 0.03360961, step = 8501 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 351.43\n",
      "INFO:tensorflow:loss = 0.063037306, step = 8601 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.66\n",
      "INFO:tensorflow:loss = 0.022497725, step = 8701 (0.288 sec)\n",
      "INFO:tensorflow:global_step/sec: 327.437\n",
      "INFO:tensorflow:loss = 0.042914934, step = 8801 (0.310 sec)\n",
      "INFO:tensorflow:global_step/sec: 339.63\n",
      "INFO:tensorflow:loss = 0.07919822, step = 8901 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.739\n",
      "INFO:tensorflow:loss = 0.054977596, step = 9001 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 323.465\n",
      "INFO:tensorflow:loss = 0.030193483, step = 9101 (0.311 sec)\n",
      "INFO:tensorflow:global_step/sec: 316.08\n",
      "INFO:tensorflow:loss = 0.059118994, step = 9201 (0.320 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.025\n",
      "INFO:tensorflow:loss = 0.02894263, step = 9301 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.728\n",
      "INFO:tensorflow:loss = 0.019489076, step = 9401 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.171\n",
      "INFO:tensorflow:loss = 0.036766697, step = 9501 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.379\n",
      "INFO:tensorflow:loss = 0.050935946, step = 9601 (0.299 sec)\n",
      "INFO:tensorflow:global_step/sec: 362.024\n",
      "INFO:tensorflow:loss = 0.017835636, step = 9701 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.924\n",
      "INFO:tensorflow:loss = 0.031217642, step = 9801 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.061\n",
      "INFO:tensorflow:loss = 0.028995764, step = 9901 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 412.432\n",
      "INFO:tensorflow:loss = 0.03132441, step = 10001 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.324\n",
      "INFO:tensorflow:loss = 0.030225202, step = 10101 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 330.44\n",
      "INFO:tensorflow:loss = 0.056032784, step = 10201 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.144\n",
      "INFO:tensorflow:loss = 0.059151538, step = 10301 (0.301 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.318\n",
      "INFO:tensorflow:loss = 0.01907683, step = 10401 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.825\n",
      "INFO:tensorflow:loss = 0.05866129, step = 10501 (0.305 sec)\n",
      "INFO:tensorflow:global_step/sec: 323.308\n",
      "INFO:tensorflow:loss = 0.02594578, step = 10601 (0.307 sec)\n",
      "INFO:tensorflow:global_step/sec: 362.809\n",
      "INFO:tensorflow:loss = 0.021075536, step = 10701 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 247.971\n",
      "INFO:tensorflow:loss = 0.028491914, step = 10801 (0.406 sec)\n",
      "INFO:tensorflow:global_step/sec: 280.637\n",
      "INFO:tensorflow:loss = 0.030953879, step = 10901 (0.353 sec)\n",
      "INFO:tensorflow:global_step/sec: 270.159\n",
      "INFO:tensorflow:loss = 0.01478898, step = 11001 (0.375 sec)\n",
      "INFO:tensorflow:global_step/sec: 286.131\n",
      "INFO:tensorflow:loss = 0.038508505, step = 11101 (0.346 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.246\n",
      "INFO:tensorflow:loss = 0.034574762, step = 11201 (0.316 sec)\n",
      "INFO:tensorflow:global_step/sec: 321.395\n",
      "INFO:tensorflow:loss = 0.054243505, step = 11301 (0.312 sec)\n",
      "INFO:tensorflow:global_step/sec: 298.432\n",
      "INFO:tensorflow:loss = 0.03519901, step = 11401 (0.338 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.945\n",
      "INFO:tensorflow:loss = 0.049500078, step = 11501 (0.316 sec)\n",
      "INFO:tensorflow:global_step/sec: 337.834\n",
      "INFO:tensorflow:loss = 0.03181567, step = 11601 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.841\n",
      "INFO:tensorflow:loss = 0.025518995, step = 11701 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 293.649\n",
      "INFO:tensorflow:loss = 0.022869674, step = 11801 (0.344 sec)\n",
      "INFO:tensorflow:global_step/sec: 323.499\n",
      "INFO:tensorflow:loss = 0.047530904, step = 11901 (0.300 sec)\n",
      "INFO:tensorflow:global_step/sec: 334.644\n",
      "INFO:tensorflow:loss = 0.03789179, step = 12001 (0.303 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.793\n",
      "INFO:tensorflow:loss = 0.017053504, step = 12101 (0.312 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.819\n",
      "INFO:tensorflow:loss = 0.018622763, step = 12201 (0.301 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.189\n",
      "INFO:tensorflow:loss = 0.027168537, step = 12301 (0.316 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.713\n",
      "INFO:tensorflow:loss = 0.05163244, step = 12401 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 326.055\n",
      "INFO:tensorflow:loss = 0.023351505, step = 12501 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 351.339\n",
      "INFO:tensorflow:loss = 0.042466387, step = 12601 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.616\n",
      "INFO:tensorflow:loss = 0.026882555, step = 12701 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.895\n",
      "INFO:tensorflow:loss = 0.04331173, step = 12801 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.194\n",
      "INFO:tensorflow:loss = 0.036629364, step = 12901 (0.303 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.072\n",
      "INFO:tensorflow:loss = 0.04917695, step = 13001 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.834\n",
      "INFO:tensorflow:loss = 0.035332035, step = 13101 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 337.616\n",
      "INFO:tensorflow:loss = 0.030816726, step = 13201 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 326.115\n",
      "INFO:tensorflow:loss = 0.029627517, step = 13301 (0.309 sec)\n",
      "INFO:tensorflow:global_step/sec: 347.751\n",
      "INFO:tensorflow:loss = 0.034982245, step = 13401 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.381\n",
      "INFO:tensorflow:loss = 0.041481517, step = 13501 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.1\n",
      "INFO:tensorflow:loss = 0.016661502, step = 13601 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 345.589\n",
      "INFO:tensorflow:loss = 0.030127164, step = 13701 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.101\n",
      "INFO:tensorflow:loss = 0.0652201, step = 13801 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.369\n",
      "INFO:tensorflow:loss = 0.017221292, step = 13901 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.719\n",
      "INFO:tensorflow:loss = 0.025369555, step = 14001 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 235.885\n",
      "INFO:tensorflow:loss = 0.032870516, step = 14101 (0.425 sec)\n",
      "INFO:tensorflow:global_step/sec: 268.093\n",
      "INFO:tensorflow:loss = 0.04054736, step = 14201 (0.369 sec)\n",
      "INFO:tensorflow:global_step/sec: 277.945\n",
      "INFO:tensorflow:loss = 0.019456197, step = 14301 (0.359 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.139\n",
      "INFO:tensorflow:loss = 0.012983553, step = 14401 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 328.398\n",
      "INFO:tensorflow:loss = 0.0348993, step = 14501 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 340.467\n",
      "INFO:tensorflow:loss = 0.02856581, step = 14601 (0.294 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.527\n",
      "INFO:tensorflow:loss = 0.036568105, step = 14701 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.895\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.027374016, step = 14801 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.692\n",
      "INFO:tensorflow:loss = 0.042642456, step = 14901 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.764\n",
      "INFO:tensorflow:loss = 0.04404347, step = 15001 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.937\n",
      "INFO:tensorflow:loss = 0.014351687, step = 15101 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.372\n",
      "INFO:tensorflow:loss = 0.025546595, step = 15201 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.91\n",
      "INFO:tensorflow:loss = 0.029243704, step = 15301 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.819\n",
      "INFO:tensorflow:loss = 0.020585781, step = 15401 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.96\n",
      "INFO:tensorflow:loss = 0.020710247, step = 15501 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.022\n",
      "INFO:tensorflow:loss = 0.0501779, step = 15601 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 347.191\n",
      "INFO:tensorflow:loss = 0.02641895, step = 15701 (0.290 sec)\n",
      "INFO:tensorflow:global_step/sec: 332.445\n",
      "INFO:tensorflow:loss = 0.051579744, step = 15801 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.901\n",
      "INFO:tensorflow:loss = 0.039642856, step = 15901 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.82\n",
      "INFO:tensorflow:loss = 0.025292177, step = 16001 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.66\n",
      "INFO:tensorflow:loss = 0.035731923, step = 16101 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.674\n",
      "INFO:tensorflow:loss = 0.01624351, step = 16201 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.635\n",
      "INFO:tensorflow:loss = 0.025383838, step = 16301 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.12\n",
      "INFO:tensorflow:loss = 0.059819404, step = 16401 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 303.101\n",
      "INFO:tensorflow:loss = 0.015097598, step = 16501 (0.327 sec)\n",
      "INFO:tensorflow:global_step/sec: 320.215\n",
      "INFO:tensorflow:loss = 0.029063577, step = 16601 (0.312 sec)\n",
      "INFO:tensorflow:global_step/sec: 302.525\n",
      "INFO:tensorflow:loss = 0.026428655, step = 16701 (0.331 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.357\n",
      "INFO:tensorflow:loss = 0.028853396, step = 16801 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.05\n",
      "INFO:tensorflow:loss = 0.025117954, step = 16901 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.999\n",
      "INFO:tensorflow:loss = 0.026971396, step = 17001 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.97\n",
      "INFO:tensorflow:loss = 0.0238612, step = 17101 (0.288 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.295\n",
      "INFO:tensorflow:loss = 0.016916126, step = 17201 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.194\n",
      "INFO:tensorflow:loss = 0.023188967, step = 17301 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.392\n",
      "INFO:tensorflow:loss = 0.014796989, step = 17401 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.063\n",
      "INFO:tensorflow:loss = 0.01962047, step = 17501 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 401.659\n",
      "INFO:tensorflow:loss = 0.024489788, step = 17601 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 349.994\n",
      "INFO:tensorflow:loss = 0.016006421, step = 17701 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 332.945\n",
      "INFO:tensorflow:loss = 0.029417561, step = 17801 (0.300 sec)\n",
      "INFO:tensorflow:global_step/sec: 327.904\n",
      "INFO:tensorflow:loss = 0.031124305, step = 17901 (0.305 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.006\n",
      "INFO:tensorflow:loss = 0.048947696, step = 18001 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.067\n",
      "INFO:tensorflow:loss = 0.027284646, step = 18101 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 362.993\n",
      "INFO:tensorflow:loss = 0.031934697, step = 18201 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.886\n",
      "INFO:tensorflow:loss = 0.037095822, step = 18301 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 347.777\n",
      "INFO:tensorflow:loss = 0.030261882, step = 18401 (0.287 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.526\n",
      "INFO:tensorflow:loss = 0.036680244, step = 18501 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.371\n",
      "INFO:tensorflow:loss = 0.0273229, step = 18601 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 332.821\n",
      "INFO:tensorflow:loss = 0.040832028, step = 18701 (0.300 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.217\n",
      "INFO:tensorflow:loss = 0.009890776, step = 18801 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 355.021\n",
      "INFO:tensorflow:loss = 0.04409997, step = 18901 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 350.713\n",
      "INFO:tensorflow:loss = 0.014017551, step = 19001 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.732\n",
      "INFO:tensorflow:loss = 0.030233305, step = 19101 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.724\n",
      "INFO:tensorflow:loss = 0.024280818, step = 19201 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.867\n",
      "INFO:tensorflow:loss = 0.01900404, step = 19301 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.361\n",
      "INFO:tensorflow:loss = 0.029496638, step = 19401 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 351.883\n",
      "INFO:tensorflow:loss = 0.025667196, step = 19501 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.074\n",
      "INFO:tensorflow:loss = 0.011690065, step = 19601 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 326.886\n",
      "INFO:tensorflow:loss = 0.011366008, step = 19701 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.377\n",
      "INFO:tensorflow:loss = 0.026949065, step = 19801 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.64\n",
      "INFO:tensorflow:loss = 0.03981626, step = 19901 (0.266 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 20000 into /tmp/tmpyxwongpm/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-08:56:47\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-20000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'linear' dict for global step 20000: architecture/adanet/ensembles = b\"\\nW\\n6adanet/iteration_0/ensemble_linear/architecture/adanetB\\x13\\x08\\x07\\x12\\x00B\\r| b'linear' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.049419954, average_loss/adanet/subnetwork = 0.049421385, average_loss/adanet/uniform_average_ensemble = 0.049421385, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.062510915, loss/adanet/subnetwork = 0.062442064, loss/adanet/uniform_average_ensemble = 0.062442064, prediction/mean/adanet/adanet_weighted_ensemble = 3.1072564, prediction/mean/adanet/subnetwork = 3.105895, prediction/mean/adanet/uniform_average_ensemble = 3.105895\n",
      "INFO:tensorflow:Saving candidate '1_layer_dnn' dict for global step 20000: architecture/adanet/ensembles = b\"\\na\\n;adanet/iteration_0/ensemble_1_layer_dnn/architecture/adanetB\\x18\\x08\\x07\\x12\\x00B\\x12| b'1_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.03979707, average_loss/adanet/subnetwork = 0.039825242, average_loss/adanet/uniform_average_ensemble = 0.039825242, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.053206038, loss/adanet/subnetwork = 0.053258948, loss/adanet/uniform_average_ensemble = 0.053258948, prediction/mean/adanet/adanet_weighted_ensemble = 3.1574967, prediction/mean/adanet/subnetwork = 3.157783, prediction/mean/adanet/uniform_average_ensemble = 3.157783\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-19-08:56:49\n",
      "INFO:tensorflow:Saving dict for global step 20000: average_loss = 0.049419954, average_loss/adanet/adanet_weighted_ensemble = 0.049419954, average_loss/adanet/subnetwork = 0.049421385, average_loss/adanet/uniform_average_ensemble = 0.049421385, global_step = 20000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.062510915, loss/adanet/adanet_weighted_ensemble = 0.062510915, loss/adanet/subnetwork = 0.062442064, loss/adanet/uniform_average_ensemble = 0.062442064, prediction/mean = 3.1072564, prediction/mean/adanet/adanet_weighted_ensemble = 3.1072564, prediction/mean/adanet/subnetwork = 3.105895, prediction/mean/adanet/uniform_average_ensemble = 3.105895\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20000: /tmp/tmpyxwongpm/model.ckpt-20000\n",
      "INFO:tensorflow:Loss for final step: 0.050165728.\n",
      "INFO:tensorflow:Starting ensemble evaluation for iteration 0\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-20000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Encountered end of input after 14 evaluations\n",
      "INFO:tensorflow:Computed ensemble metrics: adanet_loss/linear = 0.035082, adanet_loss/1_layer_dnn = 0.035733\n",
      "INFO:tensorflow:Finished ensemble evaluation for iteration 0\n",
      "INFO:tensorflow:'linear' at index 0 is moving onto the next iteration\n",
      "INFO:tensorflow:Freezing best ensemble to /tmp/tmpyxwongpm/frozen/ensemble-0.meta\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-20000\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Overwriting checkpoint with new graph for iteration 1 to /tmp/tmpyxwongpm/model.ckpt-20000\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-20000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Finished training Adanet iteration 0\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 1\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/increment.ckpt-1\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 20000 into /tmp/tmpyxwongpm/model.ckpt.\n",
      "INFO:tensorflow:loss = 0.030963283, step = 20001\n",
      "INFO:tensorflow:global_step/sec: 109.723\n",
      "INFO:tensorflow:loss = 0.03516687, step = 20101 (0.912 sec)\n",
      "INFO:tensorflow:global_step/sec: 319.386\n",
      "INFO:tensorflow:loss = 0.039819483, step = 20201 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.82\n",
      "INFO:tensorflow:loss = 0.033043236, step = 20301 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 345.085\n",
      "INFO:tensorflow:loss = 0.04677619, step = 20401 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 327.833\n",
      "INFO:tensorflow:loss = 0.03045134, step = 20501 (0.303 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.153\n",
      "INFO:tensorflow:loss = 0.041044854, step = 20601 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 339.428\n",
      "INFO:tensorflow:loss = 0.018765172, step = 20701 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 319.489\n",
      "INFO:tensorflow:loss = 0.014937611, step = 20801 (0.314 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.317\n",
      "INFO:tensorflow:loss = 0.031160578, step = 20901 (0.304 sec)\n",
      "INFO:tensorflow:global_step/sec: 339.005\n",
      "INFO:tensorflow:loss = 0.02892076, step = 21001 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.103\n",
      "INFO:tensorflow:loss = 0.032353245, step = 21101 (0.294 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.33\n",
      "INFO:tensorflow:loss = 0.025514646, step = 21201 (0.294 sec)\n",
      "INFO:tensorflow:global_step/sec: 363.753\n",
      "INFO:tensorflow:loss = 0.07582436, step = 21301 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.264\n",
      "INFO:tensorflow:loss = 0.015722122, step = 21401 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.403\n",
      "INFO:tensorflow:loss = 0.019530332, step = 21501 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.407\n",
      "INFO:tensorflow:loss = 0.021188088, step = 21601 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 363.183\n",
      "INFO:tensorflow:loss = 0.03627994, step = 21701 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 340.054\n",
      "INFO:tensorflow:loss = 0.03726136, step = 21801 (0.294 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.491\n",
      "INFO:tensorflow:loss = 0.049223118, step = 21901 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.966\n",
      "INFO:tensorflow:loss = 0.043363333, step = 22001 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 326.737\n",
      "INFO:tensorflow:loss = 0.061846524, step = 22101 (0.307 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.825\n",
      "INFO:tensorflow:loss = 0.024632651, step = 22201 (0.314 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.991\n",
      "INFO:tensorflow:loss = 0.02493213, step = 22301 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.812\n",
      "INFO:tensorflow:loss = 0.021503547, step = 22401 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 369.78\n",
      "INFO:tensorflow:loss = 0.046562873, step = 22501 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.925\n",
      "INFO:tensorflow:loss = 0.03737168, step = 22601 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 319.595\n",
      "INFO:tensorflow:loss = 0.029282672, step = 22701 (0.310 sec)\n",
      "INFO:tensorflow:global_step/sec: 350.539\n",
      "INFO:tensorflow:loss = 0.019854736, step = 22801 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.955\n",
      "INFO:tensorflow:loss = 0.037007652, step = 22901 (0.287 sec)\n",
      "INFO:tensorflow:global_step/sec: 344.881\n",
      "INFO:tensorflow:loss = 0.02780277, step = 23001 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.256\n",
      "INFO:tensorflow:loss = 0.018298808, step = 23101 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 332.12\n",
      "INFO:tensorflow:loss = 0.024072366, step = 23201 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 247.281\n",
      "INFO:tensorflow:loss = 0.040946867, step = 23301 (0.407 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.036\n",
      "INFO:tensorflow:loss = 0.02786, step = 23401 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.98\n",
      "INFO:tensorflow:loss = 0.06477849, step = 23501 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.817\n",
      "INFO:tensorflow:loss = 0.027260639, step = 23601 (0.305 sec)\n",
      "INFO:tensorflow:global_step/sec: 323.501\n",
      "INFO:tensorflow:loss = 0.021556249, step = 23701 (0.306 sec)\n",
      "INFO:tensorflow:global_step/sec: 330.981\n",
      "INFO:tensorflow:loss = 0.02770552, step = 23801 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.831\n",
      "INFO:tensorflow:loss = 0.02642341, step = 23901 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 262.541\n",
      "INFO:tensorflow:loss = 0.048741125, step = 24001 (0.381 sec)\n",
      "INFO:tensorflow:global_step/sec: 311.979\n",
      "INFO:tensorflow:loss = 0.028818483, step = 24101 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.114\n",
      "INFO:tensorflow:loss = 0.020501241, step = 24201 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 310.505\n",
      "INFO:tensorflow:loss = 0.017606102, step = 24301 (0.325 sec)\n",
      "INFO:tensorflow:global_step/sec: 339.244\n",
      "INFO:tensorflow:loss = 0.02282058, step = 24401 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 324.725\n",
      "INFO:tensorflow:loss = 0.042412892, step = 24501 (0.314 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.791\n",
      "INFO:tensorflow:loss = 0.021762803, step = 24601 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 347.824\n",
      "INFO:tensorflow:loss = 0.07007265, step = 24701 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.856\n",
      "INFO:tensorflow:loss = 0.029353665, step = 24801 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 349.545\n",
      "INFO:tensorflow:loss = 0.027338864, step = 24901 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 329.455\n",
      "INFO:tensorflow:loss = 0.022534298, step = 25001 (0.306 sec)\n",
      "INFO:tensorflow:global_step/sec: 318.612\n",
      "INFO:tensorflow:loss = 0.036688305, step = 25101 (0.313 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.918\n",
      "INFO:tensorflow:loss = 0.022969173, step = 25201 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.668\n",
      "INFO:tensorflow:loss = 0.06660849, step = 25301 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.952\n",
      "INFO:tensorflow:loss = 0.019454077, step = 25401 (0.316 sec)\n",
      "INFO:tensorflow:global_step/sec: 320.463\n",
      "INFO:tensorflow:loss = 0.020998778, step = 25501 (0.311 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.325\n",
      "INFO:tensorflow:loss = 0.026941266, step = 25601 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 324.867\n",
      "INFO:tensorflow:loss = 0.031860575, step = 25701 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 303.888\n",
      "INFO:tensorflow:loss = 0.030884288, step = 25801 (0.329 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.087\n",
      "INFO:tensorflow:loss = 0.013250217, step = 25901 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.356\n",
      "INFO:tensorflow:loss = 0.02678251, step = 26001 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.325\n",
      "INFO:tensorflow:loss = 0.043618813, step = 26101 (0.305 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 338.478\n",
      "INFO:tensorflow:loss = 0.037047483, step = 26201 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 346.557\n",
      "INFO:tensorflow:loss = 0.042897377, step = 26301 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.775\n",
      "INFO:tensorflow:loss = 0.0555079, step = 26401 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.643\n",
      "INFO:tensorflow:loss = 0.04718653, step = 26501 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 350.334\n",
      "INFO:tensorflow:loss = 0.020758241, step = 26601 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 355.577\n",
      "INFO:tensorflow:loss = 0.032562613, step = 26701 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.237\n",
      "INFO:tensorflow:loss = 0.02193155, step = 26801 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 362.032\n",
      "INFO:tensorflow:loss = 0.013505356, step = 26901 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.729\n",
      "INFO:tensorflow:loss = 0.024709681, step = 27001 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 345.474\n",
      "INFO:tensorflow:loss = 0.022348955, step = 27101 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.043\n",
      "INFO:tensorflow:loss = 0.01634366, step = 27201 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.753\n",
      "INFO:tensorflow:loss = 0.06257349, step = 27301 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.977\n",
      "INFO:tensorflow:loss = 0.03558763, step = 27401 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.272\n",
      "INFO:tensorflow:loss = 0.03818273, step = 27501 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.954\n",
      "INFO:tensorflow:loss = 0.014761034, step = 27601 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.597\n",
      "INFO:tensorflow:loss = 0.02446878, step = 27701 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.1\n",
      "INFO:tensorflow:loss = 0.027478002, step = 27801 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.143\n",
      "INFO:tensorflow:loss = 0.029287715, step = 27901 (0.287 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.769\n",
      "INFO:tensorflow:loss = 0.028243115, step = 28001 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.023\n",
      "INFO:tensorflow:loss = 0.024491819, step = 28101 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 330.091\n",
      "INFO:tensorflow:loss = 0.030791353, step = 28201 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.965\n",
      "INFO:tensorflow:loss = 0.025394369, step = 28301 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.545\n",
      "INFO:tensorflow:loss = 0.035995137, step = 28401 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 349.909\n",
      "INFO:tensorflow:loss = 0.025944877, step = 28501 (0.288 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.332\n",
      "INFO:tensorflow:loss = 0.04521303, step = 28601 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 344.883\n",
      "INFO:tensorflow:loss = 0.021536035, step = 28701 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.862\n",
      "INFO:tensorflow:loss = 0.036470562, step = 28801 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.954\n",
      "INFO:tensorflow:loss = 0.06433473, step = 28901 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.395\n",
      "INFO:tensorflow:loss = 0.046331074, step = 29001 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.331\n",
      "INFO:tensorflow:loss = 0.025070682, step = 29101 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 329.976\n",
      "INFO:tensorflow:loss = 0.046682548, step = 29201 (0.305 sec)\n",
      "INFO:tensorflow:global_step/sec: 346.012\n",
      "INFO:tensorflow:loss = 0.019442305, step = 29301 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 330.696\n",
      "INFO:tensorflow:loss = 0.013401756, step = 29401 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 328.984\n",
      "INFO:tensorflow:loss = 0.06083915, step = 29501 (0.307 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.396\n",
      "INFO:tensorflow:loss = 0.040679537, step = 29601 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 302.139\n",
      "INFO:tensorflow:loss = 0.01363329, step = 29701 (0.330 sec)\n",
      "INFO:tensorflow:global_step/sec: 307.827\n",
      "INFO:tensorflow:loss = 0.025342561, step = 29801 (0.327 sec)\n",
      "INFO:tensorflow:global_step/sec: 306.696\n",
      "INFO:tensorflow:loss = 0.022590734, step = 29901 (0.330 sec)\n",
      "INFO:tensorflow:global_step/sec: 292.859\n",
      "INFO:tensorflow:loss = 0.02721703, step = 30001 (0.335 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.471\n",
      "INFO:tensorflow:loss = 0.02375856, step = 30101 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 215.546\n",
      "INFO:tensorflow:loss = 0.04613744, step = 30201 (0.462 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.08\n",
      "INFO:tensorflow:loss = 0.045555316, step = 30301 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.825\n",
      "INFO:tensorflow:loss = 0.018026374, step = 30401 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 298.277\n",
      "INFO:tensorflow:loss = 0.043185968, step = 30501 (0.334 sec)\n",
      "INFO:tensorflow:global_step/sec: 290.73\n",
      "INFO:tensorflow:loss = 0.021179901, step = 30601 (0.345 sec)\n",
      "INFO:tensorflow:global_step/sec: 308.473\n",
      "INFO:tensorflow:loss = 0.018322831, step = 30701 (0.326 sec)\n",
      "INFO:tensorflow:global_step/sec: 309.666\n",
      "INFO:tensorflow:loss = 0.023421433, step = 30801 (0.324 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.674\n",
      "INFO:tensorflow:loss = 0.027271576, step = 30901 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 339.025\n",
      "INFO:tensorflow:loss = 0.012849338, step = 31001 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 316.141\n",
      "INFO:tensorflow:loss = 0.029278755, step = 31101 (0.319 sec)\n",
      "INFO:tensorflow:global_step/sec: 229.563\n",
      "INFO:tensorflow:loss = 0.027006434, step = 31201 (0.434 sec)\n",
      "INFO:tensorflow:global_step/sec: 311.552\n",
      "INFO:tensorflow:loss = 0.04422751, step = 31301 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 286.729\n",
      "INFO:tensorflow:loss = 0.024532497, step = 31401 (0.350 sec)\n",
      "INFO:tensorflow:global_step/sec: 248.977\n",
      "INFO:tensorflow:loss = 0.039801225, step = 31501 (0.397 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.855\n",
      "INFO:tensorflow:loss = 0.025307052, step = 31601 (0.317 sec)\n",
      "INFO:tensorflow:global_step/sec: 231.463\n",
      "INFO:tensorflow:loss = 0.02235778, step = 31701 (0.432 sec)\n",
      "INFO:tensorflow:global_step/sec: 206.63\n",
      "INFO:tensorflow:loss = 0.017295212, step = 31801 (0.485 sec)\n",
      "INFO:tensorflow:global_step/sec: 294.357\n",
      "INFO:tensorflow:loss = 0.04110069, step = 31901 (0.338 sec)\n",
      "INFO:tensorflow:global_step/sec: 308.227\n",
      "INFO:tensorflow:loss = 0.030137457, step = 32001 (0.324 sec)\n",
      "INFO:tensorflow:global_step/sec: 290.827\n",
      "INFO:tensorflow:loss = 0.011669657, step = 32101 (0.344 sec)\n",
      "INFO:tensorflow:global_step/sec: 325.82\n",
      "INFO:tensorflow:loss = 0.013874157, step = 32201 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 329.877\n",
      "INFO:tensorflow:loss = 0.02152882, step = 32301 (0.304 sec)\n",
      "INFO:tensorflow:global_step/sec: 215.961\n",
      "INFO:tensorflow:loss = 0.03557998, step = 32401 (0.463 sec)\n",
      "INFO:tensorflow:global_step/sec: 202.996\n",
      "INFO:tensorflow:loss = 0.025712108, step = 32501 (0.493 sec)\n",
      "INFO:tensorflow:global_step/sec: 265.515\n",
      "INFO:tensorflow:loss = 0.03407304, step = 32601 (0.382 sec)\n",
      "INFO:tensorflow:global_step/sec: 242.137\n",
      "INFO:tensorflow:loss = 0.021758609, step = 32701 (0.410 sec)\n",
      "INFO:tensorflow:global_step/sec: 269.835\n",
      "INFO:tensorflow:loss = 0.031030726, step = 32801 (0.367 sec)\n",
      "INFO:tensorflow:global_step/sec: 285.569\n",
      "INFO:tensorflow:loss = 0.029809276, step = 32901 (0.353 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.05\n",
      "INFO:tensorflow:loss = 0.036491204, step = 33001 (0.313 sec)\n",
      "INFO:tensorflow:global_step/sec: 321.165\n",
      "INFO:tensorflow:loss = 0.029611541, step = 33101 (0.311 sec)\n",
      "INFO:tensorflow:global_step/sec: 253.343\n",
      "INFO:tensorflow:loss = 0.021685425, step = 33201 (0.395 sec)\n",
      "INFO:tensorflow:global_step/sec: 233.024\n",
      "INFO:tensorflow:loss = 0.022499632, step = 33301 (0.427 sec)\n",
      "INFO:tensorflow:global_step/sec: 181.324\n",
      "INFO:tensorflow:loss = 0.029690256, step = 33401 (0.556 sec)\n",
      "INFO:tensorflow:global_step/sec: 256.3\n",
      "INFO:tensorflow:loss = 0.035334997, step = 33501 (0.393 sec)\n",
      "INFO:tensorflow:global_step/sec: 180.076\n",
      "INFO:tensorflow:loss = 0.014564889, step = 33601 (0.550 sec)\n",
      "INFO:tensorflow:global_step/sec: 225.56\n",
      "INFO:tensorflow:loss = 0.017262174, step = 33701 (0.443 sec)\n",
      "INFO:tensorflow:global_step/sec: 171.543\n",
      "INFO:tensorflow:loss = 0.05118932, step = 33801 (0.583 sec)\n",
      "INFO:tensorflow:global_step/sec: 355.468\n",
      "INFO:tensorflow:loss = 0.015073013, step = 33901 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.717\n",
      "INFO:tensorflow:loss = 0.018487962, step = 34001 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 332.103\n",
      "INFO:tensorflow:loss = 0.028050141, step = 34101 (0.300 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.513\n",
      "INFO:tensorflow:loss = 0.029291354, step = 34201 (0.284 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 252.9\n",
      "INFO:tensorflow:loss = 0.035877682, step = 34301 (0.389 sec)\n",
      "INFO:tensorflow:global_step/sec: 285.33\n",
      "INFO:tensorflow:loss = 0.019192742, step = 34401 (0.352 sec)\n",
      "INFO:tensorflow:global_step/sec: 329.339\n",
      "INFO:tensorflow:loss = 0.027362991, step = 34501 (0.303 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.933\n",
      "INFO:tensorflow:loss = 0.035525072, step = 34601 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.012\n",
      "INFO:tensorflow:loss = 0.03035288, step = 34701 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 81.626\n",
      "INFO:tensorflow:loss = 0.04075265, step = 34801 (1.226 sec)\n",
      "INFO:tensorflow:global_step/sec: 96.802\n",
      "INFO:tensorflow:loss = 0.05653507, step = 34901 (1.050 sec)\n",
      "INFO:tensorflow:global_step/sec: 134.492\n",
      "INFO:tensorflow:loss = 0.036090188, step = 35001 (0.725 sec)\n",
      "INFO:tensorflow:global_step/sec: 158.743\n",
      "INFO:tensorflow:loss = 0.016563129, step = 35101 (0.631 sec)\n",
      "INFO:tensorflow:global_step/sec: 160.766\n",
      "INFO:tensorflow:loss = 0.023504052, step = 35201 (0.633 sec)\n",
      "INFO:tensorflow:global_step/sec: 128.336\n",
      "INFO:tensorflow:loss = 0.026475579, step = 35301 (0.767 sec)\n",
      "INFO:tensorflow:global_step/sec: 212.78\n",
      "INFO:tensorflow:loss = 0.015348977, step = 35401 (0.470 sec)\n",
      "INFO:tensorflow:global_step/sec: 153.242\n",
      "INFO:tensorflow:loss = 0.039254375, step = 35501 (0.662 sec)\n",
      "INFO:tensorflow:global_step/sec: 227.981\n",
      "INFO:tensorflow:loss = 0.039965983, step = 35601 (0.432 sec)\n",
      "INFO:tensorflow:global_step/sec: 254.842\n",
      "INFO:tensorflow:loss = 0.051794212, step = 35701 (0.393 sec)\n",
      "INFO:tensorflow:global_step/sec: 312.228\n",
      "INFO:tensorflow:loss = 0.041655824, step = 35801 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.677\n",
      "INFO:tensorflow:loss = 0.033177037, step = 35901 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 271.369\n",
      "INFO:tensorflow:loss = 0.040016007, step = 36001 (0.366 sec)\n",
      "INFO:tensorflow:global_step/sec: 158.033\n",
      "INFO:tensorflow:loss = 0.030527392, step = 36101 (0.633 sec)\n",
      "INFO:tensorflow:global_step/sec: 145.836\n",
      "INFO:tensorflow:loss = 0.015830696, step = 36201 (0.696 sec)\n",
      "INFO:tensorflow:global_step/sec: 175.041\n",
      "INFO:tensorflow:loss = 0.026036523, step = 36301 (0.561 sec)\n",
      "INFO:tensorflow:global_step/sec: 203.391\n",
      "INFO:tensorflow:loss = 0.043933537, step = 36401 (0.496 sec)\n",
      "INFO:tensorflow:global_step/sec: 99.1193\n",
      "INFO:tensorflow:loss = 0.03373274, step = 36501 (1.038 sec)\n",
      "INFO:tensorflow:global_step/sec: 171.33\n",
      "INFO:tensorflow:loss = 0.022932356, step = 36601 (0.551 sec)\n",
      "INFO:tensorflow:global_step/sec: 203.558\n",
      "INFO:tensorflow:loss = 0.030274125, step = 36701 (0.495 sec)\n",
      "INFO:tensorflow:global_step/sec: 323.846\n",
      "INFO:tensorflow:loss = 0.028169844, step = 36801 (0.305 sec)\n",
      "INFO:tensorflow:global_step/sec: 224.601\n",
      "INFO:tensorflow:loss = 0.01876313, step = 36901 (0.449 sec)\n",
      "INFO:tensorflow:global_step/sec: 205.154\n",
      "INFO:tensorflow:loss = 0.032192342, step = 37001 (0.491 sec)\n",
      "INFO:tensorflow:global_step/sec: 167.202\n",
      "INFO:tensorflow:loss = 0.032071576, step = 37101 (0.591 sec)\n",
      "INFO:tensorflow:global_step/sec: 181.013\n",
      "INFO:tensorflow:loss = 0.014490783, step = 37201 (0.553 sec)\n",
      "INFO:tensorflow:global_step/sec: 229.467\n",
      "INFO:tensorflow:loss = 0.030645829, step = 37301 (0.443 sec)\n",
      "INFO:tensorflow:global_step/sec: 255.906\n",
      "INFO:tensorflow:loss = 0.025206108, step = 37401 (0.386 sec)\n",
      "INFO:tensorflow:global_step/sec: 269.14\n",
      "INFO:tensorflow:loss = 0.030304598, step = 37501 (0.373 sec)\n",
      "INFO:tensorflow:global_step/sec: 267.052\n",
      "INFO:tensorflow:loss = 0.039137974, step = 37601 (0.378 sec)\n",
      "INFO:tensorflow:global_step/sec: 251.786\n",
      "INFO:tensorflow:loss = 0.029267129, step = 37701 (0.402 sec)\n",
      "INFO:tensorflow:global_step/sec: 248.205\n",
      "INFO:tensorflow:loss = 0.026941095, step = 37801 (0.393 sec)\n",
      "INFO:tensorflow:global_step/sec: 241.978\n",
      "INFO:tensorflow:loss = 0.022023236, step = 37901 (0.412 sec)\n",
      "INFO:tensorflow:global_step/sec: 177.01\n",
      "INFO:tensorflow:loss = 0.039564054, step = 38001 (0.564 sec)\n",
      "INFO:tensorflow:global_step/sec: 185.259\n",
      "INFO:tensorflow:loss = 0.024186868, step = 38101 (0.547 sec)\n",
      "INFO:tensorflow:global_step/sec: 178.536\n",
      "INFO:tensorflow:loss = 0.024910225, step = 38201 (0.557 sec)\n",
      "INFO:tensorflow:global_step/sec: 288.379\n",
      "INFO:tensorflow:loss = 0.028100446, step = 38301 (0.347 sec)\n",
      "INFO:tensorflow:global_step/sec: 323.505\n",
      "INFO:tensorflow:loss = 0.04427521, step = 38401 (0.307 sec)\n",
      "INFO:tensorflow:global_step/sec: 238.44\n",
      "INFO:tensorflow:loss = 0.028431065, step = 38501 (0.426 sec)\n",
      "INFO:tensorflow:global_step/sec: 218.721\n",
      "INFO:tensorflow:loss = 0.02366386, step = 38601 (0.468 sec)\n",
      "INFO:tensorflow:global_step/sec: 274.091\n",
      "INFO:tensorflow:loss = 0.03591672, step = 38701 (0.353 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.73\n",
      "INFO:tensorflow:loss = 0.023343278, step = 38801 (0.311 sec)\n",
      "INFO:tensorflow:global_step/sec: 290.365\n",
      "INFO:tensorflow:loss = 0.032691233, step = 38901 (0.346 sec)\n",
      "INFO:tensorflow:global_step/sec: 187.732\n",
      "INFO:tensorflow:loss = 0.018502701, step = 39001 (0.534 sec)\n",
      "INFO:tensorflow:global_step/sec: 314.241\n",
      "INFO:tensorflow:loss = 0.041494712, step = 39101 (0.317 sec)\n",
      "INFO:tensorflow:global_step/sec: 280.409\n",
      "INFO:tensorflow:loss = 0.041908644, step = 39201 (0.355 sec)\n",
      "INFO:tensorflow:global_step/sec: 294.289\n",
      "INFO:tensorflow:loss = 0.018903662, step = 39301 (0.339 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.318\n",
      "INFO:tensorflow:loss = 0.022484206, step = 39401 (0.315 sec)\n",
      "INFO:tensorflow:global_step/sec: 329.414\n",
      "INFO:tensorflow:loss = 0.019705646, step = 39501 (0.304 sec)\n",
      "INFO:tensorflow:global_step/sec: 304.467\n",
      "INFO:tensorflow:loss = 0.01487639, step = 39601 (0.329 sec)\n",
      "INFO:tensorflow:global_step/sec: 309.506\n",
      "INFO:tensorflow:loss = 0.020100601, step = 39701 (0.325 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.497\n",
      "INFO:tensorflow:loss = 0.022227569, step = 39801 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 344.419\n",
      "INFO:tensorflow:loss = 0.032599714, step = 39901 (0.291 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 40000 into /tmp/tmpyxwongpm/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-08:58:17\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-40000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'previous_ensemble' dict for global step 40000: architecture/adanet/ensembles = b\"\\nM\\n,adanet/previous_ensemble/architecture/adanetB\\x13\\x08\\x07\\x12\\x00B\\r| b'linear' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.049419954, average_loss/adanet/subnetwork = 0.049421385, average_loss/adanet/uniform_average_ensemble = 0.049421385, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.062510915, loss/adanet/subnetwork = 0.062442064, loss/adanet/uniform_average_ensemble = 0.062442064, prediction/mean/adanet/adanet_weighted_ensemble = 3.1072564, prediction/mean/adanet/subnetwork = 3.105895, prediction/mean/adanet/uniform_average_ensemble = 3.105895\n",
      "INFO:tensorflow:Saving candidate 'linear' dict for global step 40000: architecture/adanet/ensembles = b\"\\nc\\n6adanet/iteration_1/ensemble_linear/architecture/adanetB\\x1f\\x08\\x07\\x12\\x00B\\x19| b'linear' | b'linear' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.04997501, average_loss/adanet/subnetwork = 0.05393383, average_loss/adanet/uniform_average_ensemble = 0.051507678, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.06265795, loss/adanet/subnetwork = 0.06527223, loss/adanet/uniform_average_ensemble = 0.06367771, prediction/mean/adanet/adanet_weighted_ensemble = 3.103242, prediction/mean/adanet/subnetwork = 3.1013088, prediction/mean/adanet/uniform_average_ensemble = 3.1036017\n",
      "INFO:tensorflow:Saving candidate '1_layer_dnn' dict for global step 40000: architecture/adanet/ensembles = b\"\\nm\\n;adanet/iteration_1/ensemble_1_layer_dnn/architecture/adanetB$\\x08\\x07\\x12\\x00B\\x1e| b'linear' | b'1_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.04485139, average_loss/adanet/subnetwork = 0.04407932, average_loss/adanet/uniform_average_ensemble = 0.043675844, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.059158117, loss/adanet/subnetwork = 0.061847888, loss/adanet/uniform_average_ensemble = 0.0585649, prediction/mean/adanet/adanet_weighted_ensemble = 3.1251307, prediction/mean/adanet/subnetwork = 3.1570365, prediction/mean/adanet/uniform_average_ensemble = 3.131466\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Finished evaluation at 2018-11-19-08:58:19\n",
      "INFO:tensorflow:Saving dict for global step 40000: average_loss = 0.04485139, average_loss/adanet/adanet_weighted_ensemble = 0.04485139, average_loss/adanet/subnetwork = 0.04407932, average_loss/adanet/uniform_average_ensemble = 0.043675844, global_step = 40000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.059158117, loss/adanet/adanet_weighted_ensemble = 0.059158117, loss/adanet/subnetwork = 0.061847888, loss/adanet/uniform_average_ensemble = 0.0585649, prediction/mean = 3.1251307, prediction/mean/adanet/adanet_weighted_ensemble = 3.1251307, prediction/mean/adanet/subnetwork = 3.1570365, prediction/mean/adanet/uniform_average_ensemble = 3.131466\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 40000: /tmp/tmpyxwongpm/model.ckpt-40000\n",
      "INFO:tensorflow:Loss for final step: 0.039867282.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Starting ensemble evaluation for iteration 1\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-40000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Encountered end of input after 14 evaluations\n",
      "INFO:tensorflow:Computed ensemble metrics: adanet_loss/previous_ensemble = 0.035082, adanet_loss/linear = 0.035063, adanet_loss/1_layer_dnn = 0.032305\n",
      "INFO:tensorflow:Finished ensemble evaluation for iteration 1\n",
      "INFO:tensorflow:'1_layer_dnn' at index 2 is moving onto the next iteration\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-0.meta with features: ['x'].\n",
      "INFO:tensorflow:Freezing best ensemble to /tmp/tmpyxwongpm/frozen/ensemble-1.meta\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-40000\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Overwriting checkpoint with new graph for iteration 2 to /tmp/tmpyxwongpm/model.ckpt-40000\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-40000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Finished training Adanet iteration 1\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 2\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/increment.ckpt-2\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 40000 into /tmp/tmpyxwongpm/model.ckpt.\n",
      "INFO:tensorflow:loss = 0.025989912, step = 40001\n",
      "INFO:tensorflow:global_step/sec: 90.37\n",
      "INFO:tensorflow:loss = 0.02916801, step = 40101 (1.113 sec)\n",
      "INFO:tensorflow:global_step/sec: 297.703\n",
      "INFO:tensorflow:loss = 0.027761856, step = 40201 (0.330 sec)\n",
      "INFO:tensorflow:global_step/sec: 277.747\n",
      "INFO:tensorflow:loss = 0.025273405, step = 40301 (0.362 sec)\n",
      "INFO:tensorflow:global_step/sec: 241.833\n",
      "INFO:tensorflow:loss = 0.038389083, step = 40401 (0.412 sec)\n",
      "INFO:tensorflow:global_step/sec: 266.325\n",
      "INFO:tensorflow:loss = 0.022687316, step = 40501 (0.378 sec)\n",
      "INFO:tensorflow:global_step/sec: 321.526\n",
      "INFO:tensorflow:loss = 0.030614233, step = 40601 (0.311 sec)\n",
      "INFO:tensorflow:global_step/sec: 284.44\n",
      "INFO:tensorflow:loss = 0.015421159, step = 40701 (0.354 sec)\n",
      "INFO:tensorflow:global_step/sec: 278.969\n",
      "INFO:tensorflow:loss = 0.013811226, step = 40801 (0.358 sec)\n",
      "INFO:tensorflow:global_step/sec: 290.788\n",
      "INFO:tensorflow:loss = 0.023330316, step = 40901 (0.345 sec)\n",
      "INFO:tensorflow:global_step/sec: 288.743\n",
      "INFO:tensorflow:loss = 0.026442524, step = 41001 (0.346 sec)\n",
      "INFO:tensorflow:global_step/sec: 285.642\n",
      "INFO:tensorflow:loss = 0.025592014, step = 41101 (0.346 sec)\n",
      "INFO:tensorflow:global_step/sec: 277.907\n",
      "INFO:tensorflow:loss = 0.023115601, step = 41201 (0.360 sec)\n",
      "INFO:tensorflow:global_step/sec: 297.768\n",
      "INFO:tensorflow:loss = 0.05770303, step = 41301 (0.334 sec)\n",
      "INFO:tensorflow:global_step/sec: 295.453\n",
      "INFO:tensorflow:loss = 0.01142644, step = 41401 (0.339 sec)\n",
      "INFO:tensorflow:global_step/sec: 283.759\n",
      "INFO:tensorflow:loss = 0.014758332, step = 41501 (0.354 sec)\n",
      "INFO:tensorflow:global_step/sec: 293.305\n",
      "INFO:tensorflow:loss = 0.019491425, step = 41601 (0.339 sec)\n",
      "INFO:tensorflow:global_step/sec: 300.111\n",
      "INFO:tensorflow:loss = 0.029118739, step = 41701 (0.340 sec)\n",
      "INFO:tensorflow:global_step/sec: 281.369\n",
      "INFO:tensorflow:loss = 0.027844112, step = 41801 (0.349 sec)\n",
      "INFO:tensorflow:global_step/sec: 290.471\n",
      "INFO:tensorflow:loss = 0.036816474, step = 41901 (0.345 sec)\n",
      "INFO:tensorflow:global_step/sec: 298.159\n",
      "INFO:tensorflow:loss = 0.02622139, step = 42001 (0.335 sec)\n",
      "INFO:tensorflow:global_step/sec: 294.541\n",
      "INFO:tensorflow:loss = 0.04619189, step = 42101 (0.341 sec)\n",
      "INFO:tensorflow:global_step/sec: 291.067\n",
      "INFO:tensorflow:loss = 0.019975033, step = 42201 (0.343 sec)\n",
      "INFO:tensorflow:global_step/sec: 282.039\n",
      "INFO:tensorflow:loss = 0.01888156, step = 42301 (0.357 sec)\n",
      "INFO:tensorflow:global_step/sec: 283.492\n",
      "INFO:tensorflow:loss = 0.01945632, step = 42401 (0.349 sec)\n",
      "INFO:tensorflow:global_step/sec: 192.471\n",
      "INFO:tensorflow:loss = 0.036027957, step = 42501 (0.528 sec)\n",
      "INFO:tensorflow:global_step/sec: 261.206\n",
      "INFO:tensorflow:loss = 0.027110588, step = 42601 (0.375 sec)\n",
      "INFO:tensorflow:global_step/sec: 207.775\n",
      "INFO:tensorflow:loss = 0.023873731, step = 42701 (0.485 sec)\n",
      "INFO:tensorflow:global_step/sec: 297.374\n",
      "INFO:tensorflow:loss = 0.014402544, step = 42801 (0.336 sec)\n",
      "INFO:tensorflow:global_step/sec: 308.774\n",
      "INFO:tensorflow:loss = 0.028773395, step = 42901 (0.325 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.548\n",
      "INFO:tensorflow:loss = 0.018499963, step = 43001 (0.304 sec)\n",
      "INFO:tensorflow:global_step/sec: 302.412\n",
      "INFO:tensorflow:loss = 0.014945269, step = 43101 (0.331 sec)\n",
      "INFO:tensorflow:global_step/sec: 261.602\n",
      "INFO:tensorflow:loss = 0.024769694, step = 43201 (0.383 sec)\n",
      "INFO:tensorflow:global_step/sec: 244.282\n",
      "INFO:tensorflow:loss = 0.03421632, step = 43301 (0.408 sec)\n",
      "INFO:tensorflow:global_step/sec: 268.562\n",
      "INFO:tensorflow:loss = 0.01957949, step = 43401 (0.376 sec)\n",
      "INFO:tensorflow:global_step/sec: 287.353\n",
      "INFO:tensorflow:loss = 0.047873802, step = 43501 (0.345 sec)\n",
      "INFO:tensorflow:global_step/sec: 299.714\n",
      "INFO:tensorflow:loss = 0.01991491, step = 43601 (0.333 sec)\n",
      "INFO:tensorflow:global_step/sec: 312.115\n",
      "INFO:tensorflow:loss = 0.015646711, step = 43701 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 309.96\n",
      "INFO:tensorflow:loss = 0.01600581, step = 43801 (0.322 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.573\n",
      "INFO:tensorflow:loss = 0.017738655, step = 43901 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 351.25\n",
      "INFO:tensorflow:loss = 0.034148037, step = 44001 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.003\n",
      "INFO:tensorflow:loss = 0.024104277, step = 44101 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.177\n",
      "INFO:tensorflow:loss = 0.012464296, step = 44201 (0.301 sec)\n",
      "INFO:tensorflow:global_step/sec: 319.276\n",
      "INFO:tensorflow:loss = 0.010506324, step = 44301 (0.313 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.467\n",
      "INFO:tensorflow:loss = 0.014509417, step = 44401 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 328.297\n",
      "INFO:tensorflow:loss = 0.027423803, step = 44501 (0.304 sec)\n",
      "INFO:tensorflow:global_step/sec: 258.6\n",
      "INFO:tensorflow:loss = 0.017840602, step = 44601 (0.387 sec)\n",
      "INFO:tensorflow:global_step/sec: 283.931\n",
      "INFO:tensorflow:loss = 0.0439362, step = 44701 (0.352 sec)\n",
      "INFO:tensorflow:global_step/sec: 179.977\n",
      "INFO:tensorflow:loss = 0.017008435, step = 44801 (0.556 sec)\n",
      "INFO:tensorflow:global_step/sec: 247.828\n",
      "INFO:tensorflow:loss = 0.020424526, step = 44901 (0.410 sec)\n",
      "INFO:tensorflow:global_step/sec: 260.792\n",
      "INFO:tensorflow:loss = 0.014782762, step = 45001 (0.380 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 268.541\n",
      "INFO:tensorflow:loss = 0.017422926, step = 45101 (0.372 sec)\n",
      "INFO:tensorflow:global_step/sec: 314.142\n",
      "INFO:tensorflow:loss = 0.015257321, step = 45201 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 289.481\n",
      "INFO:tensorflow:loss = 0.041434586, step = 45301 (0.342 sec)\n",
      "INFO:tensorflow:global_step/sec: 306.188\n",
      "INFO:tensorflow:loss = 0.015478523, step = 45401 (0.325 sec)\n",
      "INFO:tensorflow:global_step/sec: 305.61\n",
      "INFO:tensorflow:loss = 0.018797157, step = 45501 (0.328 sec)\n",
      "INFO:tensorflow:global_step/sec: 300.935\n",
      "INFO:tensorflow:loss = 0.021855002, step = 45601 (0.335 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.617\n",
      "INFO:tensorflow:loss = 0.023638379, step = 45701 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 325.89\n",
      "INFO:tensorflow:loss = 0.019294918, step = 45801 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.821\n",
      "INFO:tensorflow:loss = 0.008947721, step = 45901 (0.315 sec)\n",
      "INFO:tensorflow:global_step/sec: 321.599\n",
      "INFO:tensorflow:loss = 0.019638894, step = 46001 (0.310 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.075\n",
      "INFO:tensorflow:loss = 0.028348397, step = 46101 (0.300 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.03\n",
      "INFO:tensorflow:loss = 0.027475027, step = 46201 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.152\n",
      "INFO:tensorflow:loss = 0.02849198, step = 46301 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 326.941\n",
      "INFO:tensorflow:loss = 0.040103983, step = 46401 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.172\n",
      "INFO:tensorflow:loss = 0.03604328, step = 46501 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.488\n",
      "INFO:tensorflow:loss = 0.01593806, step = 46601 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 320.321\n",
      "INFO:tensorflow:loss = 0.02252808, step = 46701 (0.312 sec)\n",
      "INFO:tensorflow:global_step/sec: 328.121\n",
      "INFO:tensorflow:loss = 0.016630419, step = 46801 (0.310 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.355\n",
      "INFO:tensorflow:loss = 0.012053035, step = 46901 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 330.408\n",
      "INFO:tensorflow:loss = 0.023152176, step = 47001 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 325.049\n",
      "INFO:tensorflow:loss = 0.016617747, step = 47101 (0.310 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.798\n",
      "INFO:tensorflow:loss = 0.014195122, step = 47201 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.65\n",
      "INFO:tensorflow:loss = 0.046592146, step = 47301 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.16\n",
      "INFO:tensorflow:loss = 0.021329794, step = 47401 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.162\n",
      "INFO:tensorflow:loss = 0.025823805, step = 47501 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.34\n",
      "INFO:tensorflow:loss = 0.012041953, step = 47601 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.594\n",
      "INFO:tensorflow:loss = 0.021629432, step = 47701 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 301.87\n",
      "INFO:tensorflow:loss = 0.024317365, step = 47801 (0.336 sec)\n",
      "INFO:tensorflow:global_step/sec: 297.855\n",
      "INFO:tensorflow:loss = 0.019899331, step = 47901 (0.331 sec)\n",
      "INFO:tensorflow:global_step/sec: 326.931\n",
      "INFO:tensorflow:loss = 0.02293679, step = 48001 (0.305 sec)\n",
      "INFO:tensorflow:global_step/sec: 323.219\n",
      "INFO:tensorflow:loss = 0.021883659, step = 48101 (0.314 sec)\n",
      "INFO:tensorflow:global_step/sec: 312.513\n",
      "INFO:tensorflow:loss = 0.027169945, step = 48201 (0.319 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.683\n",
      "INFO:tensorflow:loss = 0.020881958, step = 48301 (0.300 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.192\n",
      "INFO:tensorflow:loss = 0.025936997, step = 48401 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 345.818\n",
      "INFO:tensorflow:loss = 0.017294597, step = 48501 (0.290 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.889\n",
      "INFO:tensorflow:loss = 0.03375015, step = 48601 (0.290 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.207\n",
      "INFO:tensorflow:loss = 0.018892322, step = 48701 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 355.109\n",
      "INFO:tensorflow:loss = 0.029176328, step = 48801 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.518\n",
      "INFO:tensorflow:loss = 0.049957052, step = 48901 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 313.776\n",
      "INFO:tensorflow:loss = 0.0377938, step = 49001 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 312.317\n",
      "INFO:tensorflow:loss = 0.019248191, step = 49101 (0.319 sec)\n",
      "INFO:tensorflow:global_step/sec: 345.229\n",
      "INFO:tensorflow:loss = 0.035298146, step = 49201 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 326.893\n",
      "INFO:tensorflow:loss = 0.014849582, step = 49301 (0.301 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.384\n",
      "INFO:tensorflow:loss = 0.010816911, step = 49401 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 340.324\n",
      "INFO:tensorflow:loss = 0.043109488, step = 49501 (0.294 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.095\n",
      "INFO:tensorflow:loss = 0.027481366, step = 49601 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.062\n",
      "INFO:tensorflow:loss = 0.011738932, step = 49701 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 330.259\n",
      "INFO:tensorflow:loss = 0.020789161, step = 49801 (0.303 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.318\n",
      "INFO:tensorflow:loss = 0.01818737, step = 49901 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 350.426\n",
      "INFO:tensorflow:loss = 0.021740664, step = 50001 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 324.096\n",
      "INFO:tensorflow:loss = 0.017986977, step = 50101 (0.309 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.453\n",
      "INFO:tensorflow:loss = 0.033880364, step = 50201 (0.311 sec)\n",
      "INFO:tensorflow:global_step/sec: 318.651\n",
      "INFO:tensorflow:loss = 0.031064691, step = 50301 (0.313 sec)\n",
      "INFO:tensorflow:global_step/sec: 255.14\n",
      "INFO:tensorflow:loss = 0.015017165, step = 50401 (0.393 sec)\n",
      "INFO:tensorflow:global_step/sec: 245.635\n",
      "INFO:tensorflow:loss = 0.027789034, step = 50501 (0.406 sec)\n",
      "INFO:tensorflow:global_step/sec: 328.65\n",
      "INFO:tensorflow:loss = 0.01457682, step = 50601 (0.306 sec)\n",
      "INFO:tensorflow:global_step/sec: 316.562\n",
      "INFO:tensorflow:loss = 0.014332016, step = 50701 (0.314 sec)\n",
      "INFO:tensorflow:global_step/sec: 303.433\n",
      "INFO:tensorflow:loss = 0.020913832, step = 50801 (0.330 sec)\n",
      "INFO:tensorflow:global_step/sec: 300.178\n",
      "INFO:tensorflow:loss = 0.022722289, step = 50901 (0.334 sec)\n",
      "INFO:tensorflow:global_step/sec: 310.136\n",
      "INFO:tensorflow:loss = 0.011204461, step = 51001 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 316.087\n",
      "INFO:tensorflow:loss = 0.020717554, step = 51101 (0.319 sec)\n",
      "INFO:tensorflow:global_step/sec: 298.198\n",
      "INFO:tensorflow:loss = 0.017500862, step = 51201 (0.331 sec)\n",
      "INFO:tensorflow:global_step/sec: 306.232\n",
      "INFO:tensorflow:loss = 0.031719446, step = 51301 (0.327 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.685\n",
      "INFO:tensorflow:loss = 0.015281688, step = 51401 (0.320 sec)\n",
      "INFO:tensorflow:global_step/sec: 303.9\n",
      "INFO:tensorflow:loss = 0.030304281, step = 51501 (0.332 sec)\n",
      "INFO:tensorflow:global_step/sec: 116.773\n",
      "INFO:tensorflow:loss = 0.016022308, step = 51601 (0.856 sec)\n",
      "INFO:tensorflow:global_step/sec: 248.698\n",
      "INFO:tensorflow:loss = 0.017542932, step = 51701 (0.405 sec)\n",
      "INFO:tensorflow:global_step/sec: 219.439\n",
      "INFO:tensorflow:loss = 0.012441665, step = 51801 (0.449 sec)\n",
      "INFO:tensorflow:global_step/sec: 311.533\n",
      "INFO:tensorflow:loss = 0.030563697, step = 51901 (0.320 sec)\n",
      "INFO:tensorflow:global_step/sec: 285.45\n",
      "INFO:tensorflow:loss = 0.023170881, step = 52001 (0.350 sec)\n",
      "INFO:tensorflow:global_step/sec: 296.557\n",
      "INFO:tensorflow:loss = 0.008133147, step = 52101 (0.340 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.696\n",
      "INFO:tensorflow:loss = 0.009614032, step = 52201 (0.316 sec)\n",
      "INFO:tensorflow:global_step/sec: 307.655\n",
      "INFO:tensorflow:loss = 0.015977228, step = 52301 (0.324 sec)\n",
      "INFO:tensorflow:global_step/sec: 258.921\n",
      "INFO:tensorflow:loss = 0.018748002, step = 52401 (0.383 sec)\n",
      "INFO:tensorflow:global_step/sec: 208.881\n",
      "INFO:tensorflow:loss = 0.018565683, step = 52501 (0.485 sec)\n",
      "INFO:tensorflow:global_step/sec: 219.772\n",
      "INFO:tensorflow:loss = 0.021935832, step = 52601 (0.452 sec)\n",
      "INFO:tensorflow:global_step/sec: 249.578\n",
      "INFO:tensorflow:loss = 0.018435717, step = 52701 (0.397 sec)\n",
      "INFO:tensorflow:global_step/sec: 269.66\n",
      "INFO:tensorflow:loss = 0.022508781, step = 52801 (0.372 sec)\n",
      "INFO:tensorflow:global_step/sec: 279.099\n",
      "INFO:tensorflow:loss = 0.017189119, step = 52901 (0.364 sec)\n",
      "INFO:tensorflow:global_step/sec: 305.528\n",
      "INFO:tensorflow:loss = 0.026252072, step = 53001 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 319.512\n",
      "INFO:tensorflow:loss = 0.019363407, step = 53101 (0.313 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 350.063\n",
      "INFO:tensorflow:loss = 0.011961583, step = 53201 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 318.362\n",
      "INFO:tensorflow:loss = 0.014506684, step = 53301 (0.319 sec)\n",
      "INFO:tensorflow:global_step/sec: 276.802\n",
      "INFO:tensorflow:loss = 0.02157983, step = 53401 (0.358 sec)\n",
      "INFO:tensorflow:global_step/sec: 268.414\n",
      "INFO:tensorflow:loss = 0.025674623, step = 53501 (0.377 sec)\n",
      "INFO:tensorflow:global_step/sec: 210.473\n",
      "INFO:tensorflow:loss = 0.011091793, step = 53601 (0.475 sec)\n",
      "INFO:tensorflow:global_step/sec: 217.217\n",
      "INFO:tensorflow:loss = 0.010901217, step = 53701 (0.460 sec)\n",
      "INFO:tensorflow:global_step/sec: 289.767\n",
      "INFO:tensorflow:loss = 0.034340452, step = 53801 (0.342 sec)\n",
      "INFO:tensorflow:global_step/sec: 294.66\n",
      "INFO:tensorflow:loss = 0.011284155, step = 53901 (0.340 sec)\n",
      "INFO:tensorflow:global_step/sec: 258.428\n",
      "INFO:tensorflow:loss = 0.0117264055, step = 54001 (0.386 sec)\n",
      "INFO:tensorflow:global_step/sec: 264.069\n",
      "INFO:tensorflow:loss = 0.01872945, step = 54101 (0.381 sec)\n",
      "INFO:tensorflow:global_step/sec: 237.266\n",
      "INFO:tensorflow:loss = 0.018314894, step = 54201 (0.420 sec)\n",
      "INFO:tensorflow:global_step/sec: 261.213\n",
      "INFO:tensorflow:loss = 0.020018114, step = 54301 (0.385 sec)\n",
      "INFO:tensorflow:global_step/sec: 263.863\n",
      "INFO:tensorflow:loss = 0.014777226, step = 54401 (0.377 sec)\n",
      "INFO:tensorflow:global_step/sec: 204.233\n",
      "INFO:tensorflow:loss = 0.017486596, step = 54501 (0.490 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.627\n",
      "INFO:tensorflow:loss = 0.022855679, step = 54601 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.196\n",
      "INFO:tensorflow:loss = 0.021124478, step = 54701 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.875\n",
      "INFO:tensorflow:loss = 0.028364774, step = 54801 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.004\n",
      "INFO:tensorflow:loss = 0.034692507, step = 54901 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 403.482\n",
      "INFO:tensorflow:loss = 0.026776077, step = 55001 (0.245 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.096\n",
      "INFO:tensorflow:loss = 0.010409971, step = 55101 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.076\n",
      "INFO:tensorflow:loss = 0.019007716, step = 55201 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 420.544\n",
      "INFO:tensorflow:loss = 0.01934947, step = 55301 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.494\n",
      "INFO:tensorflow:loss = 0.009881462, step = 55401 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.986\n",
      "INFO:tensorflow:loss = 0.023308175, step = 55501 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.289\n",
      "INFO:tensorflow:loss = 0.024727277, step = 55601 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.719\n",
      "INFO:tensorflow:loss = 0.029526902, step = 55701 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 410.715\n",
      "INFO:tensorflow:loss = 0.022972886, step = 55801 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 406.295\n",
      "INFO:tensorflow:loss = 0.020538874, step = 55901 (0.246 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.309\n",
      "INFO:tensorflow:loss = 0.022705166, step = 56001 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 328.674\n",
      "INFO:tensorflow:loss = 0.018179402, step = 56101 (0.305 sec)\n",
      "INFO:tensorflow:global_step/sec: 321.912\n",
      "INFO:tensorflow:loss = 0.0098787295, step = 56201 (0.310 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.959\n",
      "INFO:tensorflow:loss = 0.016108848, step = 56301 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.9\n",
      "INFO:tensorflow:loss = 0.025096498, step = 56401 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.304\n",
      "INFO:tensorflow:loss = 0.020246927, step = 56501 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 288.619\n",
      "INFO:tensorflow:loss = 0.014660941, step = 56601 (0.349 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.09\n",
      "INFO:tensorflow:loss = 0.021249173, step = 56701 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.005\n",
      "INFO:tensorflow:loss = 0.015496539, step = 56801 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.713\n",
      "INFO:tensorflow:loss = 0.01276795, step = 56901 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 398.67\n",
      "INFO:tensorflow:loss = 0.020246655, step = 57001 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.674\n",
      "INFO:tensorflow:loss = 0.020368543, step = 57101 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.419\n",
      "INFO:tensorflow:loss = 0.0116538, step = 57201 (0.254 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.657\n",
      "INFO:tensorflow:loss = 0.01750212, step = 57301 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.402\n",
      "INFO:tensorflow:loss = 0.013349187, step = 57401 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 285.921\n",
      "INFO:tensorflow:loss = 0.01801958, step = 57501 (0.347 sec)\n",
      "INFO:tensorflow:global_step/sec: 351.889\n",
      "INFO:tensorflow:loss = 0.022330072, step = 57601 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.331\n",
      "INFO:tensorflow:loss = 0.016723994, step = 57701 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.203\n",
      "INFO:tensorflow:loss = 0.017940443, step = 57801 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.79\n",
      "INFO:tensorflow:loss = 0.01199382, step = 57901 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.318\n",
      "INFO:tensorflow:loss = 0.024539854, step = 58001 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.509\n",
      "INFO:tensorflow:loss = 0.018657587, step = 58101 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.839\n",
      "INFO:tensorflow:loss = 0.016209621, step = 58201 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.948\n",
      "INFO:tensorflow:loss = 0.018188512, step = 58301 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.324\n",
      "INFO:tensorflow:loss = 0.028275164, step = 58401 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.229\n",
      "INFO:tensorflow:loss = 0.015987467, step = 58501 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.704\n",
      "INFO:tensorflow:loss = 0.01654964, step = 58601 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.852\n",
      "INFO:tensorflow:loss = 0.016889505, step = 58701 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 237.806\n",
      "INFO:tensorflow:loss = 0.014212422, step = 58801 (0.419 sec)\n",
      "INFO:tensorflow:global_step/sec: 305.875\n",
      "INFO:tensorflow:loss = 0.02281481, step = 58901 (0.326 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.234\n",
      "INFO:tensorflow:loss = 0.011542736, step = 59001 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.249\n",
      "INFO:tensorflow:loss = 0.024897916, step = 59101 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 345.616\n",
      "INFO:tensorflow:loss = 0.019490276, step = 59201 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.851\n",
      "INFO:tensorflow:loss = 0.01131694, step = 59301 (0.248 sec)\n",
      "INFO:tensorflow:global_step/sec: 400.787\n",
      "INFO:tensorflow:loss = 0.013082898, step = 59401 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.275\n",
      "INFO:tensorflow:loss = 0.012769192, step = 59501 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.035\n",
      "INFO:tensorflow:loss = 0.010974027, step = 59601 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 305.667\n",
      "INFO:tensorflow:loss = 0.012292547, step = 59701 (0.328 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.754\n",
      "INFO:tensorflow:loss = 0.015486717, step = 59801 (0.309 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.815\n",
      "INFO:tensorflow:loss = 0.02143418, step = 59901 (0.260 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 60000 into /tmp/tmpyxwongpm/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-08:59:45\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-60000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'previous_ensemble' dict for global step 60000: architecture/adanet/ensembles = b\"\\n^\\n,adanet/previous_ensemble/architecture/adanetB$\\x08\\x07\\x12\\x00B\\x1e| b'linear' | b'1_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.04485139, average_loss/adanet/subnetwork = 0.04407932, average_loss/adanet/uniform_average_ensemble = 0.043675844, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.059158117, loss/adanet/subnetwork = 0.061847888, loss/adanet/uniform_average_ensemble = 0.0585649, prediction/mean/adanet/adanet_weighted_ensemble = 3.1251307, prediction/mean/adanet/subnetwork = 3.1570365, prediction/mean/adanet/uniform_average_ensemble = 3.131466\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving candidate '1_layer_dnn' dict for global step 60000: architecture/adanet/ensembles = b\"\\n~\\n;adanet/iteration_2/ensemble_1_layer_dnn/architecture/adanetB5\\x08\\x07\\x12\\x00B/| b'linear' | b'1_layer_dnn' | b'1_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.042682763, average_loss/adanet/subnetwork = 0.038611967, average_loss/adanet/uniform_average_ensemble = 0.040760666, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.055231333, loss/adanet/subnetwork = 0.053511024, loss/adanet/uniform_average_ensemble = 0.05558739, prediction/mean/adanet/adanet_weighted_ensemble = 3.098613, prediction/mean/adanet/subnetwork = 3.1419017, prediction/mean/adanet/uniform_average_ensemble = 3.1349444\n",
      "INFO:tensorflow:Saving candidate '2_layer_dnn' dict for global step 60000: architecture/adanet/ensembles = b\"\\n~\\n;adanet/iteration_2/ensemble_2_layer_dnn/architecture/adanetB5\\x08\\x07\\x12\\x00B/| b'linear' | b'1_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.037821766, average_loss/adanet/subnetwork = 0.035940513, average_loss/adanet/uniform_average_ensemble = 0.0381884, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.046586804, loss/adanet/subnetwork = 0.038228974, loss/adanet/uniform_average_ensemble = 0.048374467, prediction/mean/adanet/adanet_weighted_ensemble = 3.1108563, prediction/mean/adanet/subnetwork = 3.1267877, prediction/mean/adanet/uniform_average_ensemble = 3.1299064\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-19-08:59:47\n",
      "INFO:tensorflow:Saving dict for global step 60000: average_loss = 0.037821766, average_loss/adanet/adanet_weighted_ensemble = 0.037821766, average_loss/adanet/subnetwork = 0.035940513, average_loss/adanet/uniform_average_ensemble = 0.0381884, global_step = 60000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.046586804, loss/adanet/adanet_weighted_ensemble = 0.046586804, loss/adanet/subnetwork = 0.038228974, loss/adanet/uniform_average_ensemble = 0.048374467, prediction/mean = 3.1108563, prediction/mean/adanet/adanet_weighted_ensemble = 3.1108563, prediction/mean/adanet/subnetwork = 3.1267877, prediction/mean/adanet/uniform_average_ensemble = 3.1299064\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 60000: /tmp/tmpyxwongpm/model.ckpt-60000\n",
      "INFO:tensorflow:Loss for final step: 0.021004943.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Starting ensemble evaluation for iteration 2\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-60000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Encountered end of input after 14 evaluations\n",
      "INFO:tensorflow:Computed ensemble metrics: adanet_loss/previous_ensemble = 0.032305, adanet_loss/1_layer_dnn = 0.031279, adanet_loss/2_layer_dnn = 0.026585\n",
      "INFO:tensorflow:Finished ensemble evaluation for iteration 2\n",
      "INFO:tensorflow:'2_layer_dnn' at index 2 is moving onto the next iteration\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-1.meta with features: ['x'].\n",
      "INFO:tensorflow:Freezing best ensemble to /tmp/tmpyxwongpm/frozen/ensemble-2.meta\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-60000\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Overwriting checkpoint with new graph for iteration 3 to /tmp/tmpyxwongpm/model.ckpt-60000\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-60000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Finished training Adanet iteration 2\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 3\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/increment.ckpt-3\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 60000 into /tmp/tmpyxwongpm/model.ckpt.\n",
      "INFO:tensorflow:loss = 0.016193133, step = 60001\n",
      "INFO:tensorflow:global_step/sec: 96.0512\n",
      "INFO:tensorflow:loss = 0.016663171, step = 60101 (1.043 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.021\n",
      "INFO:tensorflow:loss = 0.017350707, step = 60201 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.065\n",
      "INFO:tensorflow:loss = 0.014216094, step = 60301 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.5\n",
      "INFO:tensorflow:loss = 0.02011441, step = 60401 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.066\n",
      "INFO:tensorflow:loss = 0.013009091, step = 60501 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.788\n",
      "INFO:tensorflow:loss = 0.015400722, step = 60601 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.341\n",
      "INFO:tensorflow:loss = 0.009882641, step = 60701 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.958\n",
      "INFO:tensorflow:loss = 0.009190591, step = 60801 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.1\n",
      "INFO:tensorflow:loss = 0.01581464, step = 60901 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.247\n",
      "INFO:tensorflow:loss = 0.016352057, step = 61001 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.671\n",
      "INFO:tensorflow:loss = 0.015545483, step = 61101 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.325\n",
      "INFO:tensorflow:loss = 0.014443488, step = 61201 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.659\n",
      "INFO:tensorflow:loss = 0.026640493, step = 61301 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.103\n",
      "INFO:tensorflow:loss = 0.006407236, step = 61401 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.104\n",
      "INFO:tensorflow:loss = 0.008267016, step = 61501 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.681\n",
      "INFO:tensorflow:loss = 0.011237941, step = 61601 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.679\n",
      "INFO:tensorflow:loss = 0.017959919, step = 61701 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.906\n",
      "INFO:tensorflow:loss = 0.016271543, step = 61801 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.831\n",
      "INFO:tensorflow:loss = 0.021004671, step = 61901 (0.272 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.397\n",
      "INFO:tensorflow:loss = 0.012583302, step = 62001 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.402\n",
      "INFO:tensorflow:loss = 0.022682708, step = 62101 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 428.118\n",
      "INFO:tensorflow:loss = 0.010721676, step = 62201 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.149\n",
      "INFO:tensorflow:loss = 0.01077175, step = 62301 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 369.251\n",
      "INFO:tensorflow:loss = 0.011335289, step = 62401 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.209\n",
      "INFO:tensorflow:loss = 0.01684907, step = 62501 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.622\n",
      "INFO:tensorflow:loss = 0.011086859, step = 62601 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.023\n",
      "INFO:tensorflow:loss = 0.012787401, step = 62701 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.842\n",
      "INFO:tensorflow:loss = 0.007351443, step = 62801 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.266\n",
      "INFO:tensorflow:loss = 0.012255042, step = 62901 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.016\n",
      "INFO:tensorflow:loss = 0.010560467, step = 63001 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 363.189\n",
      "INFO:tensorflow:loss = 0.009190907, step = 63101 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.907\n",
      "INFO:tensorflow:loss = 0.013143709, step = 63201 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.293\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.018950937, step = 63301 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.861\n",
      "INFO:tensorflow:loss = 0.0125065185, step = 63401 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.235\n",
      "INFO:tensorflow:loss = 0.025653698, step = 63501 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.667\n",
      "INFO:tensorflow:loss = 0.009902426, step = 63601 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.128\n",
      "INFO:tensorflow:loss = 0.010109541, step = 63701 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.901\n",
      "INFO:tensorflow:loss = 0.007783378, step = 63801 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.62\n",
      "INFO:tensorflow:loss = 0.008886563, step = 63901 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.313\n",
      "INFO:tensorflow:loss = 0.01669297, step = 64001 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.505\n",
      "INFO:tensorflow:loss = 0.012598669, step = 64101 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.781\n",
      "INFO:tensorflow:loss = 0.0065435776, step = 64201 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 362.54\n",
      "INFO:tensorflow:loss = 0.0056856177, step = 64301 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.426\n",
      "INFO:tensorflow:loss = 0.00856412, step = 64401 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.414\n",
      "INFO:tensorflow:loss = 0.013367176, step = 64501 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.247\n",
      "INFO:tensorflow:loss = 0.011314306, step = 64601 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.102\n",
      "INFO:tensorflow:loss = 0.01617311, step = 64701 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.392\n",
      "INFO:tensorflow:loss = 0.010521907, step = 64801 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.867\n",
      "INFO:tensorflow:loss = 0.011767966, step = 64901 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.717\n",
      "INFO:tensorflow:loss = 0.0071188034, step = 65001 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.446\n",
      "INFO:tensorflow:loss = 0.00923439, step = 65101 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.997\n",
      "INFO:tensorflow:loss = 0.009812816, step = 65201 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.135\n",
      "INFO:tensorflow:loss = 0.019939449, step = 65301 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.575\n",
      "INFO:tensorflow:loss = 0.009342239, step = 65401 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.861\n",
      "INFO:tensorflow:loss = 0.009747079, step = 65501 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.13\n",
      "INFO:tensorflow:loss = 0.011791237, step = 65601 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.007\n",
      "INFO:tensorflow:loss = 0.012506308, step = 65701 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.282\n",
      "INFO:tensorflow:loss = 0.0121092815, step = 65801 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.265\n",
      "INFO:tensorflow:loss = 0.0067040995, step = 65901 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.441\n",
      "INFO:tensorflow:loss = 0.011969661, step = 66001 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.308\n",
      "INFO:tensorflow:loss = 0.012336211, step = 66101 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.454\n",
      "INFO:tensorflow:loss = 0.016482107, step = 66201 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.271\n",
      "INFO:tensorflow:loss = 0.011585791, step = 66301 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.045\n",
      "INFO:tensorflow:loss = 0.016038775, step = 66401 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.442\n",
      "INFO:tensorflow:loss = 0.018227994, step = 66501 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.902\n",
      "INFO:tensorflow:loss = 0.00875232, step = 66601 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.489\n",
      "INFO:tensorflow:loss = 0.011877631, step = 66701 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.017\n",
      "INFO:tensorflow:loss = 0.008582944, step = 66801 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.917\n",
      "INFO:tensorflow:loss = 0.006836283, step = 66901 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.734\n",
      "INFO:tensorflow:loss = 0.011311612, step = 67001 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.456\n",
      "INFO:tensorflow:loss = 0.00904309, step = 67101 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.926\n",
      "INFO:tensorflow:loss = 0.009702247, step = 67201 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.948\n",
      "INFO:tensorflow:loss = 0.019215994, step = 67301 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.015\n",
      "INFO:tensorflow:loss = 0.013199607, step = 67401 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.355\n",
      "INFO:tensorflow:loss = 0.009328953, step = 67501 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.411\n",
      "INFO:tensorflow:loss = 0.007837815, step = 67601 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.504\n",
      "INFO:tensorflow:loss = 0.012944186, step = 67701 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.413\n",
      "INFO:tensorflow:loss = 0.014152002, step = 67801 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.026\n",
      "INFO:tensorflow:loss = 0.010471663, step = 67901 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.356\n",
      "INFO:tensorflow:loss = 0.01439284, step = 68001 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 363.481\n",
      "INFO:tensorflow:loss = 0.011319883, step = 68101 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.199\n",
      "INFO:tensorflow:loss = 0.0148774795, step = 68201 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.881\n",
      "INFO:tensorflow:loss = 0.014782866, step = 68301 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.986\n",
      "INFO:tensorflow:loss = 0.012911025, step = 68401 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.344\n",
      "INFO:tensorflow:loss = 0.01032743, step = 68501 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.972\n",
      "INFO:tensorflow:loss = 0.014659831, step = 68601 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.68\n",
      "INFO:tensorflow:loss = 0.01193401, step = 68701 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.775\n",
      "INFO:tensorflow:loss = 0.015532577, step = 68801 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.064\n",
      "INFO:tensorflow:loss = 0.021762801, step = 68901 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.985\n",
      "INFO:tensorflow:loss = 0.021618707, step = 69001 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 393.788\n",
      "INFO:tensorflow:loss = 0.011765096, step = 69101 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.785\n",
      "INFO:tensorflow:loss = 0.01668916, step = 69201 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.339\n",
      "INFO:tensorflow:loss = 0.0071507227, step = 69301 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.833\n",
      "INFO:tensorflow:loss = 0.0067, step = 69401 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.806\n",
      "INFO:tensorflow:loss = 0.019270815, step = 69501 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.957\n",
      "INFO:tensorflow:loss = 0.011013599, step = 69601 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.506\n",
      "INFO:tensorflow:loss = 0.0059794104, step = 69701 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.444\n",
      "INFO:tensorflow:loss = 0.010903874, step = 69801 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.696\n",
      "INFO:tensorflow:loss = 0.012362252, step = 69901 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.826\n",
      "INFO:tensorflow:loss = 0.012381483, step = 70001 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.379\n",
      "INFO:tensorflow:loss = 0.007959043, step = 70101 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 323.064\n",
      "INFO:tensorflow:loss = 0.013683063, step = 70201 (0.309 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.58\n",
      "INFO:tensorflow:loss = 0.016028106, step = 70301 (0.311 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.41\n",
      "INFO:tensorflow:loss = 0.007750828, step = 70401 (0.310 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.45\n",
      "INFO:tensorflow:loss = 0.012662745, step = 70501 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.553\n",
      "INFO:tensorflow:loss = 0.009560704, step = 70601 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.249\n",
      "INFO:tensorflow:loss = 0.008536605, step = 70701 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.36\n",
      "INFO:tensorflow:loss = 0.012457779, step = 70801 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.125\n",
      "INFO:tensorflow:loss = 0.013018131, step = 70901 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.285\n",
      "INFO:tensorflow:loss = 0.0074925884, step = 71001 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 316.358\n",
      "INFO:tensorflow:loss = 0.0127674695, step = 71101 (0.317 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.89\n",
      "INFO:tensorflow:loss = 0.008397549, step = 71201 (0.314 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.179\n",
      "INFO:tensorflow:loss = 0.013499344, step = 71301 (0.302 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 319.912\n",
      "INFO:tensorflow:loss = 0.007563833, step = 71401 (0.313 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.905\n",
      "INFO:tensorflow:loss = 0.018885966, step = 71501 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 340.835\n",
      "INFO:tensorflow:loss = 0.0063192933, step = 71601 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.149\n",
      "INFO:tensorflow:loss = 0.012419142, step = 71701 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 337.271\n",
      "INFO:tensorflow:loss = 0.009384869, step = 71801 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.592\n",
      "INFO:tensorflow:loss = 0.00930639, step = 71901 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.326\n",
      "INFO:tensorflow:loss = 0.011872329, step = 72001 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.484\n",
      "INFO:tensorflow:loss = 0.005632778, step = 72101 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.649\n",
      "INFO:tensorflow:loss = 0.00506723, step = 72201 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 387.76\n",
      "INFO:tensorflow:loss = 0.008731108, step = 72301 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.583\n",
      "INFO:tensorflow:loss = 0.011890879, step = 72401 (0.290 sec)\n",
      "INFO:tensorflow:global_step/sec: 388.052\n",
      "INFO:tensorflow:loss = 0.010278417, step = 72501 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.507\n",
      "INFO:tensorflow:loss = 0.009217897, step = 72601 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.413\n",
      "INFO:tensorflow:loss = 0.009985749, step = 72701 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.27\n",
      "INFO:tensorflow:loss = 0.016222142, step = 72801 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.013\n",
      "INFO:tensorflow:loss = 0.011246605, step = 72901 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.327\n",
      "INFO:tensorflow:loss = 0.015886359, step = 73001 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.677\n",
      "INFO:tensorflow:loss = 0.01137105, step = 73101 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.038\n",
      "INFO:tensorflow:loss = 0.0084267855, step = 73201 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.338\n",
      "INFO:tensorflow:loss = 0.008732837, step = 73301 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.143\n",
      "INFO:tensorflow:loss = 0.012200361, step = 73401 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.164\n",
      "INFO:tensorflow:loss = 0.013456284, step = 73501 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.082\n",
      "INFO:tensorflow:loss = 0.008109665, step = 73601 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 321.361\n",
      "INFO:tensorflow:loss = 0.007848716, step = 73701 (0.313 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.441\n",
      "INFO:tensorflow:loss = 0.01653469, step = 73801 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.578\n",
      "INFO:tensorflow:loss = 0.005903042, step = 73901 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 326.337\n",
      "INFO:tensorflow:loss = 0.007809826, step = 74001 (0.306 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.339\n",
      "INFO:tensorflow:loss = 0.0118000135, step = 74101 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.818\n",
      "INFO:tensorflow:loss = 0.010862283, step = 74201 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.979\n",
      "INFO:tensorflow:loss = 0.01307364, step = 74301 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.917\n",
      "INFO:tensorflow:loss = 0.00927268, step = 74401 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 350.317\n",
      "INFO:tensorflow:loss = 0.0091843195, step = 74501 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 337.214\n",
      "INFO:tensorflow:loss = 0.011435147, step = 74601 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 334.663\n",
      "INFO:tensorflow:loss = 0.012518871, step = 74701 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.641\n",
      "INFO:tensorflow:loss = 0.019361224, step = 74801 (0.258 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.037\n",
      "INFO:tensorflow:loss = 0.016393568, step = 74901 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 389.944\n",
      "INFO:tensorflow:loss = 0.019858979, step = 75001 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 329.065\n",
      "INFO:tensorflow:loss = 0.006206369, step = 75101 (0.301 sec)\n",
      "INFO:tensorflow:global_step/sec: 332.191\n",
      "INFO:tensorflow:loss = 0.009555208, step = 75201 (0.301 sec)\n",
      "INFO:tensorflow:global_step/sec: 337.199\n",
      "INFO:tensorflow:loss = 0.014107707, step = 75301 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.034\n",
      "INFO:tensorflow:loss = 0.0075152926, step = 75401 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 344.3\n",
      "INFO:tensorflow:loss = 0.016216122, step = 75501 (0.290 sec)\n",
      "INFO:tensorflow:global_step/sec: 334.16\n",
      "INFO:tensorflow:loss = 0.012158712, step = 75601 (0.299 sec)\n",
      "INFO:tensorflow:global_step/sec: 386.753\n",
      "INFO:tensorflow:loss = 0.012325074, step = 75701 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.055\n",
      "INFO:tensorflow:loss = 0.017332906, step = 75801 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.688\n",
      "INFO:tensorflow:loss = 0.009472498, step = 75901 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 382.293\n",
      "INFO:tensorflow:loss = 0.010689353, step = 76001 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 321.23\n",
      "INFO:tensorflow:loss = 0.010871442, step = 76101 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.639\n",
      "INFO:tensorflow:loss = 0.005839846, step = 76201 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 328.738\n",
      "INFO:tensorflow:loss = 0.0077887448, step = 76301 (0.303 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.974\n",
      "INFO:tensorflow:loss = 0.012547378, step = 76401 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.67\n",
      "INFO:tensorflow:loss = 0.010709663, step = 76501 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.201\n",
      "INFO:tensorflow:loss = 0.0075437217, step = 76601 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.489\n",
      "INFO:tensorflow:loss = 0.013839469, step = 76701 (0.299 sec)\n",
      "INFO:tensorflow:global_step/sec: 329.314\n",
      "INFO:tensorflow:loss = 0.012491439, step = 76801 (0.303 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.491\n",
      "INFO:tensorflow:loss = 0.0058735143, step = 76901 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 385.651\n",
      "INFO:tensorflow:loss = 0.012763498, step = 77001 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.129\n",
      "INFO:tensorflow:loss = 0.016112193, step = 77101 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.129\n",
      "INFO:tensorflow:loss = 0.007122926, step = 77201 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.049\n",
      "INFO:tensorflow:loss = 0.013300964, step = 77301 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.235\n",
      "INFO:tensorflow:loss = 0.007759212, step = 77401 (0.314 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.387\n",
      "INFO:tensorflow:loss = 0.01582412, step = 77501 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.998\n",
      "INFO:tensorflow:loss = 0.012858916, step = 77601 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 347.841\n",
      "INFO:tensorflow:loss = 0.007881397, step = 77701 (0.287 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.26\n",
      "INFO:tensorflow:loss = 0.013149595, step = 77801 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.01\n",
      "INFO:tensorflow:loss = 0.0091339685, step = 77901 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 314.838\n",
      "INFO:tensorflow:loss = 0.016202006, step = 78001 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 311.88\n",
      "INFO:tensorflow:loss = 0.0106188655, step = 78101 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 318.503\n",
      "INFO:tensorflow:loss = 0.011859131, step = 78201 (0.314 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.542\n",
      "INFO:tensorflow:loss = 0.014015723, step = 78301 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.437\n",
      "INFO:tensorflow:loss = 0.01489076, step = 78401 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.486\n",
      "INFO:tensorflow:loss = 0.009929089, step = 78501 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 381.138\n",
      "INFO:tensorflow:loss = 0.009556154, step = 78601 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.129\n",
      "INFO:tensorflow:loss = 0.011537885, step = 78701 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.189\n",
      "INFO:tensorflow:loss = 0.010016901, step = 78801 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.225\n",
      "INFO:tensorflow:loss = 0.017832043, step = 78901 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.909\n",
      "INFO:tensorflow:loss = 0.009529863, step = 79001 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 355.323\n",
      "INFO:tensorflow:loss = 0.015711283, step = 79101 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.163\n",
      "INFO:tensorflow:loss = 0.013484484, step = 79201 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.169\n",
      "INFO:tensorflow:loss = 0.009018617, step = 79301 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 314.085\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.009111306, step = 79401 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 327.153\n",
      "INFO:tensorflow:loss = 0.008054039, step = 79501 (0.309 sec)\n",
      "INFO:tensorflow:global_step/sec: 339.574\n",
      "INFO:tensorflow:loss = 0.006679391, step = 79601 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 350.033\n",
      "INFO:tensorflow:loss = 0.008028477, step = 79701 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.011\n",
      "INFO:tensorflow:loss = 0.008896752, step = 79801 (0.311 sec)\n",
      "INFO:tensorflow:global_step/sec: 320.338\n",
      "INFO:tensorflow:loss = 0.012979875, step = 79901 (0.312 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 80000 into /tmp/tmpyxwongpm/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-09:01:01\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-80000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'previous_ensemble' dict for global step 80000: architecture/adanet/ensembles = b\"\\no\\n,adanet/previous_ensemble/architecture/adanetB5\\x08\\x07\\x12\\x00B/| b'linear' | b'1_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.037821766, average_loss/adanet/subnetwork = 0.035940513, average_loss/adanet/uniform_average_ensemble = 0.038188405, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.046586804, loss/adanet/subnetwork = 0.038228974, loss/adanet/uniform_average_ensemble = 0.04837447, prediction/mean/adanet/adanet_weighted_ensemble = 3.1108563, prediction/mean/adanet/subnetwork = 3.1267877, prediction/mean/adanet/uniform_average_ensemble = 3.1299064\n",
      "INFO:tensorflow:Saving candidate '2_layer_dnn' dict for global step 80000: architecture/adanet/ensembles = b\"\\n\\x8f\\x01\\n;adanet/iteration_3/ensemble_2_layer_dnn/architecture/adanetBF\\x08\\x07\\x12\\x00B@| b'linear' | b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.036644638, average_loss/adanet/subnetwork = 0.029604359, average_loss/adanet/uniform_average_ensemble = 0.034464236, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.040175013, loss/adanet/subnetwork = 0.03914282, loss/adanet/uniform_average_ensemble = 0.044524893, prediction/mean/adanet/adanet_weighted_ensemble = 3.080653, prediction/mean/adanet/subnetwork = 3.1459546, prediction/mean/adanet/uniform_average_ensemble = 3.1339185\n",
      "INFO:tensorflow:Saving candidate '3_layer_dnn' dict for global step 80000: architecture/adanet/ensembles = b\"\\n\\x8f\\x01\\n;adanet/iteration_3/ensemble_3_layer_dnn/architecture/adanetBF\\x08\\x07\\x12\\x00B@| b'linear' | b'1_layer_dnn' | b'2_layer_dnn' | b'3_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.036669817, average_loss/adanet/subnetwork = 0.028511057, average_loss/adanet/uniform_average_ensemble = 0.033795744, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.04017978, loss/adanet/subnetwork = 0.030393401, loss/adanet/uniform_average_ensemble = 0.041697547, prediction/mean/adanet/adanet_weighted_ensemble = 3.0804253, prediction/mean/adanet/subnetwork = 3.1202893, prediction/mean/adanet/uniform_average_ensemble = 3.1275022\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-19-09:01:03\n",
      "INFO:tensorflow:Saving dict for global step 80000: average_loss = 0.036644638, average_loss/adanet/adanet_weighted_ensemble = 0.036644638, average_loss/adanet/subnetwork = 0.029604359, average_loss/adanet/uniform_average_ensemble = 0.034464236, global_step = 80000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.040175013, loss/adanet/adanet_weighted_ensemble = 0.040175013, loss/adanet/subnetwork = 0.03914282, loss/adanet/uniform_average_ensemble = 0.044524893, prediction/mean = 3.080653, prediction/mean/adanet/adanet_weighted_ensemble = 3.080653, prediction/mean/adanet/subnetwork = 3.1459546, prediction/mean/adanet/uniform_average_ensemble = 3.1339185\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 80000: /tmp/tmpyxwongpm/model.ckpt-80000\n",
      "INFO:tensorflow:Loss for final step: 0.01150721.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Starting ensemble evaluation for iteration 3\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-80000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Encountered end of input after 14 evaluations\n",
      "INFO:tensorflow:Computed ensemble metrics: adanet_loss/previous_ensemble = 0.026585, adanet_loss/2_layer_dnn = 0.024071, adanet_loss/3_layer_dnn = 0.024077\n",
      "INFO:tensorflow:Finished ensemble evaluation for iteration 3\n",
      "INFO:tensorflow:'2_layer_dnn' at index 1 is moving onto the next iteration\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-2.meta with features: ['x'].\n",
      "INFO:tensorflow:Freezing best ensemble to /tmp/tmpyxwongpm/frozen/ensemble-3.meta\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-80000\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-3.meta with features: ['x'].\n",
      "INFO:tensorflow:Overwriting checkpoint with new graph for iteration 4 to /tmp/tmpyxwongpm/model.ckpt-80000\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-80000\n",
      "WARNING:tensorflow:`tf.train.start_queue_runners()` was called when no queue runners were defined. You can safely remove the call to this deprecated function.\n",
      "INFO:tensorflow:Finished training Adanet iteration 3\n",
      "INFO:tensorflow:Beginning training AdaNet iteration 4\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-3.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/increment.ckpt-4\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 80000 into /tmp/tmpyxwongpm/model.ckpt.\n",
      "INFO:tensorflow:loss = 0.011536109, step = 80001\n",
      "INFO:tensorflow:global_step/sec: 64.6765\n",
      "INFO:tensorflow:loss = 0.011485739, step = 80101 (1.548 sec)\n",
      "INFO:tensorflow:global_step/sec: 317.837\n",
      "INFO:tensorflow:loss = 0.015980862, step = 80201 (0.316 sec)\n",
      "INFO:tensorflow:global_step/sec: 269.064\n",
      "INFO:tensorflow:loss = 0.011237527, step = 80301 (0.372 sec)\n",
      "INFO:tensorflow:global_step/sec: 274.248\n",
      "INFO:tensorflow:loss = 0.012422724, step = 80401 (0.368 sec)\n",
      "INFO:tensorflow:global_step/sec: 214.121\n",
      "INFO:tensorflow:loss = 0.010191858, step = 80501 (0.462 sec)\n",
      "INFO:tensorflow:global_step/sec: 281.401\n",
      "INFO:tensorflow:loss = 0.011031952, step = 80601 (0.356 sec)\n",
      "INFO:tensorflow:global_step/sec: 267.723\n",
      "INFO:tensorflow:loss = 0.008389421, step = 80701 (0.373 sec)\n",
      "INFO:tensorflow:global_step/sec: 306.029\n",
      "INFO:tensorflow:loss = 0.0066847163, step = 80801 (0.328 sec)\n",
      "INFO:tensorflow:global_step/sec: 207.756\n",
      "INFO:tensorflow:loss = 0.013176909, step = 80901 (0.481 sec)\n",
      "INFO:tensorflow:global_step/sec: 310.957\n",
      "INFO:tensorflow:loss = 0.0115530435, step = 81001 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 215.508\n",
      "INFO:tensorflow:loss = 0.011496134, step = 81101 (0.465 sec)\n",
      "INFO:tensorflow:global_step/sec: 156.724\n",
      "INFO:tensorflow:loss = 0.00994376, step = 81201 (0.643 sec)\n",
      "INFO:tensorflow:global_step/sec: 189.462\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.016247045, step = 81301 (0.524 sec)\n",
      "INFO:tensorflow:global_step/sec: 281.409\n",
      "INFO:tensorflow:loss = 0.0046351785, step = 81401 (0.355 sec)\n",
      "INFO:tensorflow:global_step/sec: 184.101\n",
      "INFO:tensorflow:loss = 0.0068879593, step = 81501 (0.540 sec)\n",
      "INFO:tensorflow:global_step/sec: 292.058\n",
      "INFO:tensorflow:loss = 0.0076613384, step = 81601 (0.345 sec)\n",
      "INFO:tensorflow:global_step/sec: 274.88\n",
      "INFO:tensorflow:loss = 0.012942525, step = 81701 (0.361 sec)\n",
      "INFO:tensorflow:global_step/sec: 233.896\n",
      "INFO:tensorflow:loss = 0.012133781, step = 81801 (0.431 sec)\n",
      "INFO:tensorflow:global_step/sec: 198.759\n",
      "INFO:tensorflow:loss = 0.01579272, step = 81901 (0.503 sec)\n",
      "INFO:tensorflow:global_step/sec: 310.894\n",
      "INFO:tensorflow:loss = 0.010960525, step = 82001 (0.323 sec)\n",
      "INFO:tensorflow:global_step/sec: 282.843\n",
      "INFO:tensorflow:loss = 0.019725045, step = 82101 (0.349 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.46\n",
      "INFO:tensorflow:loss = 0.008978224, step = 82201 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 234.091\n",
      "INFO:tensorflow:loss = 0.009846812, step = 82301 (0.430 sec)\n",
      "INFO:tensorflow:global_step/sec: 239.266\n",
      "INFO:tensorflow:loss = 0.008192166, step = 82401 (0.415 sec)\n",
      "INFO:tensorflow:global_step/sec: 255.536\n",
      "INFO:tensorflow:loss = 0.015519582, step = 82501 (0.388 sec)\n",
      "INFO:tensorflow:global_step/sec: 323.003\n",
      "INFO:tensorflow:loss = 0.007954079, step = 82601 (0.309 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.549\n",
      "INFO:tensorflow:loss = 0.01154213, step = 82701 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 244.486\n",
      "INFO:tensorflow:loss = 0.006536168, step = 82801 (0.411 sec)\n",
      "INFO:tensorflow:global_step/sec: 199.061\n",
      "INFO:tensorflow:loss = 0.009054527, step = 82901 (0.502 sec)\n",
      "INFO:tensorflow:global_step/sec: 205.334\n",
      "INFO:tensorflow:loss = 0.009671277, step = 83001 (0.485 sec)\n",
      "INFO:tensorflow:global_step/sec: 266.758\n",
      "INFO:tensorflow:loss = 0.007681434, step = 83101 (0.375 sec)\n",
      "INFO:tensorflow:global_step/sec: 369.347\n",
      "INFO:tensorflow:loss = 0.009225392, step = 83201 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.486\n",
      "INFO:tensorflow:loss = 0.014606323, step = 83301 (0.301 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.133\n",
      "INFO:tensorflow:loss = 0.012142891, step = 83401 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.33\n",
      "INFO:tensorflow:loss = 0.020459354, step = 83501 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.315\n",
      "INFO:tensorflow:loss = 0.0075784763, step = 83601 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 418.419\n",
      "INFO:tensorflow:loss = 0.00932733, step = 83701 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 351.321\n",
      "INFO:tensorflow:loss = 0.0084498245, step = 83801 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 291.223\n",
      "INFO:tensorflow:loss = 0.007813556, step = 83901 (0.345 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.614\n",
      "INFO:tensorflow:loss = 0.013385512, step = 84001 (0.315 sec)\n",
      "INFO:tensorflow:global_step/sec: 204.23\n",
      "INFO:tensorflow:loss = 0.009919434, step = 84101 (0.494 sec)\n",
      "INFO:tensorflow:global_step/sec: 216.28\n",
      "INFO:tensorflow:loss = 0.00536523, step = 84201 (0.458 sec)\n",
      "INFO:tensorflow:global_step/sec: 205.4\n",
      "INFO:tensorflow:loss = 0.0065858914, step = 84301 (0.491 sec)\n",
      "INFO:tensorflow:global_step/sec: 225.772\n",
      "INFO:tensorflow:loss = 0.008099992, step = 84401 (0.449 sec)\n",
      "INFO:tensorflow:global_step/sec: 274.462\n",
      "INFO:tensorflow:loss = 0.011587471, step = 84501 (0.356 sec)\n",
      "INFO:tensorflow:global_step/sec: 259.65\n",
      "INFO:tensorflow:loss = 0.009211194, step = 84601 (0.385 sec)\n",
      "INFO:tensorflow:global_step/sec: 196.454\n",
      "INFO:tensorflow:loss = 0.01313727, step = 84701 (0.509 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.909\n",
      "INFO:tensorflow:loss = 0.010287554, step = 84801 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.434\n",
      "INFO:tensorflow:loss = 0.0095497165, step = 84901 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.687\n",
      "INFO:tensorflow:loss = 0.006867404, step = 85001 (0.264 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.787\n",
      "INFO:tensorflow:loss = 0.008990077, step = 85101 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 296.857\n",
      "INFO:tensorflow:loss = 0.009490061, step = 85201 (0.338 sec)\n",
      "INFO:tensorflow:global_step/sec: 282.664\n",
      "INFO:tensorflow:loss = 0.017177382, step = 85301 (0.353 sec)\n",
      "INFO:tensorflow:global_step/sec: 205.729\n",
      "INFO:tensorflow:loss = 0.008064292, step = 85401 (0.487 sec)\n",
      "INFO:tensorflow:global_step/sec: 369.21\n",
      "INFO:tensorflow:loss = 0.0078684, step = 85501 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 301.746\n",
      "INFO:tensorflow:loss = 0.00965209, step = 85601 (0.332 sec)\n",
      "INFO:tensorflow:global_step/sec: 314.924\n",
      "INFO:tensorflow:loss = 0.010759894, step = 85701 (0.316 sec)\n",
      "INFO:tensorflow:global_step/sec: 329.049\n",
      "INFO:tensorflow:loss = 0.011745572, step = 85801 (0.305 sec)\n",
      "INFO:tensorflow:global_step/sec: 308.558\n",
      "INFO:tensorflow:loss = 0.0070296274, step = 85901 (0.325 sec)\n",
      "INFO:tensorflow:global_step/sec: 302.113\n",
      "INFO:tensorflow:loss = 0.010643888, step = 86001 (0.330 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.117\n",
      "INFO:tensorflow:loss = 0.010507051, step = 86101 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.671\n",
      "INFO:tensorflow:loss = 0.015571639, step = 86201 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 276.939\n",
      "INFO:tensorflow:loss = 0.010290359, step = 86301 (0.360 sec)\n",
      "INFO:tensorflow:global_step/sec: 266.459\n",
      "INFO:tensorflow:loss = 0.013271468, step = 86401 (0.376 sec)\n",
      "INFO:tensorflow:global_step/sec: 306.663\n",
      "INFO:tensorflow:loss = 0.01651977, step = 86501 (0.326 sec)\n",
      "INFO:tensorflow:global_step/sec: 280.141\n",
      "INFO:tensorflow:loss = 0.008029985, step = 86601 (0.356 sec)\n",
      "INFO:tensorflow:global_step/sec: 293.453\n",
      "INFO:tensorflow:loss = 0.010947716, step = 86701 (0.341 sec)\n",
      "INFO:tensorflow:global_step/sec: 301.734\n",
      "INFO:tensorflow:loss = 0.008647747, step = 86801 (0.334 sec)\n",
      "INFO:tensorflow:global_step/sec: 309.825\n",
      "INFO:tensorflow:loss = 0.006303915, step = 86901 (0.320 sec)\n",
      "INFO:tensorflow:global_step/sec: 271.017\n",
      "INFO:tensorflow:loss = 0.009561986, step = 87001 (0.369 sec)\n",
      "INFO:tensorflow:global_step/sec: 303.957\n",
      "INFO:tensorflow:loss = 0.008877378, step = 87101 (0.329 sec)\n",
      "INFO:tensorflow:global_step/sec: 303.425\n",
      "INFO:tensorflow:loss = 0.009176455, step = 87201 (0.329 sec)\n",
      "INFO:tensorflow:global_step/sec: 316.247\n",
      "INFO:tensorflow:loss = 0.016845595, step = 87301 (0.317 sec)\n",
      "INFO:tensorflow:global_step/sec: 309.398\n",
      "INFO:tensorflow:loss = 0.012014686, step = 87401 (0.324 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.309\n",
      "INFO:tensorflow:loss = 0.008707531, step = 87501 (0.316 sec)\n",
      "INFO:tensorflow:global_step/sec: 300.172\n",
      "INFO:tensorflow:loss = 0.007192863, step = 87601 (0.333 sec)\n",
      "INFO:tensorflow:global_step/sec: 299.165\n",
      "INFO:tensorflow:loss = 0.011924338, step = 87701 (0.336 sec)\n",
      "INFO:tensorflow:global_step/sec: 285.389\n",
      "INFO:tensorflow:loss = 0.013436446, step = 87801 (0.353 sec)\n",
      "INFO:tensorflow:global_step/sec: 302.932\n",
      "INFO:tensorflow:loss = 0.010197293, step = 87901 (0.327 sec)\n",
      "INFO:tensorflow:global_step/sec: 294.814\n",
      "INFO:tensorflow:loss = 0.014043611, step = 88001 (0.338 sec)\n",
      "INFO:tensorflow:global_step/sec: 311.988\n",
      "INFO:tensorflow:loss = 0.01027085, step = 88101 (0.327 sec)\n",
      "INFO:tensorflow:global_step/sec: 294.88\n",
      "INFO:tensorflow:loss = 0.011637897, step = 88201 (0.332 sec)\n",
      "INFO:tensorflow:global_step/sec: 303.609\n",
      "INFO:tensorflow:loss = 0.014238432, step = 88301 (0.330 sec)\n",
      "INFO:tensorflow:global_step/sec: 287.759\n",
      "INFO:tensorflow:loss = 0.01166824, step = 88401 (0.348 sec)\n",
      "INFO:tensorflow:global_step/sec: 292.83\n",
      "INFO:tensorflow:loss = 0.010072509, step = 88501 (0.342 sec)\n",
      "INFO:tensorflow:global_step/sec: 293.841\n",
      "INFO:tensorflow:loss = 0.013280395, step = 88601 (0.342 sec)\n",
      "INFO:tensorflow:global_step/sec: 306.538\n",
      "INFO:tensorflow:loss = 0.0112317465, step = 88701 (0.324 sec)\n",
      "INFO:tensorflow:global_step/sec: 314.557\n",
      "INFO:tensorflow:loss = 0.01388762, step = 88801 (0.317 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.75\n",
      "INFO:tensorflow:loss = 0.019797388, step = 88901 (0.313 sec)\n",
      "INFO:tensorflow:global_step/sec: 325.577\n",
      "INFO:tensorflow:loss = 0.020835858, step = 89001 (0.305 sec)\n",
      "INFO:tensorflow:global_step/sec: 327.089\n",
      "INFO:tensorflow:loss = 0.011462444, step = 89101 (0.305 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.035\n",
      "INFO:tensorflow:loss = 0.015632685, step = 89201 (0.318 sec)\n",
      "INFO:tensorflow:global_step/sec: 298.581\n",
      "INFO:tensorflow:loss = 0.0063989875, step = 89301 (0.337 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 283.085\n",
      "INFO:tensorflow:loss = 0.0064634783, step = 89401 (0.353 sec)\n",
      "INFO:tensorflow:global_step/sec: 299.229\n",
      "INFO:tensorflow:loss = 0.015360342, step = 89501 (0.332 sec)\n",
      "INFO:tensorflow:global_step/sec: 299.435\n",
      "INFO:tensorflow:loss = 0.010400895, step = 89601 (0.334 sec)\n",
      "INFO:tensorflow:global_step/sec: 311.035\n",
      "INFO:tensorflow:loss = 0.005599436, step = 89701 (0.322 sec)\n",
      "INFO:tensorflow:global_step/sec: 301.039\n",
      "INFO:tensorflow:loss = 0.010744687, step = 89801 (0.331 sec)\n",
      "INFO:tensorflow:global_step/sec: 295.358\n",
      "INFO:tensorflow:loss = 0.012388031, step = 89901 (0.344 sec)\n",
      "INFO:tensorflow:global_step/sec: 319.242\n",
      "INFO:tensorflow:loss = 0.01192112, step = 90001 (0.309 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.833\n",
      "INFO:tensorflow:loss = 0.0075354977, step = 90101 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.975\n",
      "INFO:tensorflow:loss = 0.012779966, step = 90201 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 347.696\n",
      "INFO:tensorflow:loss = 0.014670095, step = 90301 (0.288 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.961\n",
      "INFO:tensorflow:loss = 0.0072291596, step = 90401 (0.301 sec)\n",
      "INFO:tensorflow:global_step/sec: 344.609\n",
      "INFO:tensorflow:loss = 0.010270989, step = 90501 (0.290 sec)\n",
      "INFO:tensorflow:global_step/sec: 337.395\n",
      "INFO:tensorflow:loss = 0.009464532, step = 90601 (0.298 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.536\n",
      "INFO:tensorflow:loss = 0.007902183, step = 90701 (0.290 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.837\n",
      "INFO:tensorflow:loss = 0.011717508, step = 90801 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 331.104\n",
      "INFO:tensorflow:loss = 0.012777325, step = 90901 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.405\n",
      "INFO:tensorflow:loss = 0.0074160434, step = 91001 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.708\n",
      "INFO:tensorflow:loss = 0.012485826, step = 91101 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 346.249\n",
      "INFO:tensorflow:loss = 0.008071447, step = 91201 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.417\n",
      "INFO:tensorflow:loss = 0.013975316, step = 91301 (0.260 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.028\n",
      "INFO:tensorflow:loss = 0.00720417, step = 91401 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 349.795\n",
      "INFO:tensorflow:loss = 0.017514024, step = 91501 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.38\n",
      "INFO:tensorflow:loss = 0.005741852, step = 91601 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.244\n",
      "INFO:tensorflow:loss = 0.012292721, step = 91701 (0.261 sec)\n",
      "INFO:tensorflow:global_step/sec: 362.293\n",
      "INFO:tensorflow:loss = 0.00921794, step = 91801 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 326.853\n",
      "INFO:tensorflow:loss = 0.009906772, step = 91901 (0.309 sec)\n",
      "INFO:tensorflow:global_step/sec: 324.987\n",
      "INFO:tensorflow:loss = 0.011629377, step = 92001 (0.301 sec)\n",
      "INFO:tensorflow:global_step/sec: 297.821\n",
      "INFO:tensorflow:loss = 0.005605506, step = 92101 (0.335 sec)\n",
      "INFO:tensorflow:global_step/sec: 304.993\n",
      "INFO:tensorflow:loss = 0.0045307437, step = 92201 (0.328 sec)\n",
      "INFO:tensorflow:global_step/sec: 313.298\n",
      "INFO:tensorflow:loss = 0.008470449, step = 92301 (0.319 sec)\n",
      "INFO:tensorflow:global_step/sec: 316.053\n",
      "INFO:tensorflow:loss = 0.01133478, step = 92401 (0.321 sec)\n",
      "INFO:tensorflow:global_step/sec: 344.251\n",
      "INFO:tensorflow:loss = 0.008800831, step = 92501 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 369.787\n",
      "INFO:tensorflow:loss = 0.00976243, step = 92601 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.675\n",
      "INFO:tensorflow:loss = 0.009846127, step = 92701 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 355.009\n",
      "INFO:tensorflow:loss = 0.016002495, step = 92801 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 408.886\n",
      "INFO:tensorflow:loss = 0.0117609445, step = 92901 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 336.431\n",
      "INFO:tensorflow:loss = 0.0155579, step = 93001 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.233\n",
      "INFO:tensorflow:loss = 0.010904625, step = 93101 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.378\n",
      "INFO:tensorflow:loss = 0.008038264, step = 93201 (0.293 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.319\n",
      "INFO:tensorflow:loss = 0.008628524, step = 93301 (0.259 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.425\n",
      "INFO:tensorflow:loss = 0.012041541, step = 93401 (0.277 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.47\n",
      "INFO:tensorflow:loss = 0.01198037, step = 93501 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 394.93\n",
      "INFO:tensorflow:loss = 0.0075548617, step = 93601 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 384.716\n",
      "INFO:tensorflow:loss = 0.0070423353, step = 93701 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 355.455\n",
      "INFO:tensorflow:loss = 0.0161485, step = 93801 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.523\n",
      "INFO:tensorflow:loss = 0.0058758687, step = 93901 (0.300 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.668\n",
      "INFO:tensorflow:loss = 0.0077335327, step = 94001 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.805\n",
      "INFO:tensorflow:loss = 0.011621432, step = 94101 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 359.014\n",
      "INFO:tensorflow:loss = 0.010682736, step = 94201 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 369.859\n",
      "INFO:tensorflow:loss = 0.013037581, step = 94301 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.604\n",
      "INFO:tensorflow:loss = 0.009288659, step = 94401 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 378.075\n",
      "INFO:tensorflow:loss = 0.009062381, step = 94501 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 364.844\n",
      "INFO:tensorflow:loss = 0.011259946, step = 94601 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.935\n",
      "INFO:tensorflow:loss = 0.012189042, step = 94701 (0.303 sec)\n",
      "INFO:tensorflow:global_step/sec: 323.435\n",
      "INFO:tensorflow:loss = 0.019454362, step = 94801 (0.303 sec)\n",
      "INFO:tensorflow:global_step/sec: 334.914\n",
      "INFO:tensorflow:loss = 0.016183678, step = 94901 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 366.156\n",
      "INFO:tensorflow:loss = 0.019367404, step = 95001 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 350.884\n",
      "INFO:tensorflow:loss = 0.0069637485, step = 95101 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.638\n",
      "INFO:tensorflow:loss = 0.009399349, step = 95201 (0.290 sec)\n",
      "INFO:tensorflow:global_step/sec: 347.493\n",
      "INFO:tensorflow:loss = 0.014892607, step = 95301 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 341.783\n",
      "INFO:tensorflow:loss = 0.007449462, step = 95401 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.906\n",
      "INFO:tensorflow:loss = 0.016335865, step = 95501 (0.279 sec)\n",
      "INFO:tensorflow:global_step/sec: 361.915\n",
      "INFO:tensorflow:loss = 0.0118721295, step = 95601 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.271\n",
      "INFO:tensorflow:loss = 0.011882946, step = 95701 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.237\n",
      "INFO:tensorflow:loss = 0.016062334, step = 95801 (0.300 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.301\n",
      "INFO:tensorflow:loss = 0.009712211, step = 95901 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.685\n",
      "INFO:tensorflow:loss = 0.010661191, step = 96001 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 379.291\n",
      "INFO:tensorflow:loss = 0.010320522, step = 96101 (0.262 sec)\n",
      "INFO:tensorflow:global_step/sec: 330.234\n",
      "INFO:tensorflow:loss = 0.005782142, step = 96201 (0.300 sec)\n",
      "INFO:tensorflow:global_step/sec: 363.063\n",
      "INFO:tensorflow:loss = 0.008214679, step = 96301 (0.284 sec)\n",
      "INFO:tensorflow:global_step/sec: 321.341\n",
      "INFO:tensorflow:loss = 0.012820529, step = 96401 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 296.867\n",
      "INFO:tensorflow:loss = 0.010140011, step = 96501 (0.337 sec)\n",
      "INFO:tensorflow:global_step/sec: 301.008\n",
      "INFO:tensorflow:loss = 0.007895282, step = 96601 (0.331 sec)\n",
      "INFO:tensorflow:global_step/sec: 324.301\n",
      "INFO:tensorflow:loss = 0.010195237, step = 96701 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.142\n",
      "INFO:tensorflow:loss = 0.012561707, step = 96801 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.025\n",
      "INFO:tensorflow:loss = 0.0052310387, step = 96901 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 330.199\n",
      "INFO:tensorflow:loss = 0.0128039075, step = 97001 (0.303 sec)\n",
      "INFO:tensorflow:global_step/sec: 373.815\n",
      "INFO:tensorflow:loss = 0.015987922, step = 97101 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 314.091\n",
      "INFO:tensorflow:loss = 0.0072436696, step = 97201 (0.319 sec)\n",
      "INFO:tensorflow:global_step/sec: 286.225\n",
      "INFO:tensorflow:loss = 0.013150087, step = 97301 (0.350 sec)\n",
      "INFO:tensorflow:global_step/sec: 194.641\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.007828134, step = 97401 (0.514 sec)\n",
      "INFO:tensorflow:global_step/sec: 224.604\n",
      "INFO:tensorflow:loss = 0.014394507, step = 97501 (0.441 sec)\n",
      "INFO:tensorflow:global_step/sec: 296.158\n",
      "INFO:tensorflow:loss = 0.013275066, step = 97601 (0.338 sec)\n",
      "INFO:tensorflow:global_step/sec: 196.911\n",
      "INFO:tensorflow:loss = 0.008199433, step = 97701 (0.509 sec)\n",
      "INFO:tensorflow:global_step/sec: 308.738\n",
      "INFO:tensorflow:loss = 0.012882975, step = 97801 (0.326 sec)\n",
      "INFO:tensorflow:global_step/sec: 279.348\n",
      "INFO:tensorflow:loss = 0.009654939, step = 97901 (0.355 sec)\n",
      "INFO:tensorflow:global_step/sec: 220.2\n",
      "INFO:tensorflow:loss = 0.015698953, step = 98001 (0.454 sec)\n",
      "INFO:tensorflow:global_step/sec: 270.433\n",
      "INFO:tensorflow:loss = 0.010378662, step = 98101 (0.371 sec)\n",
      "INFO:tensorflow:global_step/sec: 197.8\n",
      "INFO:tensorflow:loss = 0.011898065, step = 98201 (0.504 sec)\n",
      "INFO:tensorflow:global_step/sec: 232.261\n",
      "INFO:tensorflow:loss = 0.013942307, step = 98301 (0.431 sec)\n",
      "INFO:tensorflow:global_step/sec: 236.636\n",
      "INFO:tensorflow:loss = 0.015540784, step = 98401 (0.423 sec)\n",
      "INFO:tensorflow:global_step/sec: 233.792\n",
      "INFO:tensorflow:loss = 0.009599258, step = 98501 (0.428 sec)\n",
      "INFO:tensorflow:global_step/sec: 247.66\n",
      "INFO:tensorflow:loss = 0.009548711, step = 98601 (0.403 sec)\n",
      "INFO:tensorflow:global_step/sec: 259.349\n",
      "INFO:tensorflow:loss = 0.01138827, step = 98701 (0.387 sec)\n",
      "INFO:tensorflow:global_step/sec: 337.661\n",
      "INFO:tensorflow:loss = 0.009089751, step = 98801 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 326.524\n",
      "INFO:tensorflow:loss = 0.017089268, step = 98901 (0.308 sec)\n",
      "INFO:tensorflow:global_step/sec: 296.982\n",
      "INFO:tensorflow:loss = 0.008882964, step = 99001 (0.338 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.795\n",
      "INFO:tensorflow:loss = 0.015934268, step = 99101 (0.275 sec)\n",
      "INFO:tensorflow:global_step/sec: 281.03\n",
      "INFO:tensorflow:loss = 0.013887361, step = 99201 (0.359 sec)\n",
      "INFO:tensorflow:global_step/sec: 303.773\n",
      "INFO:tensorflow:loss = 0.008888858, step = 99301 (0.328 sec)\n",
      "INFO:tensorflow:global_step/sec: 329.048\n",
      "INFO:tensorflow:loss = 0.009305649, step = 99401 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 268.907\n",
      "INFO:tensorflow:loss = 0.008244978, step = 99501 (0.372 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.807\n",
      "INFO:tensorflow:loss = 0.006723667, step = 99601 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 239.993\n",
      "INFO:tensorflow:loss = 0.007904539, step = 99701 (0.417 sec)\n",
      "INFO:tensorflow:global_step/sec: 305.143\n",
      "INFO:tensorflow:loss = 0.008870687, step = 99801 (0.328 sec)\n",
      "INFO:tensorflow:global_step/sec: 310.022\n",
      "INFO:tensorflow:loss = 0.012510493, step = 99901 (0.324 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 100000 into /tmp/tmpyxwongpm/model.ckpt.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Importing frozen ensemble from /tmp/tmpyxwongpm/frozen/ensemble-3.meta with features: ['x'].\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-19-09:02:35\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from /tmp/tmpyxwongpm/model.ckpt-100000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving candidate 'previous_ensemble' dict for global step 100000: architecture/adanet/ensembles = b\"\\n\\x80\\x01\\n,adanet/previous_ensemble/architecture/adanetBF\\x08\\x07\\x12\\x00B@| b'linear' | b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.036644638, average_loss/adanet/subnetwork = 0.029604359, average_loss/adanet/uniform_average_ensemble = 0.034464236, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.040175013, loss/adanet/subnetwork = 0.03914282, loss/adanet/uniform_average_ensemble = 0.044524893, prediction/mean/adanet/adanet_weighted_ensemble = 3.080653, prediction/mean/adanet/subnetwork = 3.1459546, prediction/mean/adanet/uniform_average_ensemble = 3.1339185\n",
      "INFO:tensorflow:Saving candidate '2_layer_dnn' dict for global step 100000: architecture/adanet/ensembles = b\"\\n\\xa0\\x01\\n;adanet/iteration_4/ensemble_2_layer_dnn/architecture/adanetBW\\x08\\x07\\x12\\x00BQ| b'linear' | b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.033741787, average_loss/adanet/subnetwork = 0.034636576, average_loss/adanet/uniform_average_ensemble = 0.03350088, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.03923207, loss/adanet/subnetwork = 0.036618453, loss/adanet/uniform_average_ensemble = 0.041863397, prediction/mean/adanet/adanet_weighted_ensemble = 3.084939, prediction/mean/adanet/subnetwork = 3.0984871, prediction/mean/adanet/uniform_average_ensemble = 3.1268322\n",
      "INFO:tensorflow:Saving candidate '3_layer_dnn' dict for global step 100000: architecture/adanet/ensembles = b\"\\n\\xa0\\x01\\n;adanet/iteration_4/ensemble_3_layer_dnn/architecture/adanetBW\\x08\\x07\\x12\\x00BQ| b'linear' | b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' | b'3_layer_dnn' |J\\x08\\n\\x06\\n\\x04text\", average_loss/adanet/adanet_weighted_ensemble = 0.033751722, average_loss/adanet/subnetwork = 0.041267823, average_loss/adanet/uniform_average_ensemble = 0.03475043, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss/adanet/adanet_weighted_ensemble = 0.03923764, loss/adanet/subnetwork = 0.04582113, loss/adanet/uniform_average_ensemble = 0.043468982, prediction/mean/adanet/adanet_weighted_ensemble = 3.0848556, prediction/mean/adanet/subnetwork = 3.1456003, prediction/mean/adanet/uniform_average_ensemble = 3.1362548\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-19-09:02:37\n",
      "INFO:tensorflow:Saving dict for global step 100000: average_loss = 0.036644638, average_loss/adanet/adanet_weighted_ensemble = 0.036644638, average_loss/adanet/subnetwork = 0.029604359, average_loss/adanet/uniform_average_ensemble = 0.034464236, global_step = 100000, label/mean = 3.1049454, label/mean/adanet/adanet_weighted_ensemble = 3.1049454, label/mean/adanet/subnetwork = 3.1049454, label/mean/adanet/uniform_average_ensemble = 3.1049454, loss = 0.040175013, loss/adanet/adanet_weighted_ensemble = 0.040175013, loss/adanet/subnetwork = 0.03914282, loss/adanet/uniform_average_ensemble = 0.044524893, prediction/mean = 3.080653, prediction/mean/adanet/adanet_weighted_ensemble = 3.080653, prediction/mean/adanet/subnetwork = 3.1459546, prediction/mean/adanet/uniform_average_ensemble = 3.1339185\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 100000: /tmp/tmpyxwongpm/model.ckpt-100000\n",
      "INFO:tensorflow:Loss for final step: 0.011341246.\n",
      "Loss: 0.036644638\n",
      "Uniform average loss: 0.034464236\n",
      "Architecture: b\"| b'linear' | b'1_layer_dnn' | b'2_layer_dnn' | b'2_layer_dnn' |\"\n"
     ]
    }
   ],
   "source": [
    "#@test {\"skip\": true}\n",
    "results, _ = train_and_evaluate(learn_mixture_weights=True, adanet_lambda=.015)\n",
    "print(\"Loss:\", results[\"average_loss\"])\n",
    "print(\"Uniform average loss:\", results[\"average_loss/adanet/uniform_average_ensemble\"])\n",
    "print(\"Architecture:\", ensemble_architecture(results))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "s5RuM582oxo_"
   },
   "source": [
    "Learning the mixture weights with $\\lambda > 0$ produces a model with **0.0320**\n",
    "MSE. Notice that this is even better than the uniform average ensemble produced\n",
    "from the chosen subnetworks with **0.0345** MSE.\n",
    "\n",
    "Inspecting the ensemble architecture demonstrates the effects of complexity\n",
    "regularization on candidate selection. The selected subnetworks are relatively\n",
    "less complex: unlike in previous runs, the simplest subnetwork is a linear model\n",
    "and the deepest subnetwork has only 3 hidden layers.\n",
    "\n",
    "In general, learning to combine subnetwork ouputs with optimal hyperparameters\n",
    "should be at least as good assigning uniform average weights."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ajRhOvHGQu6w"
   },
   "source": [
    "## Conclusion\n",
    "\n",
    "In this tutorial, you were able to explore training an AdaNet model's mixture\n",
    "weights with $\\lambda \\ge 0$. You were also able to compare against building an\n",
    "ensemble formed by always choosing the best candidate subnetwork at each\n",
    "iteration based on it's ability to improve the ensemble's loss on the training\n",
    "set, and averaging their results.\n",
    "\n",
    "Uniform average ensembles work unreasonably well in practice, yet learning the\n",
    "mixture weights with the correct values of $\\lambda$ and $\\beta$ should always\n",
    "produce a better model when candidates have varying complexity. However, this\n",
    "does require some additional hyperparameter tuning, so practically you can train\n",
    "an AdaNet with the default mixture weights and $\\lambda=0$ first, and once you\n",
    "have confirmed that the subnetworks are training correctly, you can tune the\n",
    "mixture weight hyperparameters.\n",
    "\n",
    "While this example explored a regression task, these observations apply to using\n",
    "AdaNet on other tasks like binary-classification and multi-class classification."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "adanet_objective.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}