{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "id": "wJcYs_ERTnnI" }, "outputs": [], "source": [ "##### Copyright 2021 The TensorFlow Authors." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "cellView": "form", "id": "HMUDt0CiUJk9", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "77z2OchJTk0l" }, "source": [ "# 迁移评估\n", "\n", "\n", " \n", " \n", " \n", " \n", "
在 TensorFlow.org 上查看 在 Google Colab 运行\n", " 在 Github 上查看源代码\n", " 下载笔记本
" ] }, { "cell_type": "markdown", "metadata": { "id": "n4O6fPyYTxZv" }, "source": [ "评估是对模型进行衡量和基准测试的关键部分。\n", "\n", "本指南演示了如何将评估器任务从 TensorFlow 1 迁移到 TensorFlow 2。在 TensorFlow 1 中，当 API 以分布式方式运行时，此功能由 `tf.estimator.train_and_evaluate` 实现。在 Tensorflow 2 中，可以使用内置 `tf.keras.utils.SidecarEvaluator`，或在评估器任务上使用自定义评估循环。\n", "\n", "TensorFlow 1 (`tf.estimator.Estimator.evaluate`) 和 TensorFlow 2（`Model.fit(..., validation_data=(...))` 或 `Model.evaluate`）中都有简单的连续评估选项。当您不希望工作进程在训练和评估之间切换时，评估器任务更合适，而当您希望分布评估时，`Model.fit` 中的内置评估更合适。\n" ] }, { "cell_type": "markdown", "metadata": { "id": "pHJfmkCFUhQf" }, "source": [ "## 安装" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "VXnPvQi8Ui1F", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "import tensorflow.compat.v1 as tf1\n", "import tensorflow as tf\n", "import numpy as np\n", "import tempfile\n", "import time\n", "import os" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Tww-uIoiUlsT", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "mnist = tf.keras.datasets.mnist\n", "\n", "(x_train, y_train),(x_test, y_test) = mnist.load_data()\n", "x_train, x_test = x_train / 255.0, x_test / 255.0" ] }, { "cell_type": "markdown", "metadata": { "id": "TtlucRG_Uro_" }, "source": [ "## TensorFlow 1：使用 tf.estimator.train_and_evaluate 进行评估\n", "\n", "在 TensorFlow 1 中，可以配置 `tf.estimator` 以使用 `tf.estimator.train_and_evaluate` 评估 Estimator。\n", "\n", "在此示例中，首先定义 `tf.estimator.Estimator` 并指定训练和评估规范：" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Q8shCkV2jKcc", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "feature_columns = [tf1.feature_column.numeric_column(\"x\", shape=[28, 28])]\n", "\n", "classifier = tf1.estimator.DNNClassifier(\n", " feature_columns=feature_columns,\n", " hidden_units=[256, 32],\n", " optimizer=tf1.train.AdamOptimizer(0.001),\n", " n_classes=10,\n", " dropout=0.2\n", ")\n", "\n", "train_input_fn = tf1.estimator.inputs.numpy_input_fn(\n", " x={\"x\": x_train},\n", " y=y_train.astype(np.int32),\n", " num_epochs=10,\n", " batch_size=50,\n", " shuffle=True,\n", ")\n", "\n", "test_input_fn = tf1.estimator.inputs.numpy_input_fn(\n", " x={\"x\": x_test},\n", " y=y_test.astype(np.int32),\n", " num_epochs=10,\n", " shuffle=False\n", ")\n", "\n", "train_spec = tf1.estimator.TrainSpec(input_fn=train_input_fn, max_steps=10)\n", "eval_spec = tf1.estimator.EvalSpec(input_fn=test_input_fn,\n", " steps=10,\n", " throttle_secs=0)" ] }, { "cell_type": "markdown", "metadata": { "id": "sGP7Nyenk1gr" }, "source": [ "随后，训练和评估模型。评估在训练之间同步运行，因为它在此笔记本中被限制为本地运行，并且在训练和评估之间交替运行。但是，如果 Estimator 是以分布式方式使用的，则评估器将作为专用评估器任务运行。有关详情，请查看[分布式训练的迁移指南](https://tensorflow.google.cn/guide/migrate/multi_worker_cpu_gpu_training)。" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "xWKMsmt6jYSL", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "tf1.estimator.train_and_evaluate(estimator=classifier,\n", " train_spec=train_spec,\n", " eval_spec=eval_spec)" ] }, { "cell_type": "markdown", "metadata": { "id": "T5LtVtmvYx7J" }, "source": [ "## TensorFlow 2：评估 Keras 模型\n", "\n", "在 TensorFlow 2 中，如果您使用 `Model.fit` API 进行训练，则可以使用 `tf.keras.utils.SidecarEvaluator` 评估模型。此外，还可以在 Tensorboard 中呈现评估指标，本指南中未介绍此功能。\n", "\n", "为了帮助演示这一点，我们首先定义和训练模型：\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Ci3yB6A5lwJu", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "def create_model():\n", " return tf.keras.models.Sequential([\n", " tf.keras.layers.Flatten(input_shape=(28, 28)),\n", " tf.keras.layers.Dense(512, activation='relu'),\n", " tf.keras.layers.Dropout(0.2),\n", " tf.keras.layers.Dense(10)\n", " ])\n", "\n", "loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)\n", "\n", "model = create_model()\n", "model.compile(optimizer='adam',\n", " loss=loss,\n", " metrics=['accuracy'],\n", " steps_per_execution=10,\n", " run_eagerly=True)\n", "\n", "log_dir = tempfile.mkdtemp()\n", "model_checkpoint = tf.keras.callbacks.ModelCheckpoint(\n", " filepath=os.path.join(log_dir, 'ckpt-{epoch}'),\n", " save_weights_only=True)\n", "\n", "model.fit(x=x_train,\n", " y=y_train,\n", " epochs=1,\n", " callbacks=[model_checkpoint])" ] }, { "cell_type": "markdown", "metadata": { "id": "AhU3VTYZoDh-" }, "source": [ "然后，使用 `tf.keras.utils.SidecarEvaluator` 评估模型。在实际训练中，建议使用单独的作业进行评估，以释放工作进程资源进行训练。" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "1VOQLDNkl2bl", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "data = tf.data.Dataset.from_tensor_slices((x_test, y_test))\n", "data = data.batch(64)\n", "\n", "tf.keras.utils.SidecarEvaluator(\n", " model=model,\n", " data=data,\n", " checkpoint_dir=log_dir,\n", " max_evaluations=1\n", ").start()" ] }, { "cell_type": "markdown", "metadata": { "id": "rQUS8nO9FZlH" }, "source": [ "## 后续步骤\n", "\n", "- 要详细了解 sidecar 评估，请考虑阅读 `tf.keras.utils.SidecarEvaluator` API 文档。\n", "- 要考虑在 Keras 中交替进行训练和评估，请考虑阅读[其他内置方法](https://tensorflow.google.cn/guide/keras/train_and_evaluate)。" ] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "evaluator.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 0 }