{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "id": "wJcYs_ERTnnI" }, "outputs": [], "source": [ "##### Copyright 2021 The TensorFlow Authors." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "cellView": "form", "id": "HMUDt0CiUJk9", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "77z2OchJTk0l" }, "source": [ "# 迁移指标和优化器\n", "\n", "\n", " \n", " \n", " \n", " \n", "
在 TensorFlow.org 上查看\n", " 在 Google Colab 运行\n", " 在 Github 上查看源代码\n", " 下载笔记本\n", "
" ] }, { "cell_type": "markdown", "metadata": { "id": "meUTrR4I6m1C" }, "source": [ "在 TF1 中，`tf.metrics` 是所有指标函数的 API 命名空间。每个指标都是一个将 `label` 和 `prediction` 作为输入参数，并返回相应指标张量作为结果的函数。在 TF2 中，`tf.keras.metrics` 包含所有指标函数和对象。`Metric` 对象可以与 `tf.keras.Model` 和 `tf.keras.layers.layer` 一起使用来计算指标值。" ] }, { "cell_type": "markdown", "metadata": { "id": "YdZSoIXEbhg-" }, "source": [ "## 安装\n", "\n", "从几个必要的 TensorFlow 导入开始：" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "iE0vSfMXumKI", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "import tensorflow as tf\n", "import tensorflow.compat.v1 as tf1" ] }, { "cell_type": "markdown", "metadata": { "id": "Jsm9Rxx7s1OZ" }, "source": [ "然后，准备一个用于演示的简单数据：" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "m7rnGxsXtDkV", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "features = [[1., 1.5], [2., 2.5], [3., 3.5]]\n", "labels = [0, 0, 1]\n", "eval_features = [[4., 4.5], [5., 5.5], [6., 6.5]]\n", "eval_labels = [0, 1, 1]" ] }, { "cell_type": "markdown", "metadata": { "id": "xswk0d4xrFaQ" }, "source": [ "## TF1：具有 Estimator 的 tf.compat.v1.metrics\n", "\n", "在 TF1 中，指标可以作为 `eval_metric_ops` 添加到 `EstimatorSpec` 中，并且运算通过 `tf.metrics` 中定义的所有指标函数生成。可以按照示例了解如何使用 `tf.metrics.accuracy`。" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "lqe9obf7suIj", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "def _input_fn():\n", " return tf1.data.Dataset.from_tensor_slices((features, labels)).batch(1)\n", "\n", "def _eval_input_fn():\n", " return tf1.data.Dataset.from_tensor_slices(\n", " (eval_features, eval_labels)).batch(1)\n", "\n", "def _model_fn(features, labels, mode):\n", " logits = tf1.layers.Dense(2)(features)\n", " predictions = tf.math.argmax(input=logits, axis=1)\n", " loss = tf1.nn.sparse_softmax_cross_entropy_with_logits(labels=labels, logits=logits)\n", " optimizer = tf1.train.AdagradOptimizer(0.05)\n", " train_op = optimizer.minimize(loss, global_step=tf1.train.get_global_step())\n", " accuracy = tf1.metrics.accuracy(labels=labels, predictions=predictions)\n", " return tf1.estimator.EstimatorSpec(mode, \n", " predictions=predictions,\n", " loss=loss, \n", " train_op=train_op,\n", " eval_metric_ops={'accuracy': accuracy})\n", "\n", "estimator = tf1.estimator.Estimator(model_fn=_model_fn)\n", "estimator.train(_input_fn)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "HsOpjW5plH9Q", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "estimator.evaluate(_eval_input_fn)" ] }, { "cell_type": "markdown", "metadata": { "id": "Wk4C6qA_OaQx" }, "source": [ "此外，可以通过 `tf.estimator.add_metrics()` 直接将指标添加到 Estimator 中。" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "B2lpLOh9Owma", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "def mean_squared_error(labels, predictions):\n", " labels = tf.cast(labels, predictions.dtype)\n", " return {\"mean_squared_error\": \n", " tf1.metrics.mean_squared_error(labels=labels, predictions=predictions)}\n", "\n", "estimator = tf1.estimator.add_metrics(estimator, mean_squared_error)\n", "estimator.evaluate(_eval_input_fn)" ] }, { "cell_type": "markdown", "metadata": { "id": "KEmzBjfnsxwT" }, "source": [ "## TF2：具有 tf.keras.Model 的 Keras Metrics API\n", "\n", "在 TF2 中，`tf.keras.metrics` 包含所有指标类和函数。它们以 OOP 风格设计，并与其他 `tf.keras` API 紧密集成。所有指标都可以在 `tf.keras.metrics` 命名空间中找到，并且 `tf.compat.v1.metrics` 与 `tf.keras.metrics` 之间通常存在直接映射。\n", "\n", "在以下示例中，指标添加到 `model.compile()` 方法中。用户只需要创建指标实例，无需指定标签和预测张量。Keras 模型会将模型输出和标签发送到指标对象。" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "atVciNgPs0fw", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "dataset = tf.data.Dataset.from_tensor_slices((features, labels)).batch(1)\n", "eval_dataset = tf.data.Dataset.from_tensor_slices(\n", " (eval_features, eval_labels)).batch(1)\n", "\n", "inputs = tf.keras.Input((2,))\n", "logits = tf.keras.layers.Dense(2)(inputs)\n", "predictions = tf.math.argmax(input=logits, axis=1)\n", "model = tf.keras.models.Model(inputs, predictions)\n", "optimizer = tf.keras.optimizers.Adagrad(learning_rate=0.05)\n", "\n", "model.compile(optimizer, loss='mse', metrics=[tf.keras.metrics.Accuracy()])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Kip65sYBlKiu", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "model.evaluate(eval_dataset, return_dict=True)" ] }, { "cell_type": "markdown", "metadata": { "id": "_mcGoCm_X1V0" }, "source": [ "启用 Eager Execution 后，`tf.keras.metrics.Metric` 实例可直接用于评估 numpy 数据或 Eager 张量。`tf.keras.metrics.Metric` 对象是有状态容器。指标值可以通过 `metric.update_state(y_true, y_pred)` 进行更新，结果可以通过 `metrics.result()` 进行检索。\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "TVGn5_IhYhtG", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "accuracy = tf.keras.metrics.Accuracy()\n", "\n", "accuracy.update_state(y_true=[0, 0, 1, 1], y_pred=[0, 0, 0, 1])\n", "accuracy.result().numpy()\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "wQEV2hHtY_su", "vscode": { "languageId": "python" } }, "outputs": [], "source": [ "accuracy.update_state(y_true=[0, 0, 1, 1], y_pred=[0, 0, 0, 0])\n", "accuracy.update_state(y_true=[0, 0, 1, 1], y_pred=[1, 1, 0, 0])\n", "accuracy.result().numpy()" ] }, { "cell_type": "markdown", "metadata": { "id": "E3F3ElcyadW-" }, "source": [ "有关 `tf.keras.metrics.Metric` 的更多详情，请查看 `tf.keras.metrics.Metric` 下的 API 文档以及[迁移指南](https://tensorflow.google.cn/guide/effective_tf2#new-style_metrics_and_losses)。" ] }, { "cell_type": "markdown", "metadata": { "id": "eXKY9HEulxQC" }, "source": [ "## 将 TF1.x 优化器迁移到 Keras 优化器\n", "\n", "`tf.compat.v1.train` 中的优化器（如 [Adam 优化器](https://tensorflow.google.cn/api_docs/python/tf/compat/v1/train/AdamOptimizer)和[梯度下降优化器](https://tensorflow.google.cn/api_docs/python/tf/compat/v1/train/GradientDescentOptimizer)）在 `tf.keras.optimizers` 中具有等效项。\n", "\n", "下表总结了如何将这些旧版优化器转换为 Keras 等效项。除非需要额外的步骤（例如[更新默认学习率](../../guide/effective_tf2.ipynb#optimizer_defaults)），否则可以直接将 TF1.x 版本替换为 TF2 版本。\n", "\n", "请注意，转换优化器[可能会使旧的检查点不兼容](./migrating_checkpoints.ipynb)。\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
TF1.xTF2额外步骤
`tf.v1.train.GradientDescentOptimizer``tf.keras.optimizers.SGD`无
`tf.v1.train.MomentumOptimizer``tf.keras.optimizers.SGD`包含 `momentum` 参数
`tf.v1.train.AdamOptimizer``tf.keras.optimizers.Adam`将 `beta1` 和 `beta2` 参数重命名为 `beta_1` 和 `beta_2`
`tf.v1.train.RMSPropOptimizer``tf.keras.optimizers.RMSprop`将 `decay` 参数重命名为 `rho`
`tf.v1.train.AdadeltaOptimizer``tf.keras.optimizers.Adadelta`无
`tf.v1.train.AdagradOptimizer``tf.keras.optimizers.Adagrad`无
`tf.v1.train.FtrlOptimizer``tf.keras.optimizers.Ftrl`移除 `accum_name` 和 `linear_name` 参数
`tf.contrib.AdamaxOptimizer``tf.keras.optimizers.Adamax`将 `beta1` 和 `beta2` 参数重命名为 `beta_1` 和 `beta_2`
`tf.contrib.Nadam``tf.keras.optimizers.Nadam`将 `beta1` 和 `beta2` 参数重命名为 `beta_1` 和 `beta_2`
\n", "\n", "注：在 TF2 中，所有 ε（数值稳定性常数）现在默认为 `1e-7`，而不是 `1e-8`。在大多数用例中，这种差异可以忽略不计。" ] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "metrics_optimizers.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 0 }