{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "b518b04cbfe0"
      },
      "outputs": [],
      "source": [
        "##### Copyright 2020 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "906e07f6e562",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3e19705694d6"
      },
      "source": [
        "# <a>序贯模型</a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "defbb10e8ae3"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>     <a target=\"_blank\" href=\"https://tensorflow.google.cn/guide/keras/sequential_model\"><img src=\"https://tensorflow.google.cn/images/tf_logo_32px.png\">在 TensorFlow.org 上查看</a>\n",
        "</td>\n",
        "  <td>     <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs-l10n/blob/master/site/zh-cn/guide/keras/sequential_model.ipynb\"><img src=\"https://tensorflow.google.cn/images/colab_logo_32px.png\">在 Google Colab 中运行</a>\n",
        "</td>\n",
        "  <td>     <a target=\"_blank\" href=\"https://github.com/tensorflow/docs-l10n/blob/master/site/zh-cn/guide/keras/sequential_model.ipynb\"><img src=\"https://tensorflow.google.cn/images/GitHub-Mark-32px.png\">在 GitHub 中查看源代码</a>\n",
        "</td>\n",
        "  <td>     <a href=\"https://storage.googleapis.com/tensorflow_docs/docs-l10n/site/zh-cn/guide/keras/sequential_model.ipynb\"><img src=\"https://tensorflow.google.cn/images/download_logo_32px.png\">下载笔记本</a>\n",
        "</td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8d4ac441b1fc"
      },
      "source": [
        "## 安装"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0472bf67b2bf",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "80f7713c6b92"
      },
      "source": [
        "## 何时使用序贯模型\n",
        "\n",
        "`Sequential` 模型适用于**简单的层堆栈**，其中每个层都**恰好有一个输入张量和一个输出张量**。\n",
        "\n",
        "以下 `Sequential` 模型（仅作为示意）："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f536515be229",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "# Define Sequential model with 3 layers\n",
        "model = keras.Sequential(\n",
        "    [\n",
        "        layers.Dense(2, activation=\"relu\", name=\"layer1\"),\n",
        "        layers.Dense(3, activation=\"relu\", name=\"layer2\"),\n",
        "        layers.Dense(4, name=\"layer3\"),\n",
        "    ]\n",
        ")\n",
        "# Call model on a test input\n",
        "x = tf.ones((3, 3))\n",
        "y = model(x)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7d81502d9753"
      },
      "source": [
        "等效于此函数："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7059a8890f72",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "# Create 3 layers\n",
        "layer1 = layers.Dense(2, activation=\"relu\", name=\"layer1\")\n",
        "layer2 = layers.Dense(3, activation=\"relu\", name=\"layer2\")\n",
        "layer3 = layers.Dense(4, name=\"layer3\")\n",
        "\n",
        "# Call layers on a test input\n",
        "x = tf.ones((3, 3))\n",
        "y = layer3(layer2(layer1(x)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cdf6d2932c31"
      },
      "source": [
        "在以下情况下，序贯模型**不适用**：\n",
        "\n",
        "- 您的模型有多个输入或多个输出\n",
        "- 您的任何层都有多个输入或多个输出\n",
        "- 您需要进行层共享\n",
        "- 您需要非线性拓扑（例如残差连接、多分支模型）"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c5706d9f8eb8"
      },
      "source": [
        "## 创建序贯模型\n",
        "\n",
        "您可以通过将层列表传递给序贯构造函数来创建序贯模型："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8b3eee00f80d",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "model = keras.Sequential(\n",
        "    [\n",
        "        layers.Dense(2, activation=\"relu\"),\n",
        "        layers.Dense(3, activation=\"relu\"),\n",
        "        layers.Dense(4),\n",
        "    ]\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1939a7a4a66c"
      },
      "source": [
        "它的层可以通过 `layers` 属性访问："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "49c0448b6da2",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "model.layers"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b4c7957e9913"
      },
      "source": [
        "您还可以通过 `add()` 方法增量式创建序贯模型："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "d54fde401054",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "model = keras.Sequential()\n",
        "model.add(layers.Dense(2, activation=\"relu\"))\n",
        "model.add(layers.Dense(3, activation=\"relu\"))\n",
        "model.add(layers.Dense(4))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d16278f5a1dc"
      },
      "source": [
        "请注意，还有一个相应的 `pop()` 方法来移除层：序贯模型的行为非常类似于层列表。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "e89f35b73979",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "model.pop()\n",
        "print(len(model.layers))  # 2"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "99cb1c9a7c7a"
      },
      "source": [
        "另请注意，序贯构造函数接受 `name` 参数，就像 Keras 中的任何层或模型一样。这对于使用语义上有意义的名称来注释 TensorBoard 计算图非常有用。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "068c2f82e7cb",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "model = keras.Sequential(name=\"my_sequential\")\n",
        "model.add(layers.Dense(2, activation=\"relu\", name=\"layer1\"))\n",
        "model.add(layers.Dense(3, activation=\"relu\", name=\"layer2\"))\n",
        "model.add(layers.Dense(4, name=\"layer3\"))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a6247ff17d3a"
      },
      "source": [
        "## 预先指定输入形状\n",
        "\n",
        "一般来说，Keras 中的所有层都需要知道其输入的形状，以便能够创建其权重。因此，当您创建这样的层时，它最初没有权重："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "373ecbb5c6bd",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "layer = layers.Dense(3)\n",
        "layer.weights  # Empty"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "da150335961e"
      },
      "source": [
        "当第一次在输入上被调用时，它会创建其权重，因为权重的形状取决于输入的形状："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bf28829ce193",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "# Call layer on a test input\n",
        "x = tf.ones((1, 4))\n",
        "y = layer(x)\n",
        "layer.weights  # Now it has weights, of shape (4, 3) and (3,)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e50f21c5f247"
      },
      "source": [
        "当然，这也适用于序贯模型。当您实例化没有输入形状的序贯模型时，它不会被“构建”：它没有权重（并且调用 `model.weights` 会导致说明这一点的错误）。当模型第一次看到一些输入数据时，会创建权重："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "04479960526c",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "model = keras.Sequential(\n",
        "    [\n",
        "        layers.Dense(2, activation=\"relu\"),\n",
        "        layers.Dense(3, activation=\"relu\"),\n",
        "        layers.Dense(4),\n",
        "    ]\n",
        ")  # No weights at this stage!\n",
        "\n",
        "# At this point, you can't do this:\n",
        "# model.weights\n",
        "\n",
        "# You also can't do this:\n",
        "# model.summary()\n",
        "\n",
        "# Call the model on a test input\n",
        "x = tf.ones((1, 4))\n",
        "y = model(x)\n",
        "print(\"Number of weights after calling the model:\", len(model.weights))  # 6"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2837e3d2c798"
      },
      "source": [
        "一旦模型“已构建”，您就可以调用它的 `summary()` 方法来显示其内容："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1368bd27f679",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "model.summary()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "08cf1da27edb"
      },
      "source": [
        "不过，在增量式构建序贯模型时，它非常有用，能够显示迄今为止模型的摘要，包括当前的输出形状。在这种情况下，您应通过将 `Input` 对象传递给您的模型来启动模型，以便模型从一开始就知道其输入形状："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "e3d2024cfeeb",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "model = keras.Sequential()\n",
        "model.add(keras.Input(shape=(4,)))\n",
        "model.add(layers.Dense(2, activation=\"relu\"))\n",
        "\n",
        "model.summary()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3d965e3761a8"
      },
      "source": [
        "请注意，`Input` 对象不会显示为 `model.layers` 的一部分，因为它不是层："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8e3b0d58e7ed",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "model.layers"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8a057b1baf72"
      },
      "source": [
        "一种简单的替代方式是将 `input_shape` 参数传递给第一层："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1c6ab83d68ea",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "model = keras.Sequential()\n",
        "model.add(layers.Dense(2, activation=\"relu\", input_shape=(4,)))\n",
        "\n",
        "model.summary()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "40c14619d283"
      },
      "source": [
        "使用像这样的预定义输入形状构建的模型始终具有权重（甚至在看到任何数据之前），并且始终具有定义的输出形状。\n",
        "\n",
        "一般来说，如果您知道序贯模型的输入形状是什么，推荐的最佳做法是始终提前指定它。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "843f6b6505b3"
      },
      "source": [
        "## 常见的调试工作流：`add()` + `summary()`\n",
        "\n",
        "在构建新的序贯架构时，使用 `add()` 增量式堆叠层并经常打印模型摘要非常有用。例如，这样便能监控 `Conv2D` 和 `MaxPooling2D` 层的堆栈如何对图像特征映射进行下采样："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "46bfb8f7dc6e",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "model = keras.Sequential()\n",
        "model.add(keras.Input(shape=(250, 250, 3)))  # 250x250 RGB images\n",
        "model.add(layers.Conv2D(32, 5, strides=2, activation=\"relu\"))\n",
        "model.add(layers.Conv2D(32, 3, activation=\"relu\"))\n",
        "model.add(layers.MaxPooling2D(3))\n",
        "\n",
        "# Can you guess what the current output shape is at this point? Probably not.\n",
        "# Let's just print it:\n",
        "model.summary()\n",
        "\n",
        "# The answer was: (40, 40, 32), so we can keep downsampling...\n",
        "\n",
        "model.add(layers.Conv2D(32, 3, activation=\"relu\"))\n",
        "model.add(layers.Conv2D(32, 3, activation=\"relu\"))\n",
        "model.add(layers.MaxPooling2D(3))\n",
        "model.add(layers.Conv2D(32, 3, activation=\"relu\"))\n",
        "model.add(layers.Conv2D(32, 3, activation=\"relu\"))\n",
        "model.add(layers.MaxPooling2D(2))\n",
        "\n",
        "# And now?\n",
        "model.summary()\n",
        "\n",
        "# Now that we have 4x4 feature maps, time to apply global max pooling.\n",
        "model.add(layers.GlobalMaxPooling2D())\n",
        "\n",
        "# Finally, we add a classification layer.\n",
        "model.add(layers.Dense(10))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a2d3335a90fa"
      },
      "source": [
        "非常实用，对吧？\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "46addede37f3"
      },
      "source": [
        "## 有了模型后该怎么办\n",
        "\n",
        "一旦模型架构准备就绪，您将需要执行以下操作：\n",
        "\n",
        "- 训练您的模型、评估模型并运行推断。请参阅我们的[使用内置循环的训练和评估指南](https://tensorflow.google.cn/guide/keras/train_and_evaluate/)\n",
        "- 将模型保存到磁盘并将其还原。请参阅我们的[序列化和保存指南](https://tensorflow.google.cn/guide/keras/save_and_serialize/)。\n",
        "- 利用多个 GPU 加速模型训练。请参阅我们的[多 GPU 和分布式训练指南](https://keras.io/guides/distributed_training/)。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "608f3b03669c"
      },
      "source": [
        "## 使用序贯模型进行特征提取\n",
        "\n",
        "一旦构建了序贯模型，它的行为就类似于[函数式 API 模型](https://tensorflow.google.cn/guide/keras/functional/)。这意味着每层都有一个 `input` 和 `output` 属性。这些属性可用于执行一些巧妙的操作，例如快速创建一个模型来提取序贯模型中所有中间层的输出："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "a5888d753301",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "initial_model = keras.Sequential(\n",
        "    [\n",
        "        keras.Input(shape=(250, 250, 3)),\n",
        "        layers.Conv2D(32, 5, strides=2, activation=\"relu\"),\n",
        "        layers.Conv2D(32, 3, activation=\"relu\"),\n",
        "        layers.Conv2D(32, 3, activation=\"relu\"),\n",
        "    ]\n",
        ")\n",
        "feature_extractor = keras.Model(\n",
        "    inputs=initial_model.inputs,\n",
        "    outputs=[layer.output for layer in initial_model.layers],\n",
        ")\n",
        "\n",
        "# Call feature extractor on test input.\n",
        "x = tf.ones((1, 250, 250, 3))\n",
        "features = feature_extractor(x)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4abef35355d3"
      },
      "source": [
        "下面是一个仅从一层提取特征的类似示例："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fc404c7ac90e",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "initial_model = keras.Sequential(\n",
        "    [\n",
        "        keras.Input(shape=(250, 250, 3)),\n",
        "        layers.Conv2D(32, 5, strides=2, activation=\"relu\"),\n",
        "        layers.Conv2D(32, 3, activation=\"relu\", name=\"my_intermediate_layer\"),\n",
        "        layers.Conv2D(32, 3, activation=\"relu\"),\n",
        "    ]\n",
        ")\n",
        "feature_extractor = keras.Model(\n",
        "    inputs=initial_model.inputs,\n",
        "    outputs=initial_model.get_layer(name=\"my_intermediate_layer\").output,\n",
        ")\n",
        "# Call feature extractor on test input.\n",
        "x = tf.ones((1, 250, 250, 3))\n",
        "features = feature_extractor(x)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4e2fb64f0676"
      },
      "source": [
        "## 使用序贯模型进行迁移学习\n",
        "\n",
        "迁移学习包括冻结模型中的底层并仅训练顶层。如果您不熟悉迁移学习，请务必阅读我们的[迁移学习指南](https://tensorflow.google.cn/guide/keras/transfer_learning/)。\n",
        "\n",
        "下面是涉及序贯模型的两种常见迁移学习蓝图。\n",
        "\n",
        "首先，假设您有一个序贯模型，并且想要冻结除最后一层之外的所有层。在这种情况下，只需迭代 `model.layers` 并在除最后一层之外的每一层上设置 `layer.trainable = False`。示例代码如下：\n",
        "\n",
        "```python\n",
        "model = keras.Sequential([\n",
        "    keras.Input(shape=(784)),\n",
        "    layers.Dense(32, activation='relu'),\n",
        "    layers.Dense(32, activation='relu'),\n",
        "    layers.Dense(32, activation='relu'),\n",
        "    layers.Dense(10),\n",
        "])\n",
        "\n",
        "# Presumably you would want to first load pre-trained weights.\n",
        "model.load_weights(...)\n",
        "\n",
        "# Freeze all layers except the last one.\n",
        "for layer in model.layers[:-1]:\n",
        "  layer.trainable = False\n",
        "\n",
        "# Recompile and train (this will only update the weights of the last layer).\n",
        "model.compile(...)\n",
        "model.fit(...)\n",
        "```\n",
        "\n",
        "另一个常见的蓝图是使用序贯模型来堆叠预训练模型和一些新初始化的分类层。示例代码如下：\n",
        "\n",
        "```python\n",
        "# Load a convolutional base with pre-trained weights\n",
        "base_model = keras.applications.Xception(\n",
        "    weights='imagenet',\n",
        "    include_top=False,\n",
        "    pooling='avg')\n",
        "\n",
        "# Freeze the base model\n",
        "base_model.trainable = False\n",
        "\n",
        "# Use a Sequential model to add a trainable classifier on top\n",
        "model = keras.Sequential([\n",
        "    base_model,\n",
        "    layers.Dense(1000),\n",
        "])\n",
        "\n",
        "# Compile & train\n",
        "model.compile(...)\n",
        "model.fit(...)\n",
        "```\n",
        "\n",
        "如果您进行迁移学习，您可能会发现自己经常使用这两种模式。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fcffc33b61e5"
      },
      "source": [
        "上面是您需要了解的有关序贯模型的全部信息！\n",
        "\n",
        "要详细了解如何在 Keras 中构建模型，请参阅：\n",
        "\n",
        "- [函数式 API 指南](https://tensorflow.google.cn/guide/keras/functional/)\n",
        "- [通过子类化创建新层和模型的指南](https://tensorflow.google.cn/guide/keras/custom_layers_and_models/)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "sequential_model.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}