{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": { "id": "6bYaCABobL5q" }, "outputs": [], "source": [ "##### Copyright 2018 The TensorFlow Authors." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "cellView": "form", "id": "FlUw7tSKbtg4" }, "outputs": [], "source": [ "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "xc1srSc51n_4" }, "source": [ "# 使用 SavedModel 格式" ] }, { "cell_type": "markdown", "metadata": { "id": "-nBUqG2rchGH" }, "source": [ "
![]() | \n",
" ![]() | \n",
" ![]() | \n",
" ![]() | \n",
"
ValueError: Could not find matching function to call for canonicalized inputs ((<tf.Tensor 'args_0:0' shape=(1,) dtype=float32>,), {}). Only existing signatures are [((TensorSpec(shape=(), dtype=tf.float32, name=u'x'),), {})].\n", "" ] }, { "cell_type": "markdown", "metadata": { "id": "4Vsva3UZ-2sf" }, "source": [ "### 基本微调\n", "\n", "可以使用变量对象,还可以通过导入的函数向后传播。对于简单情形,这足以支持 SavedModel 的微调(即重新训练)。" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "PEkQNarJ-7nT" }, "outputs": [], "source": [ "optimizer = tf.keras.optimizers.SGD(0.05)\n", "\n", "def train_step():\n", " with tf.GradientTape() as tape:\n", " loss = (10. - imported(tf.constant(2.))) ** 2\n", " variables = tape.watched_variables()\n", " grads = tape.gradient(loss, variables)\n", " optimizer.apply_gradients(zip(grads, variables))\n", " return loss" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "p41NM6fF---3" }, "outputs": [], "source": [ "for _ in range(10):\n", " # \"v\" approaches 5, \"loss\" approaches 0\n", " print(\"loss={:.2f} v={:.2f}\".format(train_step(), imported.v.numpy()))" ] }, { "cell_type": "markdown", "metadata": { "id": "XuXtkHSD_KSW" }, "source": [ "### 一般微调\n", "\n", "与普通 `__call__` 相比,Keras 的 SavedModel 提供了[更多详细信息](https://github.com/tensorflow/community/blob/master/rfcs/20190509-keras-saved-model.md#serialization-details)来解决更复杂的微调情形。TensorFlow Hub 建议在共享的 SavedModel 中提供以下详细信息(如果适用),以便进行微调:\n", "\n", "- 如果模型使用随机失活,或者是训练与推断之间的前向传递不同的另一种技术(如批次归一化),则 `__call__` 方法会获取一个可选的 Python 值 `training=` 参数。该参数的默认值为 `False`,但可将其设置为 `True`。\n", "- 对于变量的对应列表,除了 `__call__` 特性,还有 `.variable` 和 `.trainable_variable` 特性。在微调过程中,`.trainable_variables` 省略了一个变量,该变量原本可训练,但打算将其冻结。\n", "- 对于 Keras 等将权重正则化项表示为层或子模型特性的框架,还有一个 `.regularization_losses` 特性。它包含一个零参数函数的列表,这些函数的值应加到总损失中。\n", "\n", "回到初始 MobileNet 示例,您可以看到一些具体操作:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Y6EUFdY8_PRD" }, "outputs": [], "source": [ "loaded = tf.saved_model.load(mobilenet_save_path)\n", "print(\"MobileNet has {} trainable variables: {}, ...\".format(\n", " len(loaded.trainable_variables),\n", " \", \".join([v.name for v in loaded.trainable_variables[:5]])))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "B-mQJ8iP_R0h" }, "outputs": [], "source": [ "trainable_variable_ids = {id(v) for v in loaded.trainable_variables}\n", "non_trainable_variables = [v for v in loaded.variables\n", " if id(v) not in trainable_variable_ids]\n", "print(\"MobileNet also has {} non-trainable variables: {}, ...\".format(\n", " len(non_trainable_variables),\n", " \", \".join([v.name for v in non_trainable_variables[:3]])))" ] }, { "cell_type": "markdown", "metadata": { "id": "qGlHlbd3_eyO" }, "source": [ "## 导出时指定签名\n", "\n", "TensorFlow Serving 之类的工具和 `saved_model_cli` 可以与 SavedModel 交互。为了帮助这些工具确定要使用的 ConcreteFunction,我们需要指定应用签名。`tf.keras.Model` 会自动指定应用签名,但是,对于自定义模块,我们必须明确声明应用签名。\n", "\n", "重要提示:除非您需要使用 Python 将模型导出到 TensorFlow 2.x 之外的环境,否则您不需要明确导出签名。如果您在寻找为特定函数强制输入签名的方式,请参阅 `tf.function` 的 [`input_signature`](https://tensorflow.google.cn/api_docs/python/tf/function#args_1) 参数。\n", "\n", "默认情况下,自定义 `tf.Module` 中不会声明签名。" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "h-IB5Xa0NxLa" }, "outputs": [], "source": [ "assert len(imported.signatures) == 0" ] }, { "cell_type": "markdown", "metadata": { "id": "BiNtaMZSI8Tb" }, "source": [ "要声明应用签名,请使用 `signatures` 关键字参数指定 ConcreteFunction。指定单个签名时,签名键为 `'serving_default'`,并将保存为常量 `tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY`。" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "_pAdgIORR2yH" }, "outputs": [], "source": [ "module_with_signature_path = os.path.join(tmpdir, 'module_with_signature')\n", "call = module.__call__.get_concrete_function(tf.TensorSpec(None, tf.float32))\n", "tf.saved_model.save(module, module_with_signature_path, signatures=call)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "nAzRHR0UT4hv" }, "outputs": [], "source": [ "imported_with_signatures = tf.saved_model.load(module_with_signature_path)\n", "list(imported_with_signatures.signatures.keys())\n" ] }, { "cell_type": "markdown", "metadata": { "id": "_gH91j1IR4tq" }, "source": [ "要导出多个签名,请将签名键的字典传递给 ConcreteFunction。每个签名键对应一个 ConcreteFunction。" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "6VYAiQmLUiox" }, "outputs": [], "source": [ "module_multiple_signatures_path = os.path.join(tmpdir, 'module_with_multiple_signatures')\n", "signatures = {\"serving_default\": call,\n", " \"array_input\": module.__call__.get_concrete_function(tf.TensorSpec([None], tf.float32))}\n", "\n", "tf.saved_model.save(module, module_multiple_signatures_path, signatures=signatures)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "8IPx_0RWEx07" }, "outputs": [], "source": [ "imported_with_multiple_signatures = tf.saved_model.load(module_multiple_signatures_path)\n", "list(imported_with_multiple_signatures.signatures.keys())" ] }, { "cell_type": "markdown", "metadata": { "id": "43_Qv2W_DJZZ" }, "source": [ "默认情况下,输出张量名称非常通用,如 `output_0`。为了控制输出的名称,请修改 `tf.function`,以便返回将输出名称映射到输出的字典。输入的名称来自 Python 函数参数名称。" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "ACKPl1X8G1gw" }, "outputs": [], "source": [ "class CustomModuleWithOutputName(tf.Module):\n", " def __init__(self):\n", " super(CustomModuleWithOutputName, self).__init__()\n", " self.v = tf.Variable(1.)\n", "\n", " @tf.function(input_signature=[tf.TensorSpec(None, tf.float32)])\n", " def __call__(self, x):\n", " return {'custom_output_name': x * self.v}\n", "\n", "module_output = CustomModuleWithOutputName()\n", "call_output = module_output.__call__.get_concrete_function(tf.TensorSpec(None, tf.float32))\n", "module_output_path = os.path.join(tmpdir, 'module_with_output_name')\n", "tf.saved_model.save(module_output, module_output_path,\n", " signatures={'serving_default': call_output})" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "1yGVy4MuH-V0" }, "outputs": [], "source": [ "imported_with_output_name = tf.saved_model.load(module_output_path)\n", "imported_with_output_name.signatures['serving_default'].structured_outputs" ] }, { "cell_type": "markdown", "metadata": { "id": "Q4bCK55x1IBW" }, "source": [ "## proto 分割\n", "\n", "注:此功能将成为 TensorFlow 2.15 版本的一部分。它目前在 Nightly 版本中提供,您可以使用 `pip install tf-nightly` 进行安装。\n", "\n", "由于 protobuf 实现的限制,proto 的大小不能超过 2GB。在尝试保存非常大的模型时,这可能会导致以下错误:\n", "\n", "```\n", "ValueError: Message tensorflow.SavedModel exceeds maximum protobuf size of 2GB: ...\n", "```\n", "\n", "```\n", "google.protobuf.message.DecodeError: Error parsing message as the message exceeded the protobuf limit with type 'tensorflow.GraphDef'\n", "```\n", "\n", "如果您希望保存超过 2GB 限制的模型,则需要使用新的 proto 分割选项进行保存:\n", "\n", "```python\n", "tf.saved_model.save(\n", " ...,\n", " options=tf.saved_model.SaveOptions(experimental_image_format=True)\n", ")\n", "```\n", "\n", "更多信息,请参阅 [Proto 分割器/合并器库指南](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/proto_splitter/in-depth-guide.md)。" ] }, { "cell_type": "markdown", "metadata": { "id": "Co6fDbzw_UnD" }, "source": [ "## 在 C++ 中加载 SavedModel\n", "\n", "SavedModel [加载器](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/cc/saved_model/loader.h)的 C++ 版本提供了一个从路径中加载 SavedModel 的 API,同时允许使用 SessionOption 和 RunOption。您必须指定与计算图相关联的标记才能加载模型。加载的 SavedModel 版本称为 SavedModelBundle,其中包含 MetaGraphDef 以及加载该版本所处的会话。\n", "\n", "```C++\n", "const string export_dir = ...\n", "SavedModelBundle bundle;\n", "...\n", "LoadSavedModel(session_options, run_options, export_dir, {kSavedModelTagTrain},\n", " &bundle);\n", "```" ] }, { "cell_type": "markdown", "metadata": { "id": "b33KuyEuAO3Z" }, "source": [ "\n", "\n", "## SavedModel 命令行接口详细信息\n", "\n", "您可以使用 SavedModel 命令行接口 (CLI) 检查和执行 SavedModel。例如,您可以使用 CLI 来检查模型的 `SignatureDef`。通过 CLI,您可以快速确认与模型相符的输入张量的 dtype 和形状。此外,如果要测试模型,您可以传入各种格式的样本输入(例如,Python 表达式),然后获取输出,使用 CLI 执行健全性检查。\n", "\n", "### 安装 SavedModel CLI\n", "\n", "一般来说,通过以下两种方式都可以安装 TensorFlow:\n", "\n", "- 安装预构建的 TensorFlow 二进制文件。\n", "- 从源代码构建 TensorFlow。\n", "\n", "如果您是通过预构建的 TensorFlow 二进制文件安装的 TensorFlow,则 SavedModel CLI 已安装到您的系统上,路径为 `bin/saved_model_cli`。\n", "\n", "如果是从源代码构建的 TensorFlow,则还必须运行以下附加命令才能构建 `saved_model_cli`:\n", "\n", "```\n", "$ bazel build //tensorflow/python/tools:saved_model_cli\n", "```\n", "\n", "### 命令概述\n", "\n", "SavedModel CLI 支持在 SavedModel 上使用以下两个命令:\n", "\n", "- `show`:用于显示 SavedModel 中可用的计算。\n", "- `run`:用于从 SavedModel 运行计算。\n", "\n", "### `show` 命令\n", "\n", "SavedModel 包含一个或多个模型变体(从技术上说,为 `v1.MetaGraphDef`),这些变体通过 tag-set 进行标识。要应用模型,您可能想知道每个模型变体中使用的具体是哪一种 `SignatureDef` ,以及它们的输入和输出是什么。那么,利用 `show` 命令,您就可以按照层级顺序检查 SavedModel 的内容。具体语法如下:\n", "\n", "```\n", "usage: saved_model_cli show [-h] --dir DIR [--all]\n", "[--tag_set TAG_SET] [--signature_def SIGNATURE_DEF_KEY]\n", "```\n", "\n", "例如,以下命令会显示 SavedModel 中的所有可用 tag-set:\n", "\n", "```\n", "$ saved_model_cli show --dir /tmp/saved_model_dir\n", "The given SavedModel contains the following tag-sets:\n", "serve\n", "serve, gpu\n", "```\n", "\n", "以下命令会显示 tag-set 的所有可用 `SignatureDef` 键:\n", "\n", "```\n", "$ saved_model_cli show --dir /tmp/saved_model_dir --tag_set serve The given SavedModel `MetaGraphDef` contains `SignatureDefs` with the following keys: SignatureDef key: \"classify_x2_to_y3\" SignatureDef key: \"classify_x_to_y\" SignatureDef key: \"regress_x2_to_y3\" SignatureDef key: \"regress_x_to_y\" SignatureDef key: \"regress_x_to_y2\" SignatureDef key: \"serving_default\"\n", "```\n", "\n", "如果 tag-set 中有*多个*标记,则必须指定所有标记(标记之间用逗号分隔)。例如:\n", "\n", "
$ saved_model_cli show --dir /tmp/saved_model_dir --tag_set serve,gpu\n", "\n", "要显示特定 `SignatureDef` 的所有输入和输出 TensorInfo,请将 `SignatureDef` 键传递给 `signature_def` 选项。如果您想知道输入张量的张量键值、dtype 和形状,以便随后执行计算图,这会非常有用。例如:\n", "\n", "```\n", "$ saved_model_cli show --dir \\ /tmp/saved_model_dir --tag_set serve --signature_def serving_default The given SavedModel SignatureDef contains the following input(s): inputs['x'] tensor_info: dtype: DT_FLOAT shape: (-1, 1) name: x:0 The given SavedModel SignatureDef contains the following output(s): outputs['y'] tensor_info: dtype: DT_FLOAT shape: (-1, 1) name: y:0 Method name is: tensorflow/serving/predict\n", "```\n", "\n", "要显示 SavedModel 中的所有可用信息,请使用 `--all` 选项。例如:\n", "\n", "
$ saved_model_cli show --dir /tmp/saved_model_dir --all<br>MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:<br><br>signature_def['classify_x2_to_y3']:<br> The given SavedModel SignatureDef contains the following input(s):<br> inputs['inputs'] tensor_info:<br> dtype: DT_FLOAT<br> shape: (-1, 1)<br> name: x2:0<br> The given SavedModel SignatureDef contains the following output(s):<br> outputs['scores'] tensor_info:<br> dtype: DT_FLOAT<br> shape: (-1, 1)<br> name: y3:0<br> Method name is: tensorflow/serving/classify<br><br>...<br><br>signature_def['serving_default']:<br> The given SavedModel SignatureDef contains the following input(s):<br> inputs['x'] tensor_info:<br> dtype: DT_FLOAT<br> shape: (-1, 1)<br> name: x:0<br> The given SavedModel SignatureDef contains the following output(s):<br> outputs['y'] tensor_info:<br> dtype: DT_FLOAT<br> shape: (-1, 1)<br> name: y:0<br> Method name is: tensorflow/serving/predict\n", "\n", "### `run` 命令\n", "\n", "调用 `run` 命令即可运行计算图计算,传递输入,然后显示输出,还可以选择保存。具体语法如下:\n", "\n", "```\n", "usage: saved_model_cli run [-h] --dir DIR --tag_set TAG_SET --signature_def\n", " SIGNATURE_DEF_KEY [--inputs INPUTS]\n", " [--input_exprs INPUT_EXPRS]\n", " [--input_examples INPUT_EXAMPLES] [--outdir OUTDIR]\n", " [--overwrite] [--tf_debug]\n", "```\n", "\n", "要将输入传递给模型,`run` 命令提供了以下三种方式:\n", "\n", "- `--inputs` 选项:可传递文件中的 NumPy ndarray。\n", "- `--input_exprs` 选项:可传递 Python 表达式。\n", "- `--input_examples` 选项:可传递 `tf.train.Example`。\n", "\n", "#### `--inputs`\n", "\n", "要传递文件中的输入数据,请指定 `--inputs` 选项,一般格式如下:\n", "\n", "```bsh\n", "--inputs