update vdb cookbook

baidubce · Mar 19, 2024 · a24ece8 · a24ece8
1 parent 520ea00
commit a24ece8
Showing 1 changed file with 164 additions and 0 deletions.
diff --git a/cookbooks/vdb_retriever.ipynb b/cookbooks/vdb_retriever.ipynb
@@ -0,0 +1,164 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# VectorDB\n",
+    "\n",
+    "## 概述\n",
+    "\n",
+    "VectorDB产品基于百度自研的“莫愁”向量数据库内核构建，充分利用了百度内部在分布式存储&数据库领域的成熟经验，实现了高可靠、高可用、强扩展和大规模的能力。详细介绍可以参考文档 [VectorDB](https://cloud.baidu.com/doc/VDB/s/Llrsoaz3l)\n",
+    "\n",
+    "## 应用场景\n",
+    "\n",
+    "### 场景一：信息相似度检索\n",
+    "\n",
+    "在当今的信息时代，快速而准确地检索文本和图片数据成为了一个重要挑战。我们的向量数据库针对这一挑战，提供了一个高效、安全且智能的解决方案。\n",
+    "\n",
+    "### 场景二：大模型问答记忆\n",
+    "\n",
+    "在利用大型语言模型进行问答交互时，保持模型与用户之间的会话连贯性是提高回答质量和相关性的关键。我们的解决方案通过实时存储和检索会话数据，有效地降低了幻觉情况的发生，提升了问答的准确性和用户体验。\n",
+    "\n",
+    "\n",
+    "### 场景三：大模型私域知识库\n",
+    "\n",
+    "在私有云环境中，构建专属的私域知识库是提升业务决策效率和精确性的关键。我们的向量数据库解决方案充分利用大模型的推理能力，针对私域数据提供定制化、高效的知识管理和检索服务。"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 开发上手\n",
+    "\n",
+    "### 第一步：环境准备，安装依赖"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install appbuilder-sdk"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 第二步：平台注册，设置Token\n",
+    "\n",
+    "- 2.1、注册AppBuilder平台账户，并申请Token，可参考文档：[AppBuilder官网创建密钥](https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1%E3%80%81%E5%88%9B%E5%BB%BA%E5%AF%86%E9%92%A5)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# AppBuilder平台注册后获取密钥\n",
+    "import os\n",
+    "import appbuilder\n",
+    "\n",
+    "# 请前往千帆AppBuilder官网创建密钥，流程详见：https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1%E3%80%81%E5%88%9B%E5%BB%BA%E5%AF%86%E9%92%A5\n",
+    "os.environ[\"APPBUILDER_TOKEN\"] = '...'"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- 2.2、同时您需要申请并开通VectorDB，并创建实例，具体操作请参考[VectorDB文档](https://cloud.baidu.com/doc/VDB/s/hlrsoazuf)\n",
+    "\n",
+    "\n",
+    "### 第三步：创建并运行VectorDB，初始化构建索引\n",
+    "\n",
+    "补充说明：\n",
+    "- `you_vdb_instance_id` 为VectorDB 实例ID，请替换为您的实例ID，在VectorDB控制台界面上可以查看\n",
+    "- `your_api_key` 为您在VectorDB上申请的账户密钥，请替换为您自己的root账户密钥，在VectorDB控制台界面上可以查看"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "embedding = appbuilder.Embedding()\n",
+    "segments = appbuilder.Message([\"文心一言大模型\", \"百度在线科技有限公司\"])\n",
+    "# 初始化构建索引\n",
+    "vector_index = appbuilder.BaiduVDBVectorStoreIndex.from_params(\n",
+    "    instance_id=\"your_instance_id\",\n",
+    "    api_key=\"your_api_key\",\n",
+    "    drop_exists=True,\n",
+    ")\n",
+    "vector_index.add_segments(segments)"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 第四步：构建完毕索引后，可以通过retriever进行检索"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "query = appbuilder.Message(\"文心一言\")\n",
+    "retriever = vector_index.as_retriever()\n",
+    "res = retriever(query)\n",
+    "print(res)\n",
+    "\n",
+    "# Message(name=msg, content=[{'text': '文心一言大模型', 'meta': '', 'score': 1.0}], mtype=list)"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "以上，便完成了VDB作为检索工具参与Appbuilder运行的流程使用，更多的VDB使用方式，请参考[VDB官方文档](https://cloud.baidu.com/doc/VDB/index.html) 与 [AppBuilder-retriever-baidu_vdb 文档](../appbuilder/core/components/retriever/baidu_vdb/README.md)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.6"
+  },
+  "orig_nbformat": 4,
+  "vscode": {
+   "interpreter": {
+    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}