Version 0.5.1

Version 0.5.1 (2024-11-21)

OpenSPG released version v0.5.1 on November 21, 2024. This version focuses on addressing user feedback and introduces a series of new features and user experience optimizations.

---

🌟 New Features

1. Support for Word Documents
   • Users can now directly upload .doc or .docx files to streamline the knowledge base construction process.

2. New Project Deletion API
   • Quickly clear and delete projects and related data through an API, compatible with the latest Neo4j image version.
3. Model Call Concurrency Setting
   • Added the builder.model.execute.num parameter, with a default concurrency of 5, to improve efficiency in large-scale knowledge base construction.

4. Improved Logging
   • Added a startup success marker in the logs to help users quickly verify if the service is running correctly.

---

⚙️ User Experience Optimizations

1. Neo4j Memory Overflow Issues
   • Addressed memory overflow problems in Neo4j during large-scale data processing, ensuring stable operation for extensive datasets.
2. Concurrent Neo4j Query Execution Issues
   • Optimized execution strategies to resolve Graph Data Science (GDS) library conflicts or failures in high-concurrency scenarios.
3. Schema Preview Prefix Issue
   • Fixed issues where extracted schema preview entities lacked necessary prefixes, ensuring consistency between extracted entities and predefined schemas.
4. Default Neo4j Password for Project Creation/Modification
   • Automatically fills a secure default password if none is specified during project creation or modification, simplifying the configuration process.
5. Frontend Bug Fixes
   • Resolved issues with JS dependencies relying on external addresses and embedded all frontend files into the image. Improved the knowledge base management interface for a smoother user experience.
6. Empty Node/Edge Type in Neo4j Writes
   • Enhanced writing logic to handle empty node or edge types during knowledge graph construction, preventing errors or data loss in such scenarios.

Version 0.5.1 (2024-11-21)

OpenSPG 在 2024 年 11 月 21 日发布了 v0.5.1 版本。此版本重点解决了用户反馈的问题，并带来了一系列新功能和用户体验的优化。

🌟 新增功能

1. 支持 word 文档的构建
   • 用户现可通过知识库管理页面直接上传 .doc 或 .docx 后缀的文件，进行知识库的构建流程。这一更新使得知识内容的导入更加便捷，提高效率。

2. 提供项目删除接口
   • 为了帮助用户更高效地管理项目，我们新增了一个项目删除接口。用户可以通过访问 http://127.0.0.1:8887/project/api/delete?projectId=xx 完成项目的快速清空与删除操作。该接口会同步清理项目下的所有schema、知识库任务、知识库问答任务以及关联的 Neo4j 数据库。
     Tips：使用此功能前，需确保已将 openspg-neo4j 镜像更新至最新版本
3. 支持模型调用并发度设置
   • 在大规模知识库构建过程中，为了提高构建效率，我们引入了模型调用的并发控制机制。用户可以通过设置 builder.model.execute.num 参数来调整并发数量，默认值设定为5。这有助于避免因模型服务性能瓶颈而导致的任务失败或系统卡顿。

4. 日志中添加启动成功标识
   • 为了让用户能够更直观地判断 OpenSPG 服务是否启动成功，我们在日志输出中加入了明确的启动成功标识。openspg-server 成功启动后，会输出这一标识。

⚙️ 用户体验优化

1. 解决大规模数据构建下 Neo4j 调用内存超限问题
   • 针对在处理大规模数据集时出现的 Neo4j 内存溢出问题，我们进行了深入分析并实施了有效的解决方案。现在，面对大规模数据集Neo4j 能保持稳定运行，有效防止了因内存不足而导致的服务中断。
2. 解决多并发下执行 Neo4j 查询导致的 GDS 加载问题
   • 在多并发场景下执行 Neo4j 查询时，图数据科学 (GDS) 库的加载会出现冲突或失败的情况。为此，我们优化了查询执行策略，确保了在高并发环境下的查询性能和稳定性。
3. 解决抽取结果 Schema 预览实体无前缀问题
   • 在之前版本中，部分用户反馈在查看抽取结果的 Schema 预览时，实体名称缺少必要的前缀信息导致抽取的实体和预定义的Schema不一致。此次更新修正了这一问题，保证了所有实体名称的完整性和准确性。
4. 创建修改项目时 Neo4j 无密码时填充默认值
   • 当用户在创建或修改项目时，如果未指定 Neo4j 密码，系统将自动填充一个安全的默认值，从而简化了配置流程，减少了用户的输入负担。
5. 前端 bugfix
   • 修复了JS依赖外部地址问题，已将前端文件全部内置到镜像内；同时针对知识库管理页面进行了多项改进，以提供更加流畅的操作体验。
6. 解决点边类型为空导致的 Neo4j 写入失败问题
   • 对于在构建知识图谱时可能出现的节点或关系类型为空的情况，我们优化了写入逻辑，确保即便在这些特殊情况下也能顺利完成数据的写入操作，避免了因类型缺失而引发的数据丢失或错误。

Version 0.5

Version 0.5 (2024-10-25)

retrieval Augmentation Generation (RAG) technology promotes the integration of domain applications with large models. However, RAG has problems such as a large gap between vector similarity and knowledge reasoning correlation, and insensitivity to knowledge logic (such as numerical values, time relationships, expert rules, etc.), which hinder the implementation of professional knowledge services. On October 25, OpenSPG released version V0.5, officially releasing the professional domain knowledge Service Framework for knowledge enhancement generation (KAG) .

Highlights of the Release Version:

1. KAG: Knowledge Augmented Generation

KAG aims to make full use of the advantages of Knowledge Graph and vector retrieval, and bi-directionally enhance large language models and knowledge graphs through four aspects to solve RAG challenges
(1) LLM-friendly semantic knowledge management
(2) Mutual indexing between the knowledge map and the original snippet.
(3) Logical symbol-guided hybrid inference engine
(4) Knowledge alignment based on semantic reasoning
KAG is significantly better than NaiveRAG, HippoRAG and other methods in multi-hop question and answer tasks. The F1 score on hotpotQA is relatively improved by 19.6, and the F1 score on 2wiki is relatively improved by 33.5

2. Knowledge base management

OpenSPG also provides a user-friendly product interface for KAG, allowing users to upload and manage documents, preview extraction results, and quiz through the visual interface after local deployment. In the knowledge question and answer session, the system not only shows the final answer, but also presents the reasoning process, thus enhancing the transparency and interpretability of the whole question and answer process. Through this product interface, users can use KAG more intuitively and easily

3. Continuous Optimization and Bug Fixes

• feat(schema): support maintenance of simplified DSL in #335
• feat(reasoner): support thinker in knext in #344
• feat(reasoner): support ProntoQA and ProofWriter. in #352
• feat(reasoner): thinker support deduction expression in #369
• feat(openspg): support kag in #372
• feat(reasoner): add udf split_part in #378
• fix(reasoner): support triple in thinker context in #341
• fix(reasoner): bugfix in graph store. in #346
• fix(reasoner): fix pattern schema extra in #351
• fix(knext): add remote client addr in #376
• fix(knext): reasoner command add default cfg config in #377

Version 0.5 (2024-10-25)

检索增强生成（RAG）技术推动了领域应用与大模型结合。然而，RAG 存在着向量相似度与知识推理相关性差距大、对知识逻辑（如数值、时间关系、专家规则等）不敏感等问题，这些都阻碍了专业知识服务的落地。10 月 25 日，OpenSPG 发布 V0.5 版本，正式发布了知识增强生成（KAG）的专业领域知识服务框架

版本亮点

1. KAG 专业领域知识服务框架

KAG 旨在充分利用知识图谱和向量检索的优势，并通过四个方面双向增强大型语言模型和知识图谱，以解决 RAG 挑战
(1) 对 LLM 友好的语义化知识管理
(2) 知识图谱与原文片段之间的互索引
(3) 逻辑符号引导的混合推理引擎
(4) 基于语义推理的知识对齐
KAG 在多跳问答任务中显著优于 NaiveRAG、HippoRAG 等方法，在 hotpotQA 上的 F1 分数相对提高了 19.6%，在 2wiki 上的 F1 分数相对提高了33.5%

2. 知识库管理

OpenSPG针对KAG 还提供了一个用户友好的产品界面，支持用户在本地部署后，通过可视化界面进行文档上传和管理、预览抽取结果、以及知识问答。在知识问答环节，系统不仅展示最终答案，还会呈现推理过程，从而增强了整个问答流程的透明度和可解释性。通过这个产品界面，用户能够更直观、更轻松地上手使用 KAG

3. 持续优化与问题修复

• feat(schema): support maintenance of simplified DSL in #335
• feat(reasoner): support thinker in knext in #344
• feat(reasoner): support ProntoQA and ProofWriter. in #352
• feat(reasoner): thinker support deduction expression in #369
• feat(openspg): support kag in #372
• feat(reasoner): add udf split_part in #378
• fix(reasoner): support triple in thinker context in #341
• fix(reasoner): bugfix in graph store. in #346
• fix(reasoner): fix pattern schema extra in #351
• fix(knext): add remote client addr in #376
• fix(knext): reasoner command add default cfg config in #377

Version 0.0.3

Version 0.0.3 (2024-08-15)

Knowledge graphs have become a crucial bridge between LLMs and AI Agents. the OpenSPG project officially released its first stable version. This release not only inherits all the powerful features of the previous beta version but also brings comprehensive improvements in stability, compatibility, and user experience, aiming to provide a more mature and reliable knowledge construction solution for enterprises and developers.

Highlights of the Release Version:

1. Unified Knowledge Extraction with LLMs

The first stable version of OpenSPG inherits and optimizes the unified knowledge extraction feature from the beta version.
This feature is based on OneKE, a Chinese-English bilingual knowledge extraction grand model jointly released by Ant Group and Zhejiang University. Through techniques like hard negative sampling and schema-rotation-based instruction construction, it enhances the generalization capability of structured information extraction.

2. Product Visualization Interface

The release further strengthens the visualization interface, offering users a more intuitive data exploration and analysis experience. You can now visually inspect modeling results on the page and conduct interactive analysis and reasoning queries.

3. Continuous Optimization and Bug Fixes

Bugfix 1: Initialization exception in knext builder (issue #236 #246)
Bugfix 2: Fixed error in reasoner transform ListOpExpr (#328)
Bugfix 3: Front-end canvas display issue in analysis reasoning (Issue #269)

The release version of OpenSPG is applicable to multiple domains, including but not limited to financial risk control, healthcare, enterprise knowledge management, and intelligent customer service. By constructing high-quality knowledge graphs, it empowers various application scenarios such as decision analysis, recommendation systems, and natural language understanding.

Version 0.0.3 (2024-08-15)

知识图谱已成为连接大模型与智能体的重要桥梁。OpenSPG 项目正式发布首个 Release 版本。这一版本承袭了此前 beta 版本的所有强大功能，在稳定性、兼容性和用户体验方面进行了全面提升，旨在为企业和开发者提供更加成熟可靠的知识构建解决方案。

版本亮点

1. 大模型统一知识抽取

OpenSPG 首个 Release 版本继承并优化了 beta 版本的大模型统一知识抽取功能。这一功能基于蚂蚁集团与浙江大学联合发布的 OneKE 大模型，专注于 Schema 可泛化的信息抽取，通过难负采样和 Schema 轮训式指令构造技术，提升了结构化信息抽取的泛化能力。

2. 产品可视化界面

Release 版本进一步强化了可视化界面，为用户提供了更加直观的数据探索和分析体验。用户现在可以在页面上直观地查看建模结果，并进行交互式分析推理查询。

3. 持续优化与问题修复

Bugfix 1：knext builder初始化异常 (issue #236 #246)
Bugfix 2：修复 reasoner transform ListOpExpr 报错 (#328)
Bugfix 3：分析推理前端画布展示问题 (Issue #269)

OpenSPG 的 Release 版本适用于多个领域，包括但不限于金融风控、医疗健康、企业知识管理、智能客服等，通过构建高质量的知识图谱，赋能决策分析、推荐系统、自然语言理解等多种应用场景。