From da398ce56f6a9562edd0d1b0ca9baa8fd5517f89 Mon Sep 17 00:00:00 2001 From: mrT23 Date: Mon, 16 Sep 2024 09:21:52 +0300 Subject: [PATCH] TLDR --- docs/docs/core-abilities/dynamic_context.md | 5 +++++ docs/docs/core-abilities/metadata.md | 8 ++++---- docs/docs/core-abilities/self_reflection.md | 17 ++++++++++++----- 3 files changed, 21 insertions(+), 9 deletions(-) diff --git a/docs/docs/core-abilities/dynamic_context.md b/docs/docs/core-abilities/dynamic_context.md index b35635544..7f07d222a 100644 --- a/docs/docs/core-abilities/dynamic_context.md +++ b/docs/docs/core-abilities/dynamic_context.md @@ -1,3 +1,8 @@ +## TL;DR + +PR-Agent uses an **asymmetric and dynamic context strategy** to improve AI analysis of code changes in pull requests. +It provides more context before changes than after, and dynamically adjusts the context based on code structure (e.g., enclosing functions or classes). +This approach balances providing sufficient context for accurate analysis, while avoiding needle-in-the-haystack information overload that could degrade AI performance or exceed token limits. ## Introduction diff --git a/docs/docs/core-abilities/metadata.md b/docs/docs/core-abilities/metadata.md index 97b448a7f..2ad5609a1 100644 --- a/docs/docs/core-abilities/metadata.md +++ b/docs/docs/core-abilities/metadata.md @@ -15,8 +15,8 @@ PR-Agent initially retrieves for each PR the following data: By default, the first command that PR-Agent executes is [`describe`](https://pr-agent-docs.codium.ai/tools/describe/), which generates three types of outputs: - PR Type (e.g. bug fix, feature, refactor, etc) -- PR Description - a bullet points summary of the PR -- Changes walkthrough - going file-by-file, PR-Agent generate a one-line summary and longer bullet points summary of the changes in the file +- PR Description - a bullet point summary of the PR +- Changes walkthrough - for each modified file, provide a one-line summary followed by a detailed bullet point list of the changes. These AI-generated outputs are now considered as part of the PR metadata, and can be used in subsequent commands like `review` and `improve`. This effectively enables multi-stage chain-of-thought analysis, without doing any additional API calls which will cost time and money. @@ -25,7 +25,7 @@ For example, when generating code suggestions for different files, PR-Agent can ``` ## File: 'src/file1.py' -### AI-generated changes summary: +### AI-generated file summary: - edited function `func1` that does X - Removed function `func2` that was not used - .... @@ -49,7 +49,7 @@ __old hunk__ ... ``` -(3) The entire PR files that where retrieved are also used to expand and enhance the PR context (see [Dynamic Context](https://pr-agent-docs.codium.ai/core-abilities/dynamic-context/)). +(3) The entire PR files that were retrieved are also used to expand and enhance the PR context (see [Dynamic Context](https://pr-agent-docs.codium.ai/core-abilities/dynamic-context/)). (4) All the metadata described above represents several level of cumulative analysis - ranging from hunk level, to file level, to PR level, to organization level. diff --git a/docs/docs/core-abilities/self_reflection.md b/docs/docs/core-abilities/self_reflection.md index 1076da259..f58a730d8 100644 --- a/docs/docs/core-abilities/self_reflection.md +++ b/docs/docs/core-abilities/self_reflection.md @@ -1,5 +1,11 @@ +## TL;DR + +PR-Agent implements a **self-reflection** process where the AI model reflects, scores, and re-ranks its own suggestions, eliminating irrelevant or incorrect ones. +This approach improves the quality and relevance of suggestions, saving users time and enhancing their experience. +Configuration options allow users to set a score threshold for further filtering out suggestions. + +## Introduction - Efficient Review with Hierarchical Presentation -### Introduction - fast review with hierarchical structure Given that not all generated code suggestions will be relevant, it is crucial to enable users to review them in a fast and efficient way, allowing quick identification and filtering of non-applicable ones. @@ -9,9 +15,10 @@ To achieve this goal, PR-Agent offers a dedicated hierarchical structure when pr - Each suggestion is first described by a one-line summary, which can be expanded to a full description by clicking on a collapsible. - Upon expanding a suggestion, the user receives a more comprehensive description, and a code snippet demonstrating the recommendation. -This hierarchical structure is designed to facilitate rapid review of each suggestion, with users spending an average of ~5-10 seconds per item. +!!! note "Fast Review" + This hierarchical structure is designed to facilitate rapid review of each suggestion, with users spending an average of ~5-10 seconds per item. -### Self-reflection and re-ranking +## Self-reflection and Re-ranking The AI model is initially tasked with generating suggestions, and outputting them in order of importance. However, in practice we observe that models often struggle to simultaneously generate high-quality code suggestions and rank them well in a single pass. @@ -30,13 +37,13 @@ Note that presenting all generated suggestions simultaneously provides the model To conclude, the self-reflection process enables PR-Agent to prioritize suggestions based on their importance, eliminate inaccurate or irrelevant proposals, and optionally exclude suggestions that fall below a specified threshold of significance. This results in a more refined and valuable set of suggestions for the user, saving time and improving the overall experience. -### Example results +## Example Results ![self_reflection](https://codium.ai/images/pr_agent/self_reflection1.png){width=768} ![self_reflection](https://codium.ai/images/pr_agent/self_reflection2.png){width=768} -### Appendix - relevant configuration options +## Appendix - Relevant Configuration Options ``` [pr_code_suggestions] self_reflect_on_suggestions = true # Enable self-reflection on code suggestions