Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve code suggestion prompt #1252

Merged
merged 6 commits into from
Sep 25, 2024
Merged

improve code suggestion prompt #1252

merged 6 commits into from
Sep 25, 2024

Conversation

mrT23
Copy link
Collaborator

@mrT23 mrT23 commented Sep 25, 2024

PR Type

Enhancement


Description

  • Refactored PR code suggestions tool to use a single prompt for all models, removing Claude-specific handling
  • Enhanced patch generation by removing trailing whitespace in the diff output
  • Updated PR code suggestions prompt with more detailed instructions and improved guidelines for generating suggestions
  • Refined Pydantic model field descriptions for better clarity and accuracy
  • Expanded and clarified instructions for suggestion evaluation in the reflection prompt
  • Added 'is_ai_metadata' flag to variables in self-reflection function for consistency

Changes walkthrough 📝

Relevant files
Enhancement
pr_code_suggestions.py
Refactor code suggestion prompt and improve patch generation

pr_agent/tools/pr_code_suggestions.py

  • Removed Claude-specific prompt handling
  • Updated patch generation to remove trailing whitespace
  • Added 'is_ai_metadata' to variables in self-reflection function
  • +4/-7     
    pr_code_suggestions_prompts.toml
    Enhance PR code suggestions prompt and model definitions 

    pr_agent/settings/pr_code_suggestions_prompts.toml

  • Updated system prompt for PR code suggestions
  • Improved instructions for generating code suggestions
  • Enhanced descriptions in Pydantic model fields
  • +26/-135
    pr_code_suggestions_reflect_prompts.toml
    Improve code suggestion reflection prompt                               

    pr_agent/settings/pr_code_suggestions_reflect_prompts.toml

  • Expanded and clarified instructions for suggestion evaluation
  • Updated format description for PR code diff presentation
  • Refined Pydantic model field descriptions
  • +52/-28 

    💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

    @codiumai-pr-agent-pro codiumai-pr-agent-pro bot added the enhancement New feature or request label Sep 25, 2024
    @Codium-ai Codium-ai deleted a comment from codiumai-pr-agent-pro bot Sep 25, 2024
    @mrT23
    Copy link
    Collaborator Author

    mrT23 commented Sep 25, 2024

    PR Code Suggestions ✨

    Latest suggestions up to 4713ae7

    CategorySuggestion                                                                                                                                    Score
    Enhancement
    Use dictionary unpacking for cleaner variable assignment

    Consider using a dictionary unpacking operator () to add the new key-value pairs
    to the variables dictionary. This can make the code more concise and easier to
    maintain.**

    pr_agent/tools/pr_code_suggestions.py [739-743]

    -variables = {'suggestion_list': suggestion_list,
    -             'suggestion_str': suggestion_str,
    -             "diff": patches_diff,
    -             'num_code_suggestions': len(suggestion_list),
    -             "is_ai_metadata": get_settings().get("config.enable_ai_metadata", False)}
    +variables = {
    +    'suggestion_list': suggestion_list,
    +    'suggestion_str': suggestion_str,
    +    "diff": patches_diff,
    +    **{
    +        'num_code_suggestions': len(suggestion_list),
    +        "is_ai_metadata": get_settings().get("config.enable_ai_metadata", False)
    +    }
    +}
    Suggestion importance[1-10]: 6

    Why: The suggestion offers a valid enhancement to improve code readability and maintainability. It correctly identifies the new variables added in the PR and proposes a more concise way to include them.

    6
    Best practice
    Use f-string for patch addition to improve readability

    Consider using an f-string instead of string concatenation for better readability
    and performance when adding the patch to example_code.

    pr_agent/tools/pr_code_suggestions.py [686]

    +example_code += f"```diff\n{patch.rstrip()}\n```\n"
     
    -
    Suggestion importance[1-10]: 3

    Why: The suggestion correctly identifies the use of an f-string, which is already implemented in the PR. While it's a valid best practice, it doesn't add value as the change has already been made.

    3

    Previous suggestions

    Suggestions up to commit 6f14f9c
    CategorySuggestion                                                                                                                                    Score
    Maintainability
    Break down the long system prompt into smaller sections for improved readability and maintainability

    Consider breaking down the long system prompt into smaller, more manageable sections
    using TOML's multi-line string syntax. This can improve readability and make it
    easier to maintain the prompt content.

    pr_agent/settings/pr_code_suggestions_prompts.toml [2-10]

    -system="""You are PR-Reviewer, an AI specializing in Pull Request (PR) code analysis and suggestions.
    +system = """
    +You are PR-Reviewer, an AI specializing in Pull Request (PR) code analysis and suggestions.
     Your task is to examine the provided code diff, focusing on new lines (prefixed with '+'), and offer concise, actionable suggestions to fix possible bugs and problems, and enhance code quality, readability, and performance.
     
    +The PR code diff will be presented in the following structured format:
    +"""
     
    -The PR code diff will be presented in the following structured format:
    +diff_format = """
     ======
     ## File: 'src/file1.py'
     {%- if is_ai_metadata %}
     ### AI-generated changes summary:
    +"""
     
    +system = system + diff_format
    +
    Suggestion importance[1-10]: 6

    Why: This suggestion provides a meaningful improvement to code maintainability and readability. Breaking down the long prompt into smaller sections can indeed make it easier to manage and update in the future.

    6
    Enhancement
    Use dictionary unpacking for cleaner variable assignment in the variables dictionary

    Consider using a dictionary unpacking operation to merge the existing variables with
    the new ones in the variables dictionary. This can make the code more concise and
    easier to maintain.

    pr_agent/tools/pr_code_suggestions.py [739-743]

    -variables = {'suggestion_list': suggestion_list,
    -             'suggestion_str': suggestion_str,
    -             "diff": patches_diff,
    -             'num_code_suggestions': len(suggestion_list),
    -             "is_ai_metadata": get_settings().get("config.enable_ai_metadata", False)}
    +variables = {
    +    'suggestion_list': suggestion_list,
    +    'suggestion_str': suggestion_str,
    +    "diff": patches_diff,
    +    **{
    +        'num_code_suggestions': len(suggestion_list),
    +        "is_ai_metadata": get_settings().get("config.enable_ai_metadata", False)
    +    }
    +}
    Suggestion importance[1-10]: 4

    Why: The suggestion offers a minor improvement in code organization and readability. However, the benefit is relatively small, and the current implementation is already clear and functional.

    4
    Best practice
    Use f-string for more efficient string formatting in example code assignment

    Consider using an f-string for string formatting in the example_code assignment.
    This can improve readability and performance compared to the current string
    concatenation approach.

    pr_agent/tools/pr_code_suggestions.py [686]

    -example_code += f"```diff\n{patch.rstrip()}\n```\n"
    +example_code = f"```diff\n{patch.rstrip()}\n```\n"
    Suggestion importance[1-10]: 2

    Why: The suggestion is technically correct but offers minimal improvement. The existing code already uses an f-string, and changing from '+=' to '=' doesn't significantly impact performance or readability.

    2
    Suggestions up to commit 6f14f9c
    CategorySuggestion                                                                                                                                    Score
    Best practice
    Use f-string for concatenating patch to example_code

    Consider using an f-string for better readability and performance when concatenating
    the patch to example_code.

    pr_agent/tools/pr_code_suggestions.py [686]

    +example_code += f"```diff\n{patch.rstrip()}\n```\n"
     
    -
    • Apply this suggestion
    Suggestion importance[1-10]: 7

    Why: This suggestion improves code readability and potentially performance, making it a worthwhile enhancement.

    7
    Use a more descriptive name for the variables dictionary

    Consider using a more descriptive name for the variables dictionary to better
    reflect its purpose, such as template_variables or prompt_context.

    pr_agent/settings/pr_code_suggestions_prompts.toml [739-743]

    -variables = {'suggestion_list': suggestion_list,
    -             'suggestion_str': suggestion_str,
    -             "diff": patches_diff,
    -             'num_code_suggestions': len(suggestion_list),
    -             "is_ai_metadata": get_settings().get("config.enable_ai_metadata", False)}
    +template_variables = {
    +    'suggestion_list': suggestion_list,
    +    'suggestion_str': suggestion_str,
    +    "diff": patches_diff,
    +    'num_code_suggestions': len(suggestion_list),
    +    "is_ai_metadata": get_settings().get("config.enable_ai_metadata", False)
    +}
     
    • Apply this suggestion
    Suggestion importance[1-10]: 6

    Why: This suggestion enhances code readability and maintainability, but it's not addressing a critical issue or bug.

    6
    Use a more specific term instead of "AI" in the variable name

    Consider using a more specific term instead of "AI" in the variable name
    is_ai_metadata to better reflect its purpose, such as is_generated_metadata or
    is_auto_generated_metadata.

    pr_agent/settings/pr_code_suggestions_prompts.toml [76]

    -"is_ai_metadata": get_settings().get("config.enable_ai_metadata", False),
    +"is_generated_metadata": get_settings().get("config.enable_generated_metadata", False),
     
    • Apply this suggestion
    Suggestion importance[1-10]: 5

    Why: While the suggestion improves clarity, it's a minor change that doesn't significantly impact functionality or maintainability.

    5
    Enhancement
    Use rstrip() instead of strip('\n') to preserve leading whitespace

    Consider using the rstrip() method instead of strip('\n') to remove only trailing
    newlines while preserving leading whitespace.

    pr_agent/tools/pr_code_suggestions.py [683]

    -patch = "\n".join(patch_orig.splitlines()[5:]).strip('\n')
    +patch = "\n".join(patch_orig.splitlines()[5:]).rstrip()
     
    • Apply this suggestion
    Suggestion importance[1-10]: 6

    Why: The suggestion is valid and improves code clarity, but it's a minor enhancement with limited impact on functionality.

    6

    @mrT23 mrT23 merged commit 511c5a3 into main Sep 25, 2024
    2 checks passed
    @mrT23 mrT23 deleted the tr/prompts_refactor branch September 25, 2024 18:23
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    enhancement New feature or request
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    2 participants