Enhance AI handler logging and add main PR language attribute to AI h…

[mrT23]

User description

…andler in various tools

---

Type

enhancement

---

Description

• Enhanced AI handler logging with detailed information including system/user prompts, output, finish reason, and main PR language.
• Assigned main PR language to AI handler across various PR tools for better context handling.
• Maintained CLI debugging functionality based on verbosity level.

---

Changes walkthrough

Relevant files

Enhancement

10 files litellm_ai_handler.py Enhanced AI Handler Logging and Main PR Language Logging  pr_agent/algo/ai_handlers/litellm_ai_handler.py Enhanced AI response logging with detailed debugging information including system and user prompts, output, finish reason, and main PR language. Added condition to log main PR language or default to 'unknown' if not available. Maintained functionality for CLI debugging based on verbosity level. +13/-2    pr_add_docs.py Assign Main PR Language to AI Handler in PR Docs Tool        pr_agent/tools/pr_add_docs.py Assigned main PR language to AI handler. +2/-0      pr_code_suggestions.py Assign Main PR Language to AI Handler in PR Code Suggestions Tool pr_agent/tools/pr_code_suggestions.py Assigned main PR language to AI handler. +1/-0      pr_description.py Assign Main PR Language to AI Handler in PR Description Tool pr_agent/tools/pr_description.py Assigned main PR language to AI handler. +1/-0      pr_generate_labels.py Assign Main PR Language to AI Handler in PR Label Generation Tool pr_agent/tools/pr_generate_labels.py Assigned main PR language to AI handler. +2/-1      pr_information_from_user.py Assign Main PR Language to AI Handler in PR Information Gathering Tool pr_agent/tools/pr_information_from_user.py Assigned main PR language to AI handler. +2/-0      pr_line_questions.py Determine and Assign Main PR Language to AI Handler in PR Line Questions Tool pr_agent/tools/pr_line_questions.py Determined and assigned main PR language to AI handler. +4/-1      pr_questions.py Assign Main PR Language to AI Handler in PR Questions Tool pr_agent/tools/pr_questions.py Assigned main PR language to AI handler. +2/-0      pr_reviewer.py Assign Main PR Language to AI Handler in PR Reviewer Tool pr_agent/tools/pr_reviewer.py Assigned main PR language to AI handler. +2/-0      pr_update_changelog.py Assign Main PR Language to AI Handler in PR Changelog Update Tool pr_agent/tools/pr_update_changelog.py Assigned main PR language to AI handler. +3/-0

---

✨ PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[mrT23]

/review auto_approve

[codiumai-pr-agent-pro]

Auto-approved PR

[codiumai-pr-agent-pro]

PR Description updated to latest commit (7434528)

• Copy walkthrough table to "Files Changed" Tab

[codiumai-pr-agent-pro]

PR Review

(Review updated until commit 669e076)

⏱️ Estimated effort to review [1-5]	2, because the changes are straightforward and consistent across multiple files, focusing on enhancing logging and assigning a main PR language attribute. The logic is not complex, but reviewing the correctness of the logging details and the proper assignment of the main PR language across various tools requires some attention.
🧪 Relevant tests	No
🔍 Possible issues	Logging Sensitive Data: Ensure that the enhanced logging in litellm_ai_handler.py does not inadvertently log sensitive information. Since the logging now includes system and user prompts, output, and finish reason, it's crucial to verify that none of these could contain sensitive data.
🔒 Security concerns	No
🔀 Multiple PR themes	Sub-PR theme: Enhance AI Handler Logging Relevant files: pr_agent/algo/ai_handlers/litellm_ai_handler.py
	Sub-PR theme: Assign Main PR Language to AI Handler Across Tools Relevant files: pr_agent/tools/pr_add_docs.py pr_agent/tools/pr_code_suggestions.py pr_agent/tools/pr_description.py pr_agent/tools/pr_generate_labels.py pr_agent/tools/pr_information_from_user.py pr_agent/tools/pr_line_questions.py pr_agent/tools/pr_questions.py pr_agent/tools/pr_reviewer.py pr_agent/tools/pr_update_changelog.py

---

✨ Review tool usage guide:

---

Overview:
The review tool scans the PR code changes, and generates a PR review. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.
When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

With a configuration file, use the following template:

[pr_reviewer]
some_config1=...
some_config2=...

Utilizing extra instructions The review tool can be configured with extra instructions, which can be used to guide the model to a feedback tailored to the needs of your project. Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify the relevant sub-tool, and the relevant aspects of the PR that you want to emphasize. Examples for extra instructions: [pr_reviewer] # /review # extra_instructions=""" In the 'possible issues' section, emphasize the following: - Does the code logic cover relevant edge cases? - Is the code logic clear and easy to understand? - Is the code logic efficient? ... """ Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

How to enable\disable automation When you first install PR-Agent app, the default mode for the review tool is: pr_commands = ["/review", ...] meaning the review tool will run automatically on every PR, with the default configuration. Edit this field to enable/disable the tool, or to change the used configurations

Auto-labels The review tool can auto-generate two specific types of labels for a PR: a possible security issue label, that detects possible security issues (enable_review_labels_security flag) a Review effort [1-5]: x label, where x is the estimated effort to review the PR (enable_review_labels_effort flag)

Extra sub-tools The review tool provides a collection of possible feedbacks about a PR. It is recommended to review the possible options, and choose the ones relevant for your use case. Some of the feature that are disabled by default are quite useful, and should be considered for enabling. For example: require_score_review, require_soc2_ticket, require_can_be_split_review, and more.

Auto-approve PRs By invoking: /review auto_approve The tool will automatically approve the PR, and add a comment with the approval. To ensure safety, the auto-approval feature is disabled by default. To enable auto-approval, you need to actively set in a pre-defined configuration file the following: [pr_reviewer] enable_auto_approval = true (this specific flag cannot be set with a command line argument, only in the configuration file, committed to the repository) You can also enable auto-approval only if the PR meets certain requirements, such as that the estimated_review_effort is equal or below a certain threshold, by adjusting the flag: [pr_reviewer] maximal_review_effort = 5

More PR-Agent commands To invoke the PR-Agent, add a comment using one of the following commands: /review: Request a review of your Pull Request. /describe: Update the PR title and description based on the contents of the PR. /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback. /ask <QUESTION>: Ask a question about the PR. /update_changelog: Update the changelog based on the PR's contents. /add_docs 💎: Generate docstring for new components introduced in the PR. /generate_labels 💎: Generate labels for the PR based on the PR's contents. /analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component. See the tools guide for more details. To list the possible configuration parameters, add a /config comment.

See the review usage page for a comprehensive guide on using this tool.

[codiumai-pr-agent-pro]

PR Code Suggestions

Category	Suggestions
Enhancement	Improve the readability and consistency of the logging format. Consider using a more structured logging format for the full response. Instead of directly logging the dictionary, you could use structured logging features or format the message to make it more readable and consistent with other logs. pr_agent/algo/ai_handlers/litellm_ai_handler.py [140] -get_logger().debug("Full_response", artifact=response_log) +get_logger().debug(f"Full response: system={system}, user={user}, output={resp}, finish_reason={finish_reason}, main_pr_language={self.main_pr_language if hasattr(self, 'main_pr_language') else 'unknown'}")
Best practice	Prevent unintentional alteration of the original response dictionary. To avoid potential issues with direct dictionary manipulation, consider creating a copy of the response dictionary before modifying it. This ensures that the original response is not unintentionally altered. pr_agent/algo/ai_handlers/litellm_ai_handler.py [131] -response_log = response.dict() +response_log = response.dict().copy()
	Add validation for self.main_pr_language to ensure it has a valid value. Consider adding error handling or validation for the result of get_main_pr_language to ensure that self.main_pr_language has a valid value before using it. pr_agent/tools/pr_line_questions.py [25-27] self.main_pr_language = get_main_pr_language( self.git_provider.get_languages(), self.git_provider.get_files() ) +if not self.main_pr_language: + raise ValueError("Failed to determine the main PR language.")
Maintainability	Initialize main_pr_language in the class constructor to simplify access. Instead of checking for the attribute main_pr_language with hasattr, consider initializing main_pr_language in the class constructor. This makes the code cleaner and avoids the need for conditional checks every time the attribute is accessed. pr_agent/algo/ai_handlers/litellm_ai_handler.py [136-139] -if hasattr(self, 'main_pr_language'): - response_log['main_pr_language'] = self.main_pr_language -else: - response_log['main_pr_language'] = 'unknown' +# This line should be added in the class constructor (__init__) +self.main_pr_language = 'unknown' +# Replace the conditional block with a direct assignment +response_log['main_pr_language'] = self.main_pr_language
Possible issue	Ensure main_language is defined before use to avoid potential errors. Ensure that main_language is always defined before assigning it to self.ai_handler.main_pr_language. This could involve setting a default value or raising an error if main_language is not set. pr_agent/tools/pr_add_docs.py [29] -self.ai_handler.main_pr_language = self.main_language +if hasattr(self, 'main_language'): + self.ai_handler.main_pr_language = self.main_language +else: + raise ValueError("main_language must be defined before assigning to ai_handler")

---

✨ Improve tool usage guide:

---

Overview:
The improve tool scans the PR code changes, and automatically generates suggestions for improving the PR code. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on a PR.
When commenting, to edit configurations related to the improve tool (pr_code_suggestions section), use the following template:

/improve --pr_code_suggestions.some_config1=... --pr_code_suggestions.some_config2=...

With a configuration file, use the following template:

[pr_code_suggestions]
some_config1=...
some_config2=...

Enabling\disabling automation When you first install the app, the default mode for the improve tool is: pr_commands = ["/improve --pr_code_suggestions.summarize=true", ...] meaning the improve tool will run automatically on every PR, with summarization enabled. Delete this line to disable the tool from running automatically.

Utilizing extra instructions Extra instructions are very important for the improve tool, since they enable to guide the model to suggestions that are more relevant to the specific needs of the project. Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify relevant aspects that you want the model to focus on. Examples for extra instructions: [pr_code_suggestions] # /improve # extra_instructions=""" Emphasize the following aspects: - Does the code logic cover relevant edge cases? - Is the code logic clear and easy to understand? - Is the code logic efficient? ... """ Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

A note on code suggestions quality While the current AI for code is getting better and better (GPT-4), it's not flawless. Not all the suggestions will be perfect, and a user should not accept all of them automatically. Suggestions are not meant to be simplistic. Instead, they aim to give deep feedback and raise questions, ideas and thoughts to the user, who can then use his judgment, experience, and understanding of the code base. Recommended to use the 'extra_instructions' field to guide the model to suggestions that are more relevant to the specific needs of the project, or use the custom suggestions 💎 tool With large PRs, best quality will be obtained by using 'improve --extended' mode.

More PR-Agent commands To invoke the PR-Agent, add a comment using one of the following commands: /review: Request a review of your Pull Request. /describe: Update the PR title and description based on the contents of the PR. /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback. /ask <QUESTION>: Ask a question about the PR. /update_changelog: Update the changelog based on the PR's contents. /add_docs 💎: Generate docstring for new components introduced in the PR. /generate_labels 💎: Generate labels for the PR based on the PR's contents. /analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component. See the tools guide for more details. To list the possible configuration parameters, add a /config comment.

See the improve usage page for a more comprehensive guide on using this tool.

[mrT23]

Persistent review updated to latest commit 669e076

[mrT23]

Persistent review updated to latest commit 669e076