Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): disable unstable scap-related perf test from perf PR comment and check #2017

Merged
merged 2 commits into from
Aug 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 23 additions & 2 deletions .github/workflows/create-comment-perf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,34 @@ jobs:
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
# Taken from https://github.com/actions/github-script/blob/main/.github/workflows/pull-request-test.yml
script: |
var fs = require('fs');
var issue_number = Number(fs.readFileSync('./NR'));
var comment_body = fs.readFileSync('./COMMENT');
await github.rest.issues.createComment({

// Get the existing comments.
const {data: comments} = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue_number,
body: comment_body.toString('utf8')
});

// Find any comment already made by the bot.
const botComment = comments.find(comment => comment.user.id === 41898282 && comment.body.includes('# Perf diff from master'));

if (botComment) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: botComment.id,
body: comment_body.toString('utf8')
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue_number,
body: comment_body.toString('utf8')
});
}
26 changes: 14 additions & 12 deletions .github/workflows/perf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,11 +60,12 @@ jobs:
head -n10 "perf_tests_diff.txt" >> ./pr/COMMENT
echo "\`\`\`" >> ./pr/COMMENT
echo "" >> ./pr/COMMENT
echo "# Perf diff from master - scap file" >> ./pr/COMMENT
echo "\`\`\`" >> ./pr/COMMENT
head -n10 "perf_scap_diff.txt" >> ./pr/COMMENT
echo "\`\`\`" >> ./pr/COMMENT
echo "" >> ./pr/COMMENT
# Drop unstable perf results!
# echo "# Perf diff from master - scap file" >> ./pr/COMMENT
# echo "\`\`\`" >> ./pr/COMMENT
# head -n10 "perf_scap_diff.txt" >> ./pr/COMMENT
# echo "\`\`\`" >> ./pr/COMMENT
# echo "" >> ./pr/COMMENT
echo "# Heap diff from master - unit tests" >> ./pr/COMMENT
echo "\`\`\`" >> ./pr/COMMENT
tail -n3 "heaptrack_tests_diff.txt" >> ./pr/COMMENT
Expand Down Expand Up @@ -98,13 +99,14 @@ jobs:

# Check will fail if sum of all differences is >= 1%.
# But we will always comment with the perf diff from master
- name: Check >= 1% threshold - perf scap file
if: always() # Even if other threshold checks failed
run: |
sum=$(awk '{sum+=sprintf("%f",$2)}END{printf "%.6f\n",sum}' perf_scap_diff.txt | tr ',' '.')
if (( $(echo "$sum >= 1.0" | bc -l) )); then
exit 1
fi
# CHECK DISABLED: UNSTABLE PERF TEST.
# - name: Check >= 1% threshold - perf scap file
# if: always() # Even if other threshold checks failed
# run: |
# sum=$(awk '{sum+=sprintf("%f",$2)}END{printf "%.6f\n",sum}' perf_scap_diff.txt | tr ',' '.')
# if (( $(echo "$sum >= 1.0" | bc -l) )); then
# exit 1
# fi

# Check will fail if there is any heap memory usage difference >= 1M,
# or if there is new memory leaked.
Expand Down
Loading