Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Send Jobs to Orca #275

Merged
merged 75 commits into from
Aug 12, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
d8189aa
feat: grading job json generation
williams-jack Jul 2, 2023
08c1eb0
refactor: move script location and delay computation
williams-jack Jul 5, 2023
dee2012
fix: destructuring array doesn't work in each loop
williams-jack Jul 5, 2023
b8aee58
refactor: resource_files moved out of grader
williams-jack Jul 5, 2023
51ef051
refactor: move delay_for_sub to grader.rb
williams-jack Jul 6, 2023
137fab1
fix: replace cmd strings with string[]
williams-jack Jul 21, 2023
aa445a8
feat: zip processing for junit grader
williams-jack Jul 21, 2023
86b4e9a
fix: typos, conditional logic, and tree cmds
williams-jack Jul 25, 2023
926d678
feature: include starter and/or testing zips in job
williams-jack Jul 27, 2023
43803fa
feat: bottlenose saves Orca TAP
williams-jack Jul 27, 2023
1aee430
Merge branch 'master' into feature/handle-orca-response
williams-jack Jul 27, 2023
e7b5fac
Merge branch 'master' into feature/create-grading-job-jsons
williams-jack Jul 27, 2023
12be1b3
chore: rename grader_path to grader_dir
williams-jack Jul 27, 2023
404da95
chore: rename java dockerfile with shasum name
williams-jack Jul 27, 2023
f68bb9d
feat: docker sha for java image spec
williams-jack Jul 27, 2023
f38728b
feat: orca response shell logs and errors saved
williams-jack Jul 28, 2023
8385fa0
fix: response key used; route post spec
williams-jack Jul 28, 2023
423d106
Merge pull request #2 from williams-jack/feature/handle-orca-response
williams-jack Jul 29, 2023
d3f7f14
Merge pull request #1 from williams-jack/feature/create-grading-job-j…
williams-jack Jul 29, 2023
43ddb54
feat: orca POST req and secret creation
williams-jack Jul 29, 2023
ee8eabb
fix: typo and addition of status to exception
williams-jack Jul 29, 2023
c7717c6
feat: response route with secret handling
williams-jack Jul 29, 2023
da0df7d
fix: rm secret file after verification
williams-jack Jul 29, 2023
de5951b
qa: testing all states given for response handling
williams-jack Jul 30, 2023
ab74ec5
fix: response handler params and resource fetching
williams-jack Jul 30, 2023
0f9017d
refactor: pull out retry/backoff with req into helper
williams-jack Jul 30, 2023
8f148e0
fix: orca_response remove before action
williams-jack Jul 30, 2023
f885167
feat: settings orca default URL for dev
williams-jack Jul 30, 2023
0773736
fix: orca_response should not find grade
williams-jack Jul 30, 2023
0ef18f0
refactor: remove mailerhost site_url Settings refs
williams-jack Jul 31, 2023
8762c43
Merge branch 'master' into feature/send-jobs-to-orca
williams-jack Jun 27, 2024
fda2171
chore: revert app controller
williams-jack Jul 5, 2024
7d39199
fix: correct exponential retry pattern
williams-jack Jul 5, 2024
8851c06
chore: revert settings.rb
williams-jack Jul 6, 2024
2d29aee
refactor: clean up delay calc for sub
williams-jack Jul 6, 2024
2008eeb
fix: job sent to orca inside thread
williams-jack Jul 6, 2024
204bbfb
chore: rm sha named dockerfile
williams-jack Jul 6, 2024
a6a6b6b
feat: generate image build config for junit grader
williams-jack Jul 6, 2024
4981260
fix: update grades controller with new data defs
williams-jack Jul 9, 2024
42a40b3
fix: grader processor import and file path issue
williams-jack Jul 12, 2024
cb3b5cb
Merge branch 'master' into feature/send-jobs-to-orca
williams-jack Jul 12, 2024
86acd62
feat: new status column on graders to indicate orca usage
williams-jack Jul 13, 2024
d67a447
feat: checkbox for setting useOrca in status json
williams-jack Jul 14, 2024
5e45213
fix: deugging junit grader for orca compatibility
williams-jack Jul 16, 2024
4b3c4e4
refactor: copy grades controller orca logic to api
williams-jack Jul 16, 2024
c6c1e1c
refactor: move grades_controller test to test/api
williams-jack Jul 18, 2024
3c58c50
feat: initial funcitonality to render orca output
williams-jack Jul 19, 2024
140c0a2
refactor: simplify hashes as method returns
williams-jack Jul 19, 2024
eebaba4
Merge branch 'master' into feature/send-jobs-to-orca
williams-jack Jul 19, 2024
3e590a6
feat: block grader secrets from file endpoint
williams-jack Jul 19, 2024
6612be5
refactor: move grader secret path check to grader class
williams-jack Jul 20, 2024
997e7c7
Merge branch 'master' into feat/add-orca-status
williams-jack Jul 20, 2024
9c6a574
feat: initial controller and ui for build results
williams-jack Jul 20, 2024
17ef005
feat: build log ui and link from edit assignment
williams-jack Jul 21, 2024
470d111
fix: only render red highlight for specific error
williams-jack Jul 21, 2024
0a1df40
fix: orca_response doesn't need assignment or course id
williams-jack Jul 22, 2024
705f49e
chore: remove old junit script json file
williams-jack Jul 23, 2024
ca6ea90
feat: render orca tap output for junit
williams-jack Jul 23, 2024
f212585
fix: starter and testing zip files generated properly
williams-jack Jul 23, 2024
b9b5ec4
fix: change mime types and adjust script based on orca run
williams-jack Jul 23, 2024
b24b015
Merge branch 'feat/add-orca-status' into feature/send-jobs-to-orca
williams-jack Jul 25, 2024
67ac27e
refactor: move build result to file, use status col as boolean
williams-jack Jul 26, 2024
641abd6
feat: build request, rebuilding, logs, and statuses
williams-jack Jul 31, 2024
b90d8d3
feat: handle new job status data definition from orca
williams-jack Aug 2, 2024
4c65ed3
feat: use orca api key for mutation endpoints
williams-jack Aug 2, 2024
dddccb8
fix: use find module to include dot files in zip
williams-jack Aug 2, 2024
eed9fc0
fix: missing host in response_url for image build
williams-jack Aug 3, 2024
60a3ee8
feat: tap comparison between bottlenose and orca
williams-jack Aug 3, 2024
484d335
feat: rails agnostic script to push custom grader to orca
williams-jack Aug 4, 2024
b0ef09c
fix: up/down chevron dynamic based on click
williams-jack Aug 4, 2024
5f9de07
refactor: updated tap comparison handling
williams-jack Aug 5, 2024
5ea0ca0
feat: get all submissions with orca tap diff
williams-jack Aug 5, 2024
c8c77ce
feat: lock down orca-related controllers and views to admins
williams-jack Aug 5, 2024
cec0e70
fix: updated grades api tests
williams-jack Aug 5, 2024
731c183
test: grades api missing key and job status update
williams-jack Aug 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions app/models/grade.rb
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,32 @@ def full_log
self.grading_output&.sub(/#{go.basename}$/, "details.log")
end

def can_compare_orca_tap?
return false unless has_orca_output?
!(self.grading_output.nil? || orca_output['output'].nil?)
end

def orca_tap_comparison
local_tap = TapParser.new(File.read(self.grading_output_path))
orca_tap = TapParser.new(orca_output['output'])
unless local_tap.test_count == orca_tap.test_count
return "Test count does not match; Bottlenose: #{local_tap.test_count} / Orca: #{orca_tap.test_count}"
end
local_names_to_weights = local_tap.tests.map { |t| [t[:comment], t[:info]['weight']] }.to_h
williams-jack marked this conversation as resolved.
Show resolved Hide resolved
orca_names_to_weights = orca_tap.tests.map { |t| [t[:comment], t[:info]['weight']] }.to_h
unless local_names_to_weights.keys.sort == orca_names_to_weights.keys.sort
return 'Bottlenose and Orca TAP test names do not match.'
end
unless local_names_to_weights.all? { |n, w| orca_names_to_weights[n] == w }
return 'Bottlenose and Orca TAP weights do not match.'
end
unless local_tap.points_earned == orca_tap.points_earned
williams-jack marked this conversation as resolved.
Show resolved Hide resolved
return 'Bottlenose and Orca scores don\'t match. ' \
"Bottlenose #{local_tap.points_earned} / Orca: #{orca_tap.points_earned}"
end
'Bottlenose TAP and Orca TAP match.'
end

def grading_output_path
if self.grading_output
Upload.full_path_for(self.grading_output)
Expand Down
13 changes: 9 additions & 4 deletions app/views/grades/_orca_output.html.erb
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,13 @@
<h4>No output from Orca yet...</h4>
<% else %>
<% unless @orca_output['output'].nil? %>
<h4>Output:</h4>
<pre class="output"><%= @orca_output['output'] %></pre>
<h4>
<% if @grade.can_compare_orca_tap? %>
<%= @grade.orca_tap_comparison %>
<% else %>
Waiting on Bottlenose to return TAP output...
<% end %>
</h4>
<% end %>
<% unless @orca_output['errors'].blank? %>
<h4>Errors:</h4>
Expand All @@ -19,7 +24,7 @@
<div style="background-color: #eaeaea; padding: 15px 0 10px 10px; cursor: pointer;"
class="collapsed" data-toggle="collapse" data-target="#detailed-build"
id="details-collapse">
<h3 style="margin: 0 0 0 2px;">Detailed Build</h3>
<h3 style="margin: 0 0 0 2px;">Detailed Build <small><span id="details-chevron" class="glyphicon glyphicon-chevron-right"></span></small></h3>
williams-jack marked this conversation as resolved.
Show resolved Hide resolved
</div>
<div id="detailed-build" class="collapse" style="border: 1px solid #ddd; padding-right: 10px;">
<ol>
Expand All @@ -46,5 +51,5 @@
</ol>
</div>
</div>
<%end%>
<% end %>
<% end %>
Loading