-
Notifications
You must be signed in to change notification settings - Fork 8
/
README.Rmd
145 lines (97 loc) · 5.22 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
---
title: 'OPER 782 Analytic Deployment Review'
date: '`r format(Sys.Date(), "%d %B %Y")`'
output:
github_document:
df_print: kable
---
<!-- badges: start -->
[](https://travis-ci.org/AFIT-R/OPER782.WI2020)
<!-- badges: end -->
Part of the analytic development process is ensuring that the algorithms you develop will be reproducible and portable for an end-user. Often an end-user’s machine is not configured in the same way as the developer’s – leading to unexpected side effects or errors. To ensure that your methods are reproducible & portable, your analytic data products will be reviewed on multiple ‘new’ machines representing those used by potential end-users.
In lieu of a final exam, each student will be tasked to review two analytic projects developed by students in the course. Using the information presented in the GitHub repository README file and any documentation files provided with the analytic, the reviewer will review the analytic in these areas:
- Accuracy
+ Are the results accurate?
+ Is it clear how the results were computed?
- Compilation
+ Did the analytic work? (i.e. install correctly, startup correctly)
+ Were errors encountered when executing code according to the documentation?
- Ease of use
+ Is it clear how to use the analytic? (i.e. is the documentation clear)
+ Are the visualizations/plots interpretable?
Reviewers will capture their findings by (1) generating an issue on the analytic project’s Github page for each finding and (2) present all findings in a final report (.Rmd file – html_document). Both the report and the Github issues should include a brief discussion of the problem with one or more reproducible examples.
After reviewing the analytic, the reviewer will then assign it one of the following grades. __Note: I will perform a second review on all projects to ensure that grades are assigned fairly. If my final grade differs greatly from the grade provided by the reviewer – it will be an indication of poor engagement by the reviewer.__
- Outstanding – ready to publish/deploy (50 pts)
+ All proposal tasks completed
+ Documentation is thorough and clear
+ No errors or warnings during build/use
- Excellent – very little rework required (45 pts)
+ Few proposal tasks incomplete
+ Documentation is clear and mostly complete
+ Minor errors or warnings during build/use
- Satisfactory – extensive rework required (40 pts)
+ Several proposal tasks incomplete
+ Documentation is limited and/or poorly written
+ Severe errors during build or in use
- Unsatisfactory – complete restart required (35 pts)
+ Little progress made on proposal tasks
+ Little to no documentation
+ Cannot be compiled, unusable
```{r, echo=FALSE}
# Define funtion to randomly assign N
# students as reviewers
reviews <- function(n.choices = 13) {
base = seq(1,as.integer(n.choices))
rev1 = rep(0, length(base))
rev2 = rep(0, length(base))
resample <- function(x, ...) x[sample.int(length(x), ...)]
for(i in 1:(length(base)-1)) {
choices1 <- base[base != base[i] & !base %in% rev1[1:i]]
rev1[i] <- resample(choices1, size = 1)
}
rev1[length(base)] <- base[!base %in% rev1]
for(i in 1:(length(base)-1)) {
choices2 <- base[base != base[i] & base != rev1[i] & !base %in% rev2[1:i]]
rev2[i] <- resample(choices2, size = 1)
}
rev2[length(base)] <- base[!base %in% rev2]
df <- data.frame(base = base,
rev1 = rev1,
rev2 = rev2)
return(df)
}
```
```{r, echo=FALSE}
students <- c("Marc Chale",
"Brandon Hufstetler",
"Benjamin Leiby",
"Benjamin Mayo",
"Ryan Walton",
"Clarence Williams",
"Clayton Wilson")
git_url <- c('https://github.com/marcchale/Rexis',
"https://github.com/bjhufstetler/LeapFrog",
'https://github.com/mrkaliv/logreg',
'https://github.com/HarmonicAnova21/FRame.BuildR',
'https://github.com/ryanwalton57/allyR',
'https://github.com/clarencew0083/recSystem',
"https://github.com/clawilso15/Text.Replace")
```
The table below lists each developer, the URL of their respective project, and the assigned reviewers.
```{r devtable, echo=FALSE, warning=FALSE}
df <- reviews(length(students))
review_table <- data.frame(Developer = students,
Git_URL = git_url,
Reviewer1 = students[df[,2]],
Reviewer2 = students[df[,3]])
knitr::kable(review_table,
caption = 'List of developers, projects, and assigned reviewers for OPER782 (Winter 2020)')
```
The assigned reviewers are to:
1. Download the project using `devtools::install_github('devname/projectname')`
2. Review the documentation provided
3. Test out the project in accordance with the documentation
4. Write a one-page review of the project using the rubric provided above.
5. Fork the project repository
6. Add your review to the inst/ directory of the project
7. Submit a pull request to the project developer to have them include the review as part of their project.