-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
3bcfae5
commit 7d8b309
Showing
6 changed files
with
139 additions
and
0 deletions.
There are no files selected for viewing
Binary file added
BIN
+17.8 KB
tables/S1 Table_Full reporting of agreement and reliability for statistical raters.docx
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
% latex table generated in R 4.3.3 by xtable 1.8-4 package | ||
% Sun May 26 19:04:15 2024 | ||
\begin{table}[H] | ||
\begin{widestuff} | ||
\caption{Observed prevalence and 95\% confidence interval for reported statistical behaviours.} | ||
\begin{flushleft} \begin{tabular}{p{9cm}lll} | ||
\toprule | ||
{\bf Variables} & {\bf N} & {\bf n (\%) } & {\bf 95\% CI} \\ | ||
\midrule | ||
Coefficients & 95 & 77 (81\%) & 72\%, 88\% \\ | ||
Confidence intervals & 95 & 44 (46\%) & 37\%, 56\% \\ | ||
Standard error & 95 & 19 (20\%) & 13\%, 29\% \\ | ||
\ $R^2$ & 95 & 38 (40\%) & 31\%, 50\% \\ | ||
F/t statistics & 95 & 16 (17\%) & 11\%, 26\% \\ | ||
Degrees of freedom & 95 & 8 (8\%) & 4\%, 16\% \\ | ||
Number of observations in models & 95 & 45 (47\%) & 38\%, 57\% \\ | ||
Has the direction of the parameter estimates been interpreted? & 95 & 63 (66\%) & 56\%, 75\% \\ | ||
Has the size of the parameter estimates been interpreted? & 95 & 22 (23\%) & 16\%, 33\% \\ | ||
Have p-values been reported? & 95 & & \\ | ||
No & & 8 (8\%) & 4\%, 16\% \\ | ||
Mostly reported categorically & & 17 (18\%) & 11\%, 27\% \\ | ||
Mostly reported continuously & & 70 (74\%) & 64\%, 81\% \\ | ||
Have authors discussed the scientific importance of parameter estimates? & 95 & & \\ | ||
No & & 66 (69\%) & 60\%, 78\% \\ | ||
Yes, but only in a generic way & & 18 (19\%) & 12\%, 28\% \\ | ||
Yes, linked size of effect back to outcome variable & & 11 (12\%) & 7\%, 20\% \\ | ||
Was collinearity of X variables in models evaluated? & 95 & & \\ | ||
Not required & & 22 (23\%) & 16\%, 33\% \\ | ||
No & & 60 (63\%) & 53\%, 72\% \\ | ||
Yes & & 13 (14\%) & 8\%, 22\% \\ | ||
Were any continuous variables transformed not including categorisation? & 95 & & \\ | ||
No & & 65 (68\%) & 59\%, 77\% \\ | ||
Yes, but did not describe reasoning for transformation & & 13 (14\%) & 8\%, 22\% \\ | ||
Yes, described reasoning for transformation & & 17 (18\%) & 11\%, 27\% \\ | ||
Were continuous variables on a very large or small scales in the model scaled appropriately? & 95 & & \\ | ||
Not required & & 50 (53\%) & 43\%, 62\% \\ | ||
Unclear & & 9 (9\%) & 5\%, 17\% \\ | ||
No & & 5 (5\%) & 2\%, 12\% \\ | ||
Yes & & 31 (33\%) & 24\%, 43\% \\ | ||
\bottomrule | ||
\end{tabular} \end{flushleft} \begin{flushleft} N = Number of papers; n (\%) = Prevalence; 95\% CI = Wilson 95\% confidence intervals. \end{flushleft} | ||
\end{widestuff} | ||
\end{table} | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
% latex table generated in R 4.3.3 by xtable 1.8-4 package | ||
% Sun May 26 19:04:15 2024 | ||
\begin{table}[H] | ||
\begin{widestuff} | ||
\caption{Observed prevalence and 95\% confidence interval model selection and reporting.} | ||
\begin{flushleft} \begin{tabular}{p{9cm}lll} | ||
\toprule | ||
{\bf Variables} & {\bf N} & {\bf n (\%) } & {\bf 95\% CI} \\ | ||
\midrule | ||
Is there any process for selecting the variables included in the final model? & 95 & & \\ | ||
Unclear & & 19 (20\%) & 13\%, 29\% \\ | ||
Univariate modeling only (one X variable) & & 22 (23\%) & 16\%, 33\% \\ | ||
Model based on reference literature or author knowledge & & 36 (38\%) & 29\%, 48\% \\ | ||
Significant variables from univariate analysis were included in a multivariable model & & 3 (3\%) & 1\%, 9\% \\ | ||
Used recognised statistical modeling strategy & & 13 (14\%) & 8\%, 22\% \\ | ||
Other & & 2 (2\%) & 1\%, 7\% \\ | ||
Which variable selection strategy was used? & 95 & & \\ | ||
No recognised modeling strategy & & 82 (86\%) & 78\%, 92\% \\ | ||
Forwards & & 2 (2\%) & 1\%, 7\% \\ | ||
Backwards & & 4 (4\%) & 2\%, 10\% \\ | ||
Stepwise & & 7 (7\%) & 4\%, 14\% \\ | ||
Information criterion & & 0 (0\%) & 0\%, 4\% \\ | ||
Regularisation methods & & 0 (0\%) & 0\%, 4\% \\ | ||
Other & & 0 (0\%) & 0\%, 4\% \\ | ||
Does the paper mention any statistical significance criteria for including variables? & 95 & & \\ | ||
No recognised modeling strategy & & 82 (86\%) & 78\%, 92\% \\ | ||
No & & 7 (7\%) & 4\%, 14\% \\ | ||
Yes & & 6 (6\%) & 3\%, 13\% \\ | ||
\bottomrule | ||
\end{tabular} \end{flushleft} \begin{flushleft} N = Number of papers; n (\%) = Prevalence; 95\% CI = Wilson 95\% confidence intervals. \end{flushleft} | ||
\end{widestuff} | ||
\end{table} | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
% latex table generated in R 4.3.3 by xtable 1.8-4 package | ||
% Sun May 26 19:04:16 2024 | ||
\begin{table}[H] | ||
\begin{widestuff} | ||
\caption{ Agreement and reliability of statistical raters for 95 papers.} | ||
\begin{flushleft} \begin{tabular}{llllllllll} | ||
\toprule & \multicolumn{3}{l}{\textbf{Rating 1 vs Rating 2}} & \multicolumn{3}{l}{\textbf{Rating 1 vs Prevalence}} & \multicolumn{3}{l}{\textbf{Rating 2 vs Prevalence}} \\ | ||
\cline{2-10} | ||
{\bf Variable} & {\bf Agree} & {\bf Gwet} & {\bf 95\% CI} & {\bf Agree} & {\bf Gwet} & {\bf 95\% CI} & {\bf Agree} & {\bf Gwet} & {\bf 95\% CI} \\ | ||
\midrule | ||
Degrees of freedom & 92\% & 0.91 & 0.84, 0.98 & 96\% & 0.95 & 0.90, 1.00 & 92\% & 0.90 & 0.84, 0.97 \\ | ||
Variable selection strategy & 92\% & 0.91 & 0.85, 0.97 & 94\% & 0.93 & 0.88, 0.99 & 89\% & 0.89 & 0.82, 0.96 \\ | ||
Model significance criteria & 91\% & 0.90 & 0.83, 0.97 & 92\% & 0.91 & 0.84, 0.97 & 91\% & 0.90 & 0.83, 0.96 \\ | ||
F/t statistics & 87\% & 0.82 & 0.72, 0.93 & 92\% & 0.89 & 0.80, 0.97 & 89\% & 0.85 & 0.75, 0.95 \\ | ||
Standard error & 86\% & 0.79 & 0.66, 0.91 & 92\% & 0.88 & 0.79, 0.96 & 91\% & 0.85 & 0.75, 0.95 \\ | ||
Coefficient & 85\% & 0.78 & 0.66, 0.90 & 94\% & 0.91 & 0.83, 0.98 & 92\% & 0.87 & 0.78, 0.96 \\ | ||
Confidence intervals & 85\% & 0.71 & 0.56, 0.85 & 88\% & 0.77 & 0.64, 0.90 & 88\% & 0.77 & 0.64, 0.90 \\ | ||
R-Squared & 82\% & 0.65 & 0.50, 0.81 & 89\% & 0.79 & 0.67, 0.92 & 84\% & 0.70 & 0.55, 0.84 \\ | ||
p-values & 75\% & 0.67 & 0.54, 0.79 & 86\% & 0.82 & 0.73, 0.92 & 83\% & 0.78 & 0.68, 0.89 \\ | ||
Direction interpreted & 72\% & 0.44 & 0.26, 0.63 & 80\% & 0.63 & 0.47, 0.79 & 77\% & 0.55 & 0.38, 0.72 \\ | ||
variables transformed & 69\% & 0.61 & 0.48, 0.75 & 78\% & 0.72 & 0.60, 0.83 & 78\% & 0.71 & 0.59, 0.83 \\ | ||
Size interpreted & 65\% & 0.43 & 0.23, 0.62 & 76\% & 0.62 & 0.46, 0.78 & 73\% & 0.55 & 0.38, 0.73 \\ | ||
Collinearity evaluated & 61\% & 0.45 & 0.30, 0.59 & 77\% & 0.68 & 0.56, 0.80 & 72\% & 0.60 & 0.47, 0.74 \\ | ||
N in models & 57\% & 0.16 & -0.05, 0.37 & 74\% & 0.49 & 0.31, 0.67 & 71\% & 0.41 & 0.22, 0.60 \\ | ||
Importance of parameters & 48\% & 0.27 & 0.11, 0.42 & 66\% & 0.54 & 0.40, 0.69 & 58\% & 0.42 & 0.27, 0.57 \\ | ||
Scaled appropriately & 40\% & 0.24 & 0.11, 0.38 & 52\% & 0.39 & 0.25, 0.53 & 46\% & 0.32 & 0.18, 0.45 \\ | ||
Process variable selection & 37\% & 0.25 & 0.13, 0.36 & 59\% & 0.51 & 0.39, 0.63 & 48\% & 0.39 & 0.26, 0.51 \\ | ||
\bottomrule | ||
\end{tabular} \end{flushleft} \begin{flushleft} Agree = Observed agreement, Gwet = Gwet agreement coefficient; 95\% CI = Gwet 95\% confidence intervals. \end{flushleft} | ||
\end{widestuff} | ||
\end{table} | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
% latex table generated in R 4.3.3 by xtable 1.8-4 package | ||
% Sun May 26 19:04:16 2024 | ||
\begin{table}[ht] | ||
\begin{widestuff} | ||
\caption{Social Ecological Model a system-wide approach to reform statistical practices adapted from Bronfenbrenner [66]} | ||
\begin{tabular}{p{5cm}p{13cm}} | ||
\toprule | ||
{\bf Level} & {\bf Description} \\ | ||
\midrule | ||
Individual & - Individual characteristics that influence behaviour change, including knowledge, statistical literacy, beliefs, attitudes, and personal traits such as gender, years of experience and job security. \\ | ||
Interpersonal & - Formal and informal social networks and support systems that can influence individual behaviours to promote interpersonal growth that encourages good statistical practice including peers and co-workers, and mentors. \\ | ||
Community & - Formal or informal social norms can limit or encourage good statistical and research behaviours among individuals, groups, or organisations. | ||
\\ | ||
Organisational & - Organisations rules and regulations for operations, for example, ethics committees in hospitals or universities. | ||
|
||
- Access to research infrastructure, including statistical resources such as statistical programs and educational resources. | ||
|
||
- Access to qualified statisticians. | ||
|
||
- Sustainable research metrics with quality valued over quantity. | ||
|
||
- Organisational oversite of research misconduct. \\ | ||
Policy Environment & - Local, state, national and global policies regarding the allocation of resources for meta-research to tackle systematic issues. | ||
|
||
- Regulatory bodies, state, national and international integrity commissions, for the oversite of research misconduct. \\ | ||
\bottomrule | ||
\end{tabular} | ||
\end{widestuff} | ||
\end{table} | ||
|