Documentation (#88)

* Update maintainer to Megan (#69) * Update README.Rmd (#64) Co-authored-by: Jennit07 <67372904+Jennit07@users.noreply.github.com> * Bug - speed up `get_chi()` (#68) * Update to dev version * Make testthat run in parallel * Update variables to pass tests * Update indiv number of variables * Change exists tests to read * Set an environment var to make testthat use multiple CPUs * Revert changes and deal with NA chi/anon_chi * Update documentation * Style package * Update tests so that they pass * Style package * fix tests * Render `README.md` after changes to the `.Rmd` version * exclude from tests for now --------- Co-authored-by: James McMahon <james.mcmahon@phs.scot> Co-authored-by: Jennit07 <Jennit07@users.noreply.github.com> * Render `README.md` after changes to the `.Rmd` version (#70) Co-authored-by: github-merge-queue[bot] <github-merge-queue[bot]@users.noreply.github.com> Co-authored-by: Jennit07 <67372904+Jennit07@users.noreply.github.com> * Bump actions/checkout from 3 to 4 (#66) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jennit07 <67372904+Jennit07@users.noreply.github.com> * Bump peter-evans/create-pull-request from 4 to 5 (#65) Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4 to 5. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v4...v5) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jennit07 <67372904+Jennit07@users.noreply.github.com> * Bump stefanzweifel/git-auto-commit-action from 4 to 5 (#67) Bumps [stefanzweifel/git-auto-commit-action](https://github.com/stefanzweifel/git-auto-commit-action) from 4 to 5. - [Release notes](https://github.com/stefanzweifel/git-auto-commit-action/releases) - [Changelog](https://github.com/stefanzweifel/git-auto-commit-action/blob/master/CHANGELOG.md) - [Commits](stefanzweifel/git-auto-commit-action@v4...v5) --- updated-dependencies: - dependency-name: stefanzweifel/git-auto-commit-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jennit07 <67372904+Jennit07@users.noreply.github.com> * Bump JamesIves/github-pages-deploy-action from 4.4.3 to 4.5.0 (#71) Bumps [JamesIves/github-pages-deploy-action](https://github.com/jamesives/github-pages-deploy-action) from 4.4.3 to 4.5.0. - [Release notes](https://github.com/jamesives/github-pages-deploy-action/releases) - [Commits](JamesIves/github-pages-deploy-action@v4.4.3...v4.5.0) --- updated-dependencies: - dependency-name: JamesIves/github-pages-deploy-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/upload-artifact from 3 to 4 (#72) Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3 to 4. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@v3...v4) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jennit07 <67372904+Jennit07@users.noreply.github.com> * Bump peter-evans/create-pull-request from 5 to 6 (#74) Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 5 to 6. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v5...v6) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/cache from 3 to 4 (#73) Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@v3...v4) --- updated-dependencies: - dependency-name: actions/cache dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jennit07 <67372904+Jennit07@users.noreply.github.com> * Update README.md (#75) Updated to include reading in LTC 'catch all' variables * change in episode file cost variable vector (#76) Co-authored-by: marjom02 <megan.mcnicol2@nhs.scot> * force keytime format to hms (#77) * force keytime format to hms * Update documentation * visible binding for global variables like ‘keytime1’ * minor changes * fix keytime in column names * import hms --------- Co-authored-by: lizihao-anu <lizihao-anu@users.noreply.github.com> * Bump JamesIves/github-pages-deploy-action from 4.5.0 to 4.6.0 (#79) Bumps [JamesIves/github-pages-deploy-action](https://github.com/jamesives/github-pages-deploy-action) from 4.5.0 to 4.6.0. - [Release notes](https://github.com/jamesives/github-pages-deploy-action/releases) - [Commits](JamesIves/github-pages-deploy-action@v4.5.0...v4.6.0) --- updated-dependencies: - dependency-name: JamesIves/github-pages-deploy-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * add vignette for SLFhelper documentation * Style package * Hide messages * remove conflict * Style package * Split up documentation into 3 vignettes * add a comparison table to show the efficiency improvement * Update - round memory size * replace columns by col_select and add tidyselect * Style package * update ep_file_vars and indiv_file_vars * add session memory recommendation * Update R-CMD-check.yaml * fix cmd build error --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: James McMahon <james.mcmahon@phs.scot> Co-authored-by: Jennit07 <67372904+Jennit07@users.noreply.github.com> Co-authored-by: Jennit07 <Jennit07@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-merge-queue[bot] <github-merge-queue[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Megan McNicol <43570769+SwiftySalmon@users.noreply.github.com> Co-authored-by: marjom02 <megan.mcnicol2@nhs.scot> Co-authored-by: lizihao-anu <lizihao-anu@users.noreply.github.com> Co-authored-by: Jennifer Thom <jennifer.thom@phs.scot>
Public-Health-Scotland · Aug 19, 2024 · 85ed82f · 85ed82f
1 parent 0cb714f
commit 85ed82f
Show file tree

Hide file tree

Showing 26 changed files with 832 additions and 152 deletions.
diff --git a/.github/workflows/R-CMD-check.yaml b/.github/workflows/R-CMD-check.yaml
@@ -18,14 +18,14 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        r_version: ['3.6.1', '4.0.2', '4.1.2', 'release', 'devel']
+        r_version: ['4.0.2', '4.1.2', 'release']
 
     env:
       GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
       R_KEEP_PKG_SOURCE: yes
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - uses: r-lib/actions/setup-pandoc@v2
 

diff --git a/.github/workflows/document.yaml b/.github/workflows/document.yaml
@@ -13,7 +13,7 @@ jobs:
       GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
     steps:
       - name: Checkout repo
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         with:
           fetch-depth: 0
 
@@ -34,7 +34,7 @@ jobs:
 
       - name: Commit and create a Pull Request on development
         if: ${{ github.ref == 'refs/heads/development' }}
-        uses: peter-evans/create-pull-request@v4
+        uses: peter-evans/create-pull-request@v6
         with:
           commit-message: "Update documentation"
           branch: document_development
@@ -46,6 +46,6 @@ jobs:
 
       - name: Commit and push changes on all other branches
         if: ${{ github.ref != 'refs/heads/development' }}
-        uses: stefanzweifel/git-auto-commit-action@v4
+        uses: stefanzweifel/git-auto-commit-action@v5
         with:
           commit_message: "Update documentation"
diff --git a/.github/workflows/lint.yaml b/.github/workflows/lint.yaml
@@ -14,7 +14,7 @@ jobs:
     env:
       GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - uses: r-lib/actions/setup-r@v2
         with:

diff --git a/.github/workflows/pkgdown.yaml b/.github/workflows/pkgdown.yaml
@@ -22,7 +22,7 @@ jobs:
     permissions:
       contents: write
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - uses: r-lib/actions/setup-pandoc@v2
 
@@ -41,7 +41,7 @@ jobs:
 
       - name: Deploy to GitHub pages 🚀
         if: github.event_name != 'pull_request'
-        uses: JamesIves/github-pages-deploy-action@v4.4.3
+        uses: JamesIves/github-pages-deploy-action@v4.6.0
         with:
           clean: false
           branch: gh-pages

diff --git a/.github/workflows/render-README.yaml b/.github/workflows/render-README.yaml
@@ -13,7 +13,7 @@ jobs:
       GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
     steps:
       - name: Checkout repo
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         with:
           fetch-depth: 0
 
@@ -35,7 +35,7 @@ jobs:
 
       - name: Commit and create a Pull Request on production
         if: ${{ github.ref == 'refs/heads/production' }}
-        uses: peter-evans/create-pull-request@v5
+        uses: peter-evans/create-pull-request@v6
         with:
           commit-message: "Render `README.md` after changes to the `.Rmd` version"
           branch: render_readme
@@ -47,6 +47,6 @@ jobs:
 
       - name: Commit and push changes on all other branches
         if: ${{ github.ref != 'refs/heads/production' }}
-        uses: stefanzweifel/git-auto-commit-action@v4
+        uses: stefanzweifel/git-auto-commit-action@v5
         with:
           commit_message: "Render `README.md` after changes to the `.Rmd` version"
diff --git a/.github/workflows/style.yaml b/.github/workflows/style.yaml
@@ -13,7 +13,7 @@ jobs:
       GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
     steps:
       - name: Checkout repo
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         with:
           fetch-depth: 0
 
@@ -46,7 +46,7 @@ jobs:
         shell: Rscript {0}
 
       - name: Cache styler
-        uses: actions/cache@v3
+        uses: actions/cache@v4
         with:
           path: ${{ steps.styler-location.outputs.location }}
           key: ${{ runner.os }}-styler-${{ github.sha }}
@@ -60,7 +60,7 @@ jobs:
 
       - name: Commit and create a Pull Request on development
         if: ${{ github.ref == 'refs/heads/development' }}
-        uses: peter-evans/create-pull-request@v4
+        uses: peter-evans/create-pull-request@v6
         with:
           commit-message: "Style package"
           branch: document_development
@@ -72,6 +72,6 @@ jobs:
 
       - name: Commit and push changes on all other branches
         if: ${{ github.ref != 'refs/heads/development' }}
-        uses: stefanzweifel/git-auto-commit-action@v4
+        uses: stefanzweifel/git-auto-commit-action@v5
         with:
           commit_message: "Style package"
diff --git a/.github/workflows/test-coverage.yaml b/.github/workflows/test-coverage.yaml
@@ -15,7 +15,7 @@ jobs:
       GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - uses: r-lib/actions/setup-r@v2
         with:
@@ -44,7 +44,7 @@ jobs:
 
       - name: Upload test results
         if: failure()
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
           name: coverage-test-failures
           path: ${{ runner.temp }}/package
diff --git a/DESCRIPTION b/DESCRIPTION
@@ -4,15 +4,14 @@ Title: Useful functions for working with the Source Linkage Files
 Version: 0.10.0.9000
 Authors@R: c(
     person("Public Health Scotland", , , "phs.source@phs.scot", role = "cph"),
-    person("James", "McMahon", , "james.mcmahon@phs.scot", role = c("cre", "aut"),
-           comment = c(ORCID = "0000-0002-5380-2029"))
+    person("James", "McMahon", , "james.mcmahon@phs.scot", role = c("aut"),
+           comment = c(ORCID = "0000-0002-5380-2029")),
+    person("Megan", "McNicol", , "megan.mcnicol2@phs.scot", role = c("cre", "aut"))
   )
-Description: This package provides a few helper functions for working with
-    the Source Linkage Files (SLFs). The functions are mainly focussed on
-    making the first steps of analysis easier. They can read in and filter
-    the files in an efficient way using minimal syntax. If you find a bug
-    or have any ideas for new functions or improvements get in touch or
-    submit a pull request.
+Description: This package provides helper functions for working with
+    the Source Linkage Files (SLFs). The functions are mainly focused on
+    making the first steps of analysis easier. They can read and filter
+    the files efficiently using minimal code.
 License: MIT + file LICENSE
 URL: https://public-health-scotland.github.io/slfhelper/,
     https://github.com/Public-Health-Scotland/slfhelper
@@ -25,6 +24,7 @@ Imports:
     dplyr (>= 1.1.2),
     fs (>= 1.6.2),
     fst (>= 0.9.8),
+    hms,
     lifecycle (>= 1.0.3),
     magrittr (>= 2.0.3),
     openssl (>= 2.0.6),

diff --git a/R/get_anon_chi.R b/R/get_anon_chi.R
@@ -54,7 +54,11 @@ get_anon_chi <- function(chi_cohort, chi_var = "chi", drop = TRUE, check = TRUE)
   lookup <- tibble::tibble(
     chi = unique(chi_cohort[[chi_var]])
   ) %>%
-    dplyr::mutate(anon_chi = convert_chi_to_anon_chi(.data$chi))
+    dplyr::mutate(
+      chi = dplyr::if_else(is.na(.data$chi), "", .data$chi),
+      anon_chi = purrr::map_chr(.data$chi, openssl::base64_encode),
+      anon_chi = dplyr::if_else(.data$anon_chi == "", NA_character_, .data$anon_chi)
+    )
 
   chi_cohort <- chi_cohort %>%
     dplyr::left_join(

diff --git a/R/get_chi.R b/R/get_chi.R
@@ -19,8 +19,11 @@ get_chi <- function(data, anon_chi_var = "anon_chi", drop = TRUE) {
   lookup <- tibble::tibble(
     anon_chi = unique(data[[anon_chi_var]])
   ) %>%
-    dplyr::mutate(chi = convert_anon_chi_to_chi(.data$anon_chi))
-
+    dplyr::mutate(
+      anon_chi = dplyr::if_else(is.na(.data$anon_chi), "", .data$anon_chi),
+      chi = unname(convert_anon_chi_to_chi(.data$anon_chi)),
+      chi = dplyr::if_else(.data$chi == "", NA_character_, .data$chi)
+    )
   data <- data %>%
     dplyr::left_join(
       lookup,
@@ -36,17 +39,10 @@ get_chi <- function(data, anon_chi_var = "anon_chi", drop = TRUE) {
   return(data)
 }
 
-convert_anon_chi_to_chi <- function(anon_chi) {
-  chi <- purrr::map_chr(
-    anon_chi,
-    ~ dplyr::case_match(.x,
-      NA_character_ ~ NA_character_,
-      "" ~ "",
-      .default = openssl::base64_decode(.x) %>%
-        substr(2, 2) %>%
-        paste0(collapse = "")
-    )
-  )
+convert_anon_chi_to_chi <- Vectorize(function(anon_chi) {
+  chi <- openssl::base64_decode(anon_chi) %>%
+    substr(2, 2) %>%
+    paste0(collapse = "")
 
   return(chi)
-}
+})
diff --git a/R/read_slf.R b/R/read_slf.R
@@ -182,6 +182,17 @@ read_slf_episode <- function(
       dev = dev
     )
   )
+
+  if ("keytime1" %in% colnames(data)) {
+    data <- data %>%
+      dplyr::mutate(keytime1 = hms::as_hms(.data$keytime1))
+  }
+  if ("keytime2" %in% colnames(data)) {
+    data <- data %>%
+      dplyr::mutate(keytime2 = hms::as_hms(.data$keytime2))
+  }
+
+  return(data)
 }
 
 #' Read a Source Linkage individual file

diff --git a/README.Rmd b/README.Rmd
@@ -23,13 +23,19 @@ knitr::opts_chunk$set(
 
 # slfhelper
 
-The goal of slfhelper is to provide some easy-to-use functions that make working with the Source Linkage Files as painless and efficient as possible.
+The goal of slfhelper is to provide some easy-to-use functions that make working with the Source Linkage Files as painless and efficient as possible. It is only intended for use by PHS employees and will only work on the PHS R infrastructure.
 
 ## Installation
 
-The preferred method of installation is to use the [{`pak`} package](https://pak.r-lib.org/), which does an excellent job of handling the errors which sometimes occur. 
+The simplest way to install to the PHS Posit Workbench environment is to use the [PHS Package Manager](https://ppm.publichealthscotland.org/client/#/repos/3/packages/slfhelper), this will be the default setting and means you can install `slfhelper` as you would any other package.
 
-```{r package_install}
+``` {r package_install_ppm}
+install.packages("slfhelper")
+```
+
+If this doesn't work you can install it directly from GitHub, there are a number of ways to do this, we recommend the [{`pak`} package](https://pak.r-lib.org/).
+
+```{r package_install_github}
 # Install pak (if needed)
 install.packages("pak")
 
@@ -41,9 +47,9 @@ pak::pak("Public-Health-Scotland/slfhelper")
 
 ### Read a file
 
-**Note:** Reading a full file is quite slow and will use a lot of memory, we would always recommend doing a column selection to only keep the variables that you need for your analysis. Just doing this will dramatically speed up the read-time.
+**Note:** Reading a full file is quite slow and will use a lot of memory, we would always recommend doing a column selection to only keep the variables that you need for your analysis. Just doing this will dramatically speed up the read time.
 
-We provide some data snippets to help with the column selection and filtering.
+We provide some data snippets to help with column selection and filtering.
 
 ```{r helper_data}
 library(slfhelper)
@@ -99,11 +105,11 @@ ep_1718 <- read_slf_episode(c("1718", "1819", "1920"),
 ) %>%
   get_chi()
 
-# Change chi numbers from data above back to anon_chi
+# Change chi numbers from the data above back to anon_chi
 ep_1718_anon <- ep_1718 %>%
   get_anon_chi(chi_var = "chi")
 
-# Add anon_chi to cohort sample
+# Add anon_chi to the cohort sample
 chi_cohort <- chi_cohort %>%
   get_anon_chi(chi_var = "upi_number")
 ```
diff --git a/README.md b/README.md
@@ -14,13 +14,24 @@ stable](https://img.shields.io/badge/lifecycle-stable-brightgreen.svg)](https://
 
 The goal of slfhelper is to provide some easy-to-use functions that make
 working with the Source Linkage Files as painless and efficient as
-possible.
+possible. It is only intended for use by PHS employees and will only
+work on the PHS R infrastructure.
 
 ## Installation
 
-The preferred method of installation is to use the [{`pak`}
-package](https://pak.r-lib.org/), which does an excellent job of
-handling the errors which sometimes occur.
+The simplest way to install to the PHS Posit Workbench environment is to
+use the [PHS Package
+Manager](https://ppm.publichealthscotland.org/client/#/repos/3/packages/slfhelper),
+this will be the default setting and means you can install `slfhelper`
+as you would any other package.
+
+``` r
+install.packages("slfhelper")
+```
+
+If this doesn’t work you can install it directly from GitHub, there are
+a number of ways to do this, we recommend the [{`pak`}
+package](https://pak.r-lib.org/).
 
 ``` r
 # Install pak (if needed)
@@ -37,9 +48,9 @@ pak::pak("Public-Health-Scotland/slfhelper")
 **Note:** Reading a full file is quite slow and will use a lot of
 memory, we would always recommend doing a column selection to only keep
 the variables that you need for your analysis. Just doing this will
-dramatically speed up the read-time.
+dramatically speed up the read time.
 
-We provide some data snippets to help with the column selection and
+We provide some data snippets to help with column selection and
 filtering.
 
 ``` r
@@ -54,6 +65,31 @@ View(partnerships)
 
 # See a list with descriptions for the recids
 View(recids)
+
+# See a list of Long term conditions
+View(ltc_vars)
+
+# See a list of bedday related variables
+View(ep_file_bedday_vars)
+
+# See a list of cost related variables
+View(ep_file_cost_vars)
+```
+
+``` r
+library(slfhelper)
+
+# Read a group of variables e.g. LTCs (arth, asthma, atrialfib etc)
+# A nice 'catch all' for reading in all of the LTC variables
+ep_1718 <- read_slf_episode("1718", col_select = c("anon_chi", ltc_vars))
+
+# Read in a group of variables e.g. bedday related variables (yearstay, stay, apr_beddays etc)
+# A 'catch all' for reading in bedday related variables
+ep_1819 <- read_slf_episode("1819", col_select = c("anon_chi", ep_file_bedday_vars))
+
+# Read in a group of variables e.g. cost related variables (cost_total_net, apr_cost)
+# A 'catch all' for reading in cos related variables
+ep_1920 <- read_slf_episode("1920", col_select = c("anon_chi", ep_file_cost_vars))
 ```
 
 ``` r
@@ -97,11 +133,11 @@ ep_1718 <- read_slf_episode(c("1718", "1819", "1920"),
 ) %>%
   get_chi()
 
-# Change chi numbers from data above back to anon_chi
+# Change chi numbers from the data above back to anon_chi
 ep_1718_anon <- ep_1718 %>%
   get_anon_chi(chi_var = "chi")
 
-# Add anon_chi to cohort sample
+# Add anon_chi to the cohort sample
 chi_cohort <- chi_cohort %>%
   get_anon_chi(chi_var = "upi_number")
 ```
diff --git a/data/ep_file_cost_vars.rda b/data/ep_file_cost_vars.rda
diff --git a/data/ep_file_vars.rda b/data/ep_file_vars.rda
diff --git a/data/indiv_file_vars.rda b/data/indiv_file_vars.rda