Skip to content

Commit

Permalink
stubs for io data storage and retention periods
Browse files Browse the repository at this point in the history
  • Loading branch information
taylordowns2000 committed Feb 1, 2024
1 parent 3f5d507 commit e99b911
Show file tree
Hide file tree
Showing 3 changed files with 61 additions and 4 deletions.
24 changes: 24 additions & 0 deletions docs/manage-projects/io-data-storage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
title: I/O Data Storage
---

In the "Data Storage" section, you can configure what you'd like Lightning to do
with the actual _data_ that's processed (or produced) by your workflow runs.

### Why would I store input/output data along with run logs?

One of the most powerful features of the platform is the ability to "replay"
work orders. If a particular case referral fails, an administrator can find the
failed work order, make a change to the workflow or wait for a destination
system to come online, and then reprocess that work order without having to
fetch (or re-send) the data from a source system.

### Why would I chose to _NOT_ store I/O data?

Some of our users are processing incredibly sensitive data (such as medical
records) and might want to ensure that after a workflow is run there is no
patient data left on OpenFn servers.

Enabling this "zero-persistence" feature for I/O data is an attractive option
for folks who want to use OpenFn on the cloud but are worried about data
sovereignty.
30 changes: 30 additions & 0 deletions docs/manage-projects/retention-periods.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
title: History Retention
---

In the "Data Storage" section, you can configure how long you'd like your run
history to be stored for a particular project.

:::tip

Note that when running Lightning on OpenFn.org, a project's plan may limit how
long we'll store your history. You can always store less, but paid plans get
longer storage.

:::

### What is history?

History includes work orders, runs, logs, and any associated input/output
dataclips.

### Why would I want to reduce my history retention period for a project?

Some project administrators chose to store history for a shorter duration (a
month or 90-days) to reduce data storage costs or to limit their data footprint.

### What happens when old work orders, runs, logs, or dataclips get removed?

If you've configured any automatic data exports or archiving, you can still
access your history in those CSVs, but all history will be removed from the
OpenFn platform.
11 changes: 7 additions & 4 deletions sidebars-main.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,8 @@ module.exports = {
'design/design-workflow',
'design/api-discovery',
'design/mapping-specs',
'design/workflow-specs'
'design/workflow-specs',
//'design/discovery'

],
},
// {
Expand Down Expand Up @@ -64,13 +63,17 @@ module.exports = {
label: 'Monitor History',
items: [
'monitor-history/activity-history',
'monitor-history/status-codes'
'monitor-history/status-codes',
],
},
{
type: 'category',
label: 'Manage Projects',
items: ['manage-projects/platform-mgmt'],
items: [
'manage-projects/platform-mgmt',
'manage-projects/retention-periods',
'manage-projects/io-data-storage',
],
},
{
type: 'category',
Expand Down

0 comments on commit e99b911

Please sign in to comment.