Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add interval to the capture create/edits #1194

Closed
travjenkins opened this issue Aug 5, 2024 · 7 comments
Closed

Add interval to the capture create/edits #1194

travjenkins opened this issue Aug 5, 2024 · 7 comments
Assignees
Labels
create Creating an entity edit Editing an entity enhancement New feature or request

Comments

@travjenkins
Copy link
Member

travjenkins commented Aug 5, 2024

Waiting On estuary/flow#1750

We need to add the interval property on the process for creating and editing Captures. This will be added in a similar space as backfill' and auto discover` stuff. This field should really only be shown for certain connectors as it does not always make sense.

Design

The input should end up something like this:
image
image

Spec

image

@kiahna-tucker
Copy link
Member

The column, default_capture_interval, on the connector_tags table dictates whether the field should appear and what the default value should be.

@kiahna-tucker
Copy link
Member

kiahna-tucker commented Oct 24, 2024

The capture interval field has four options in the time unit menu: seconds, minutes, hours, and interval (in that order). The interval option was added to support capture intervals that span more than one time unit (e.g., four hours and thirty minutes). Please note the following characteristics:

  • A compound interval can take the shape of a Postgres interval (i.e., HH:MM:SS) or be in the format the specification expects (e.g., 1h 20m 10s, 1h 10s, etc.).

  • The selected time unit will be updated to interval accordingly if the field input matches a supported, interval format.

  • Plain, numeric input is considered invalid when the selected time unit is interval.

  • When the selected time unit moves from the interval option, the entered interval will be parsed in alignment with the new time unit. If the entered interval does not contain a segment that corresponds to the new time unit, the field will be empty. For example, say the entered interval is 1h 30m. If the hours option is selected, the field will contain 1. If the minutes option is selected, the field will contain 30. If the seconds option is selected, the field will be empty.

  • When the selected time unit moved to the interval option, the entered value remains unaltered. Manual formatting is required if it is not already in a supported, interval format.

@dyaffe let me know if you would like any changes to the described behavior.

@dyaffe
Copy link
Member

dyaffe commented Oct 25, 2024

@kiahna-tucker, I'm admittedly having a bit of trouble following exactly what the implementation is above but I think it's likely fine. For reference, when users input time intervals in other parts of the system, we usually just provide a list of preset values they can choose. Eg. 0s, 30s, 1m, 5m, 15m, 30m, 1h, 2h, 4h which would also be fine.

@travjenkins
Copy link
Member Author

Edge cases

Assume the max is around 584,942,417,355 years however that fails (but it can change based on the system)

99999999999years 99999999999months 99999999999days 99999999999hours 99999999999years 99999999999months 99999999999days 99999999999seconds - 217227016627years 8months 25days 9h 10m 39s

550000000000years - 550000000000year
600000000000years - fails

9999999999999999999seconds - 316880878140years 3months 14days 10h 5m 51s
99999999999999999999seconds - fails

@kiahna-tucker kiahna-tucker added the blocked Progress on the issue cannot ensue until a decision is made. label Nov 1, 2024
@kiahna-tucker
Copy link
Member

In the UI/UX meeting held on 10/30, the team decided to considerably simplify the capture interval field UX: reducing it to an autocompleted field with a set of predefined options (e.g., 0s, 30s, 1m, 5m, 15m, 30m, 1h, 2h, 4h, etc.). The field will support a value that is not one of the provided options and, as far as I am aware, this value does not need to be added to the set of options.

@kiahna-tucker kiahna-tucker removed the blocked Progress on the issue cannot ensue until a decision is made. label Nov 12, 2024
@dyaffe
Copy link
Member

dyaffe commented Nov 12, 2024

Looks great to me

@kiahna-tucker
Copy link
Member

Release Note: As of 11/15 at 9:40am, there are no capture connectors that have default_capture_interval set in order to show the interval field in production.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
create Creating an entity edit Editing an entity enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants