Replies: 3 comments
-
@wllgrnt Thank you for posting this idea! I want to make sure that more team members get to see it before we decide to move on it. |
Beta Was this translation helpful? Give feedback.
0 replies
-
@jcampbell I think it is a good enhancement - thoughts? |
Beta Was this translation helpful? Give feedback.
0 replies
-
+1 on this request |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Is your feature request related to a problem? Please describe.
In our tables we have a fixed number of rows that fail the column expectations for historic reasons, but the tables grow all the time so the required
mostly
percentage to ensure new bad data is caught shrinks every day. If we could extend the idea behind themostly
kwarg to allow me to set a number of rows I expect to fail (i.e. success is when unexpected_count > bla, I think) then we could do much more stringent testing.Describe the solution you'd like
Two obvious options would be:
Check if
mostly
is a float[0,1)
. If so, run the normal mostly logic. Otherwise ifmostly
is an int, interpret it as the maximum number of unexpected rows to allow. (Otherwise handle the invalid argument as per usual).Add a new kwarg called e.g.
mostly_absolute
and check that the function caller doesn't try to use both.I defer to the core devs on which they'd prefer, happy to submit a PR once there's consensus on the best signature!
Beta Was this translation helpful? Give feedback.
All reactions