-
Notifications
You must be signed in to change notification settings - Fork 69
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(targets): Safely skip parsing record field as date-time if it is missing in schema #1844
fix(targets): Safely skip parsing record field as date-time if it is missing in schema #1844
Conversation
Codecov Report
@@ Coverage Diff @@
## main #1844 +/- ##
==========================================
+ Coverage 87.16% 87.39% +0.22%
==========================================
Files 59 59
Lines 5120 5123 +3
Branches 827 828 +1
==========================================
+ Hits 4463 4477 +14
+ Misses 463 451 -12
- Partials 194 195 +1
... and 1 file with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
…exception-for-key-not-present-in-schema
…exception-for-key-not-present-in-schema
…exception-for-key-not-present-in-schema
…exception-for-key-not-present-in-schema
…exception-for-key-not-present-in-schema
singer_sdk/sinks/core.py
Outdated
if key not in schema["properties"]: | ||
self.logger.debug("No schema for record field '%s'", key) | ||
continue |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using the in
operator is a bit faster than try-except:
import timeit
setup = """
big_dict = {
i: i * i
for i in range(50_000)
}
"""
try_except = """
for i in range(70_000):
try:
x = big_dict[i]
except KeyError:
continue
"""
in_operator = """
for i in range(70_000):
if i in big_dict:
x = big_dict[i]
"""
print(timeit.timeit(setup=setup, stmt=try_except, number=1000)) # 4.430988875003095
print(timeit.timeit(setup=setup, stmt=in_operator, number=1000)) # 2.137157458000729
…exception-for-key-not-present-in-schema
…exception-for-key-not-present-in-schema
…exception-for-key-not-present-in-schema
Closes #1836
📚 Documentation preview 📚: https://meltano-sdk--1844.org.readthedocs.build/en/1844/