-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[PRMP-1063] blank bulk upload reports don't create a file #455
Conversation
if data_rows: | ||
self.write_additional_report_items_to_csv( | ||
file_name=file_name, headers=headers, rows_to_write=data_rows | ||
) | ||
|
||
logger.info("Uploading daily success report file to S3") | ||
self.s3_service.upload_file( | ||
s3_bucket_name=self.reports_bucket, | ||
file_key=f"{self.s3_key_prefix}/{file_name}", | ||
file_name=f"/tmp/{file_name}", | ||
) | ||
logger.info("Uploading daily success report file to S3") | ||
self.s3_service.upload_file( | ||
s3_bucket_name=self.reports_bucket, | ||
file_key=f"{self.s3_key_prefix}/{file_name}", | ||
file_name=f"/tmp/{file_name}", | ||
) | ||
else: | ||
logger.info("No data to report for daily success report file") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it possible to take this out to a new function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It feels like scope creep to me as the only functionality that has changed properly is line 173, though I agree that the code would be cleaner this way. @abbas-khan10 what do you reckon?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have no preference, I had considered this when first implementing as there is repetition. What were you thinking @NogaNHS? Something like:
def write_and_upload_additional_reports(file_name, headers, data_rows) -> bool:
if data_rows:
self.write_additional_report_items_to_csv(
file_name=file_name, headers=headers, rows_to_write=data_rows
)
logger.info("Uploading daily success report file to S3")
self.s3_service.upload_file(
s3_bucket_name=self.reports_bucket,
file_key=f"{self.s3_key_prefix}/{file_name}",
file_name=f"/tmp/{file_name}",
)
return true
return false
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The bool would serve as a way of customising the log message for each report type as we want to be specific with which report wasn't created (open to discussion obviously) e.g. for the code above:
report_uploaded = write_and_upload_additional_reports(file_name, headers, data_rows)
if not report_uploaded:
logger.info("No data to report for daily success report file")
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As opposed to doing a true/false return, I was going to suggest something more like this, so that it only calls 'write and upload' if it actually needs to:
if data_rows:
logger.info("Uploading daily success report file to S3")
self.write_and_upload_additional_reports(file_name, headers, data_rows)
else:
logger.info("No data to report for daily success report file")
def write_and_upload_additional_reports(self, file_name, headers, data_rows):
self.write_additional_report_items_to_csv(
file_name=file_name, headers=headers, rows_to_write=data_rows
)
self.s3_service.upload_file(
s3_bucket_name=self.reports_bucket,
file_key=f"{self.s3_key_prefix}/{file_name}",
file_name=f"/tmp/{file_name}",
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Logic has been broken out into a new function
logger.info("Uploading daily restricted report file to S3") | ||
self.write_and_upload_additional_reports(file_name, headers, data_rows) | ||
else: | ||
logger.info("No data to report for daily deceased report file") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
daily restricted*
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Corrected
@@ -400,3 +375,14 @@ def get_times_for_scan(self) -> tuple[datetime, datetime]: | |||
self.s3_key_prefix = f"bulk-upload-reports/{date_folder_name}" | |||
|
|||
return start_timestamp, end_timestamp | |||
|
|||
def write_and_upload_additional_reports(self, file_name, headers, data_rows): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could we add param types here please? See above method params for examples
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added
108eac4
to
801e89b
Compare
def write_and_upload_additional_reports( | ||
self, | ||
file_name: str, | ||
headers: list[str], | ||
data_rows: list[list[str]], | ||
): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a small unit test for this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Discussed with Abbas - this logic is being tested by each of the daily report unit tests.
No description provided.