Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PRMP-1063] blank bulk upload reports don't create a file #455

Merged
merged 5 commits into from
Oct 30, 2024

Conversation

AndyFlintNHS
Copy link
Contributor

No description provided.

abbas-khan10
abbas-khan10 previously approved these changes Oct 29, 2024
Comment on lines 161 to 173
if data_rows:
self.write_additional_report_items_to_csv(
file_name=file_name, headers=headers, rows_to_write=data_rows
)

logger.info("Uploading daily success report file to S3")
self.s3_service.upload_file(
s3_bucket_name=self.reports_bucket,
file_key=f"{self.s3_key_prefix}/{file_name}",
file_name=f"/tmp/{file_name}",
)
logger.info("Uploading daily success report file to S3")
self.s3_service.upload_file(
s3_bucket_name=self.reports_bucket,
file_key=f"{self.s3_key_prefix}/{file_name}",
file_name=f"/tmp/{file_name}",
)
else:
logger.info("No data to report for daily success report file")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to take this out to a new function?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels like scope creep to me as the only functionality that has changed properly is line 173, though I agree that the code would be cleaner this way. @abbas-khan10 what do you reckon?

Copy link
Contributor

@abbas-khan10 abbas-khan10 Oct 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no preference, I had considered this when first implementing as there is repetition. What were you thinking @NogaNHS? Something like:

def write_and_upload_additional_reports(file_name, headers, data_rows) -> bool:
    if data_rows:
            self.write_additional_report_items_to_csv(
                file_name=file_name, headers=headers, rows_to_write=data_rows
            )

            logger.info("Uploading daily success report file to S3")
            self.s3_service.upload_file(
                s3_bucket_name=self.reports_bucket,
                file_key=f"{self.s3_key_prefix}/{file_name}",
                file_name=f"/tmp/{file_name}",
            )
            return true
       return false

Copy link
Contributor

@abbas-khan10 abbas-khan10 Oct 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bool would serve as a way of customising the log message for each report type as we want to be specific with which report wasn't created (open to discussion obviously) e.g. for the code above:

report_uploaded = write_and_upload_additional_reports(file_name, headers, data_rows)

if not report_uploaded:
    logger.info("No data to report for daily success report file")

Copy link
Contributor Author

@AndyFlintNHS AndyFlintNHS Oct 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As opposed to doing a true/false return, I was going to suggest something more like this, so that it only calls 'write and upload' if it actually needs to:

        if data_rows:
            logger.info("Uploading daily success report file to S3")
            self.write_and_upload_additional_reports(file_name, headers, data_rows)
        else:
            logger.info("No data to report for daily success report file")

    def write_and_upload_additional_reports(self, file_name, headers, data_rows):
        self.write_additional_report_items_to_csv(
            file_name=file_name, headers=headers, rows_to_write=data_rows
        )

        self.s3_service.upload_file(
            s3_bucket_name=self.reports_bucket,
            file_key=f"{self.s3_key_prefix}/{file_name}",
            file_name=f"/tmp/{file_name}",
        )

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic has been broken out into a new function

logger.info("Uploading daily restricted report file to S3")
self.write_and_upload_additional_reports(file_name, headers, data_rows)
else:
logger.info("No data to report for daily deceased report file")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

daily restricted*

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected

@@ -400,3 +375,14 @@ def get_times_for_scan(self) -> tuple[datetime, datetime]:
self.s3_key_prefix = f"bulk-upload-reports/{date_folder_name}"

return start_timestamp, end_timestamp

def write_and_upload_additional_reports(self, file_name, headers, data_rows):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we add param types here please? See above method params for examples

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

abbas-khan10
abbas-khan10 previously approved these changes Oct 29, 2024
Comment on lines +379 to +384
def write_and_upload_additional_reports(
self,
file_name: str,
headers: list[str],
data_rows: list[list[str]],
):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a small unit test for this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed with Abbas - this logic is being tested by each of the daily report unit tests.

@AndyFlintNHS AndyFlintNHS merged commit 8d1dc7d into main Oct 30, 2024
8 checks passed
@AndyFlintNHS AndyFlintNHS deleted the PRMP-1063 branch October 30, 2024 11:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants