Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle invalid domains more gracefully #53

Open
wants to merge 4 commits into
base: develop
Choose a base branch
from

Conversation

S4lt5
Copy link

@S4lt5 S4lt5 commented Nov 21, 2022

🗣 Description

As per #42, invalid domains should be handled gracefully.

These are checked for in main() before running the executor, and put in a separate "invalid domains" bucket.

Also, as part of code cleanup, the domain class was moved to a separate python file.

There is a LOT to do and a lot of redundant call chains to be removed, but each journey starts with a single step!

💭 Motivation and context

🧪 Testing

Added unit tests for valid/invalid cdnCheck() and main() calls with valid/invalid domains

Sample output

findcdn list "*.domain.com"
0 Domains Validated.
1 domain(s) rejected, listed below:
	*.domain.com
0it [00:00, ?it/s]
{
    "date": "11/21/2022, 11:44:09",
    "cdn_count": "0",
    "domains": {},
    "invalid_domains": [
        "*.domain.com"
    ]
}
Domain processing completed.
0 domains had CDN's out of 1.
1 Domains Validated.
2 domain(s) rejected, listed below:
	*.foo.bar
	wildcard domain
[Pending: 0 jobs]==[Threads: 1]: 100%|████████████████████████████████████████████████████| 1/1 [00:01<00:00,  1.26s/it]
{
    "date": "11/21/2022, 11:44:53",
    "cdn_count": "1",
    "domains": {
        "cisa.gov": {
            "IP": "'104.117.51.217'",
            "cdns": "'.edgekey.net', '.akamaitechnologies.fr'",
            "cdns_by_names": "'Akamai', 'Akamai'"
        }
    },
    "invalid_domains": [
        "*.foo.bar",
        "wildcard domain"
    ]
}
Domain processing completed.
1 domains had CDN's out of 3.

✅ Pre-approval checklist

  • This PR has an informative and human-readable title.
  • Changes are limited to a single goal - eschew scope creep!
  • All future TODOs are captured in issues, which are referenced
    in code comments.
  • All relevant type-of-change labels have been added.
  • I have read the CONTRIBUTING document.
  • These code changes follow cisagov code standards.
  • All relevant repo and/or project documentation has been updated
    to reflect the changes in this PR.
  • Tests have been added and/or modified to cover the changes in this PR.
  • All new and existing tests pass.

✅ Pre-merge checklist

  • Revert dependencies to default branches.
  • Finalize version.

✅ Post-merge checklist

  • Add a tag or create a release.

@lgtm-com
Copy link

lgtm-com bot commented Nov 21, 2022

This pull request introduces 1 alert when merging b3b320d into 7bbf50e - view on LGTM.com

new alerts:

  • 1 for Unused import

Heads-up: LGTM.com's PR analysis will be disabled on the 5th of December, and LGTM.com will be shut down ⏻ completely on the 16th of December 2022. Please enable GitHub code scanning, which uses the same CodeQL engine ⚙️ that powers LGTM.com. For more information, please check out our post on the GitHub blog.

Copy link
Collaborator

@Pascal-0x90 Pascal-0x90 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the changes. I think this is a great addition. Going to say this should be merged on the note that I am also going to be working more of the PRs too so this may also end up being rearranged. I want to get rid of the complexity you keep pointing out because I am also noticing that as well.

Copy link
Collaborator

@Pascal-0x90 Pascal-0x90 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing I forgot to mention, please make sure you run pre-commit hooks before merging so the tests pass. Looks like pre-commit hooks had some issues.

@S4lt5
Copy link
Author

S4lt5 commented Nov 23, 2022

I think those "test_list_broken/est_file_broken" tests that fail, were failing before. I actually don't even understand the tests to be honest.

I'll double check with the other branches.

@Pascal-0x90
Copy link
Collaborator

Good point. The tests should probably get a good review as well while I am on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants