Add general_merge tests and a test data generator module #32
Conversation
the generate_data module has functions to create IP networks and addrs and create AS files as we would get from RPKI or RIR.
There was a problem hiding this comment.
Added a few comments. I think adding such a test with generated data is always interesting but it's not where I would have started :) So something I would still like to have is very basic test vectors with static data, primarily because these are very easy to reason about and after we have set it up once it becomes trivial to add all kinds of edge cases we will stumble over. We could have them in CSV file like this one from BIP340: https://github.com/bitcoin/bips/blob/master/bip-0340/test-vectors.csv But let's add this here first since it's already in pretty good shape.
| from random import randint | ||
|
|
||
|
|
||
| def generate_ip(ip_type="v4", subnet_size="16"): |
There was a problem hiding this comment.
nit: The generate_data.py file could be moved into a util or shared folder. But can be done as a follow-up when more tests are added.
| def generate_asns(count): | ||
| asns = [] | ||
| while count > 0: | ||
| asn = randint(1, 100000) |
There was a problem hiding this comment.
Let's use our own max asn value for the end of the range here, i.e. max_encode. Can also use the default for now since we are pretty married to it currently with the assumptions in the core code: 33521664.
| return network | ||
|
|
||
|
|
||
| def generate_ip_networks( |
There was a problem hiding this comment.
Hm, the ip networks (and to some degree asns too) generate methods don't have guarantees that the generated result is a set without overlap, right? While the chance seems low, there could be overlapping results which means the test outcomes are not deterministic. I think it would be neat if these functions could always generate a set. This would then allow us to make stricter testing. I.e. we could be sure that merging two sets of 10 with overlap of 1 have a result of 19 entries.
| """ | ||
| extra_new = [] | ||
| for extra in extra_items: | ||
| included = 0 |
There was a problem hiding this comment.
nit: Could use boolean here
| end_ip = int(ipaddress.IPv4Address("255.255.255.255")) | ||
| subnet_mask = end_ip << (32 - subnet_size) |
There was a problem hiding this comment.
I think this is cleaner:
subnet_mask = ipaddress.IPv4Network(f"0.0.0.0/{subnet_size}", strict=False).netmask
|
|
||
| TEST_FILE_PATHS = { |
There was a problem hiding this comment.
I think the files are not cleaned up after each test run yet, right? should do that to avoid test polution.
Initial approach to tests, starting with the merge process. The generate_data module has functions to create IP networks and addrs and create AS files as we would get from RPKI or RIR.