Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added generate_app_test_data management command. #809

Open
wants to merge 7 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changes/809.housekeeping
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Added management command `generate_app_test_data` to generate sample data for development environments.
14 changes: 14 additions & 0 deletions docs/dev/dev_environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -482,3 +482,17 @@ invoke generate-app-config-schema
```

This command can only guess the schema, so it's up to the developer to manually update the schema as needed.

### Test Data Generation

To quickly generate test data for developing against this app, you can use the following command:

!!! danger
The `--flush` flag will completely empty your database and replace it with test data. This command should never be run in a production environment.

```bash
nautobot-server generate_app_test_data --flush
nautobot-server createsuperuser
```

This uses the [`generate_test_data`](https://docs.nautobot.com/projects/core/en/stable/user-guide/administration/tools/nautobot-server/#generate_test_data) management command from Nautobot core to generate the Statuses, Platforms, Device Types, Devices, etc. Nautobot version 2.2.0 is the minimum version required for devices to be generated.
113 changes: 113 additions & 0 deletions nautobot_golden_config/management/commands/generate_app_test_data.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to call out that this name should be more specific to the particular app. If another app were to implement the same command for their own testing, they would conflict with each other. While Django handles this, this could cause a different generate_app_test_data run than what we may want.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like for this to be called by core's generate_test_data so hopefully this would move into a separate module and be referenced by NautobotAppConfig. I don't plan on rolling out this management command to every app.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to nautobot-server generate_gc_test_data

Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
"""Generate test data for the Golden Config app."""

import random

from django.core.management import call_command
from django.core.management.base import BaseCommand
from django.db import DEFAULT_DB_ALIAS
from nautobot.core.factory import get_random_instances
from nautobot.dcim.models import Platform
from netutils.lib_mapper import NETUTILSPARSER_LIB_MAPPER_REVERSE

from nautobot_golden_config.models import (
ComplianceFeature,
ComplianceRule,
ConfigCompliance,
GoldenConfig,
)


class Command(BaseCommand):
"""Populate the database with various data as a baseline for testing (automated or manual)."""

help = __doc__

def add_arguments(self, parser): # noqa: D102
parser.add_argument(
"--flush",
action="store_true",
help="Flush any existing data from the database before generating new data.",
)
parser.add_argument(
"--database",
default=DEFAULT_DB_ALIAS,
help='The database to generate the test data in. Defaults to the "default" database.',
)

def _generate_static_data(self):
platforms = get_random_instances(
Platform.objects.filter(devices__isnull=False).distinct(),
minimum=2,
maximum=4,
)
devices = [p.devices.first() for p in platforms]

# Ensure platform has a valid network_driver or compliance generation will fail
for platform in platforms:
if platform.network_driver not in NETUTILSPARSER_LIB_MAPPER_REVERSE:
platform.network_driver = random.choice(list(NETUTILSPARSER_LIB_MAPPER_REVERSE.keys())) # noqa: S311
platform.save()

# Create ComplianceFeatures
compliance_features = []
message = "Creating 8 ComplianceFeatures..."
self.stdout.write(message)
for i in range(1, 9):
name = f"ComplianceFeature{i}"
compliance_features.append(
ComplianceFeature.objects.create(name=name, slug=name, description=f"Test ComplianceFeature {i}")
)

# Create ComplianceRules
count = len(compliance_features) * len(platforms)
message = f"Creating {count} ComplianceRules..."
self.stdout.write(message)
for feature in compliance_features:
for platform in platforms:
ComplianceRule.objects.create(
feature=feature,
platform=platform,
description=f"Test ComplianceRule for {feature.name} on {platform.name}",
match_config=f"match {feature.name} on {platform.name}",
)

# Create ConfigCompliances
count = len(devices) * len(compliance_features)
message = f"Creating {count} ConfigCompliances..."
self.stdout.write(message)
for device in devices:
for rule in ComplianceRule.objects.filter(platform=device.platform):
is_compliant = random.choice([True, False]) # noqa: S311
ConfigCompliance.objects.create(
device=device,
rule=rule,
compliance=is_compliant,
compliance_int=int(is_compliant),
intended=rule.match_config,
actual=rule.match_config if is_compliant else f"mismatch {rule.feature.name}",
)

# Create GoldenConfigs
message = f"Creating {len(devices)} GoldenConfigs..."
self.stdout.write(message)
for device in devices:
GoldenConfig.objects.create(
device=device,
backup_config=f"backup config for {device.name}",
intended_config=f"intended config for {device.name}",
compliance_config=f"compliance config for {device.name}",
)

# TODO: Create ConfigRemoves
# TODO: Create ConfigReplaces
# TODO: Create RemediationSettings
# TODO: Create ConfigPlans

def handle(self, *args, **options):
"""Entry point to the management command."""
# Call nautobot core's generate_test_data command to generate data for core models
call_command("generate_test_data", flush=options["flush"])

self._generate_static_data()

self.stdout.write(self.style.SUCCESS(f"Database {options['database']} populated with app data successfully!"))