Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Device Type Components - Module Bays and Console Ports #100

Open
kennethdammyr opened this issue Dec 5, 2023 · 13 comments
Open

Add Device Type Components - Module Bays and Console Ports #100

kennethdammyr opened this issue Dec 5, 2023 · 13 comments

Comments

@kennethdammyr
Copy link

Hi

When updating a device with the script add_device_type_components.py, both interfaces, module bays and console ports are populated. However, the tab menues for module bays and console ports does not get updated with the new data. In this screenshot, you may see how the device has got 17 module bays, but only showing 1 (which was there from before) in the menu:
image

If there are no module bays or console ports, the tab menu item is not displayed. Hence you will not have the oportunity to use them. As shown here:
image

Interfaces works as expected.
Any idea what could cause this?

Netbox v3.6.6

@BriacD
Copy link

BriacD commented Jan 29, 2024

I just noticed this issue with PDUs and adding Power Outlets. Have you come across a solution? Script works great and does exactly what I've been needing but this is making it hard to move forward with all the changes I need to make.

@UniverseSandbox
Copy link

Same Issue here. Does someone have a solution?

@BriacD
Copy link

BriacD commented Mar 8, 2024

I upgraded to the latest version and that fixed my issue.

@ip-rx
Copy link

ip-rx commented Mar 25, 2024

This is also affecting me. v3.7.4

Same issue. Ran script and created missing components (Interfaces, Console Port, PSU Module bays). Only Interfaces are accessible directly via the device's page. To find Console or PSU Mod Bays, I have to search them up in components list.

Not breaking, but certainly an annoyance.

With console port from script only
image

Added a dummy console port
image

@ip-rx
Copy link

ip-rx commented Mar 28, 2024

@candlerb found a fix and already submitted a PR for it

#101

my team has tested this and it works.

@ip-rx
Copy link

ip-rx commented Mar 28, 2024

The next issue with this bug:

Any devices that were created and affected by this bug, will still have an incorrect count of objects.

device.consoleports.count() (actual) and device.console_port_count (meta) will be inconsistent. The fix to the script calls a .save() which engages the function utilities.counters.update_counter(), but the bug caused device.console_port_count to be permanently wrong.

I threw together a script that just calcs the delta and feeds the inputs into utilities.counters.update_counter(). You can run it against any device pretty safely as it gets the literal count of each component and feeds the correct delta (positive or negative integer) into update_counter()

This logic could easily be inserted into the main script as a way to fix the issue retroactively when ran on a device.

"""
This script runs a counter check on each device's components and updates discrepancies
"""

from dcim.models import Manufacturer, DeviceType, Device
from extras.scripts import Script, ObjectVar, MultiObjectVar
from utilities.counters import update_counter


class FixDeviceComponentCounts(Script):
    class Meta:
        name = "Fix Device Component Counts"
        description = (
            "run a counter check on each device's components and updates discrepancies"
        )

    manufacturer = ObjectVar(
        model=Manufacturer,
        required=False,
    )
    device_type = ObjectVar(
        model=DeviceType,
        query_params={
            "manufacturer_id": "$manufacturer",
        },
        required=False,
    )
    devices = MultiObjectVar(
        model=Device,
        query_params={
            "device_type_id": "$device_type",
        },
    )

    def run(self, data, commit):
        for device in data["devices"]:
            device.full_clean()
            for counter_name, count_attr_name in [
                ("console_port_count", "consoleports"),
                ("console_server_port_count", "consoleserverports"),
                ("power_port_count", "powerports"),
                ("power_outlet_count", "poweroutlets"),
                ("interfaces_count", "interfaces"),
                ("rear_port_count", "rearports"),
                ("front_port_count", "frontports"),
                ("device_bay_count", "devicebays"),
                ("module_bay_count", "modulebays"),
            ]:
                actual_attr_count = getattr(device, count_attr_name).count()
                expected_attr_count = getattr(device, counter_name)
                # delta = device.consoleports.count() - device.console_port_count
                delta = actual_attr_count - expected_attr_count
                if delta != 0:
                    update_counter(device._meta.model, device.pk, counter_name, delta)

                    self.log_success(
                        "%s (%d): updated %s from %d to %d"
                        % (
                            device.name,
                            device.id,
                            counter_name,
                            expected_attr_count,
                            actual_attr_count,
                        )
                    )

Device with incorrect counts
image

Execution of script above
image

Same device with corrected counts
image

@kkthxbye-code
Copy link
Contributor

@ip-rx - there's a management command for that.

https://github.com/netbox-community/netbox/blob/develop/netbox/utilities/management/commands/calculate_cached_counts.py

@candlerb
Copy link
Contributor

candlerb commented Mar 28, 2024

That's the right answer, or more succinctly:

/opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py calculate_cached_counts

will fix the problem.

@ip-rx
Copy link

ip-rx commented Mar 28, 2024

Even better, thanks!

@jburgon
Copy link

jburgon commented Sep 20, 2024

When I run this script I get an error about the module bays. Anyone know what this might mean? If I comment out the line for module bays then it works. Obviously module bays don't get applied but everything else does.

`An exception occurred: IntegrityError: null value in column "level" of relation "dcim_modulebay" violates not-null constraint DETAIL: Failing row contains (2024-09-20 18:40:03.531782+00, 2024-09-20 18:40:03.531805+00, {}, 261, PIM1, PIM00000001, Mini-PIM 1, PIM1, , 118, null, null, null, null, null, null).

Traceback (most recent call last):
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute
return self.cursor.execute(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib64/python3.11/site-packages/psycopg/cursor.py", line 97, in execute
raise ex.with_traceback(None)
psycopg.errors.NotNullViolation: null value in column "level" of relation "dcim_modulebay" violates not-null constraint
DETAIL: Failing row contains (2024-09-20 18:40:03.531782+00, 2024-09-20 18:40:03.531805+00, {}, 261, PIM1, PIM00000001, Mini-PIM 1, PIM1, , 118, null, null, null, null, null, null).

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/netbox/netbox/extras/jobs.py", line 45, in run_script
script.output = script.run(data, commit)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/netbox/scripts/add_device_type_components.py", line 66, in run
klass.objects.bulk_create(items)
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/query.py", line 835, in bulk_create
returned_columns = self._batched_insert(
^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/query.py", line 1875, in _batched_insert
self._insert(
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/query.py", line 1847, in _insert
return query.get_compiler(using=using).execute_sql(returning_fields)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/sql/compiler.py", line 1823, in execute_sql
cursor.execute(sql, params)
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute
return self._execute_with_wrappers(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
return executor(sql, params, many, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute
with self.db.wrap_database_errors:
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/utils.py", line 91, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute
return self.cursor.execute(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib64/python3.11/site-packages/psycopg/cursor.py", line 97, in execute
raise ex.with_traceback(None)
django.db.utils.IntegrityError: null value in column "level" of relation "dcim_modulebay" violates not-null constraint
DETAIL: Failing row contains (2024-09-20 18:40:03.531782+00, 2024-09-20 18:40:03.531805+00, {}, 261, PIM1, PIM00000001, Mini-PIM 1, PIM1, , 118, null, null, null, null, null, null).`

@candlerb
Copy link
Contributor

I guess it's because the module model changed in Netbox v4.1.0 to allow nested (hierarchical) modules.

@joekripal
Copy link

I'm running into the same error as @jburgon. I tried to piece together a fix don't seem to be able to.

@rizlas
Copy link

rizlas commented Oct 30, 2024

@ip-rx - there's a management command for that.

https://github.com/netbox-community/netbox/blob/develop/netbox/utilities/management/commands/calculate_cached_counts.py

That's the right answer, or more succinctly:

/opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py calculate_cached_counts

will fix the problem.

Dunno where if you found this, but you are legend. Thank you!

Just for ref another issue: netbox-community/netbox#14446

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants