Skip to content

GraphQL: IPAddressType.assigned_object causes N+1 queries #22787

Description

@llamafilm

NetBox Version

v4.6.5

Python Version

3.12

Area(s) of Concern

  • User Interface
  • REST API
  • GraphQL API
  • Python ORM
  • Other

Observations

Summary

Querying ip_address_list over GraphQL with assigned_object (and nested fields like device { site, location, device_type }) issues roughly 8 SQL queries per row instead of a handful of batched queries. On ~5,600 IP addresses in prod, this takes over 2 minutes and ~51000 SQL statements.

Reproduction

query {
  ip_address_list(filters: {assigned: true}) {
    address
    assigned_object {
      ... on InterfaceType {
        device {
          site { name }
          location { name }
          device_type { manufacturer { name } }
          primary_ip4 { id }
          contacts { contact { name } priority }
        }
      }
    }
  }
}

Counted via django.db.connection.execute_wrapper on v4.6.5 (CaptureQueriesContext
caps at 9000 queries, too low to observe the full blowup here). Breakdown by table
for ~5,600 rows:

Table Queries
ipam_ipaddress (assigned_object_type/id refetch + device.primary_ip4) 16,898
dcim_manufacturer 5,786
dcim_device 5,642
dcim_site 5,642
dcim_devicetype 5,642
tenancy_contactassignment 5,642
dcim_location 5,635
other (module/module_type/etc.) ~300

Total: 51,191. Expected: a small constant independent of row count.

Root cause

IPAddressType.assigned_object (ipam/graphql/types.py) resolves a
GenericForeignKey:

@strawberry_django.field(prefetch_related='assigned_object')
def assigned_object(self) -> ...:
    return self.assigned_object

Two problems:

  1. assigned_object_type/assigned_object_id are excluded from the schema (exclude=[...] on the type), so the optimizer defers those columns — but the GFK descriptor still needs them, triggering a per-row refetch. ConfigContextMixin already works around this identical issue with only=['local_context_data']; assigned_object needs only=['assigned_object_type', 'assigned_object_id'].
  2. prefetch_related='assigned_object' batches the GFK itself per content type, but can't customize per-type querysets, so nested fields (device.site, device.device_type.manufacturer, device.contacts, etc.) aren't covered and resolve one query per row. The REST API's IPAddressViewSet already solves this with GenericPrefetch, supplying a select_related-customized queryset per type — the GraphQL type needs the same, extended one level deeper to cover commonly requested device fields.

Proposed Changes

Proposed fix

Override get_queryset using GenericPrefetch, gated on assigned_object being selected, plus only=[...] on the field itself:

@classmethod
def get_queryset(cls, queryset, info: Info, **kwargs):
    queryset = super().get_queryset(queryset, info, **kwargs)
    selected = {f.name for f in info.selected_fields[0].selections}
    if 'assigned_object' in selected:
        return queryset.prefetch_related(
            GenericPrefetch(
                'assigned_object',
                [
                    models.FHRPGroup.objects.all(),
                    Interface.objects.select_related('cable', 'device'),
                    VMInterface.objects.select_related('virtual_machine'),
                ],
            ),
        )
    return queryset

@strawberry_django.field(only=['assigned_object_type', 'assigned_object_id'])
def assigned_object(self) -> ...:
    return self.assigned_object

Measured on the same ~5,600-row query: 51,191 → 34,264 queries (−33%), ~20s → ~15s. Isolating the two changes shows they address distinct per-row costs and compound rather than overlap:

Variant Queries
Baseline (unpatched) 51,191
only=[...] alone 45,546
GenericPrefetch alone 45,550
Both combined 34,264

The dcim_device row disappears entirely (5,642 → 0) once assigned_object is prefetched with select_related('device'), and the ipam_ipaddress refetch drops from 16,898 to 5,612. Further nested fields (device.site, device.device_type, device.contacts, etc.) remain N+1, same as today.

In my particular use, it would be helpful to prefetch even more fields (like device.site, device.device_type, and device.contacts), but I left those out because that's not a common query.

I have a working patch and can open a PR.

Metadata

Metadata

Assignees

Labels

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions