NetBox Version
v4.6.5
Python Version
3.12
Area(s) of Concern
Observations
Summary
Querying ip_address_list over GraphQL with assigned_object (and nested fields like device { site, location, device_type }) issues roughly 8 SQL queries per row instead of a handful of batched queries. On ~5,600 IP addresses in prod, this takes over 2 minutes and ~51000 SQL statements.
Reproduction
query {
ip_address_list(filters: {assigned: true}) {
address
assigned_object {
... on InterfaceType {
device {
site { name }
location { name }
device_type { manufacturer { name } }
primary_ip4 { id }
contacts { contact { name } priority }
}
}
}
}
}
Counted via django.db.connection.execute_wrapper on v4.6.5 (CaptureQueriesContext
caps at 9000 queries, too low to observe the full blowup here). Breakdown by table
for ~5,600 rows:
| Table |
Queries |
ipam_ipaddress (assigned_object_type/id refetch + device.primary_ip4) |
16,898 |
dcim_manufacturer |
5,786 |
dcim_device |
5,642 |
dcim_site |
5,642 |
dcim_devicetype |
5,642 |
tenancy_contactassignment |
5,642 |
dcim_location |
5,635 |
| other (module/module_type/etc.) |
~300 |
Total: 51,191. Expected: a small constant independent of row count.
Root cause
IPAddressType.assigned_object (ipam/graphql/types.py) resolves a
GenericForeignKey:
@strawberry_django.field(prefetch_related='assigned_object')
def assigned_object(self) -> ...:
return self.assigned_object
Two problems:
assigned_object_type/assigned_object_id are excluded from the schema (exclude=[...] on the type), so the optimizer defers those columns — but the GFK descriptor still needs them, triggering a per-row refetch. ConfigContextMixin already works around this identical issue with only=['local_context_data']; assigned_object needs only=['assigned_object_type', 'assigned_object_id'].
prefetch_related='assigned_object' batches the GFK itself per content type, but can't customize per-type querysets, so nested fields (device.site, device.device_type.manufacturer, device.contacts, etc.) aren't covered and resolve one query per row. The REST API's IPAddressViewSet already solves this with GenericPrefetch, supplying a select_related-customized queryset per type — the GraphQL type needs the same, extended one level deeper to cover commonly requested device fields.
Proposed Changes
Proposed fix
Override get_queryset using GenericPrefetch, gated on assigned_object being selected, plus only=[...] on the field itself:
@classmethod
def get_queryset(cls, queryset, info: Info, **kwargs):
queryset = super().get_queryset(queryset, info, **kwargs)
selected = {f.name for f in info.selected_fields[0].selections}
if 'assigned_object' in selected:
return queryset.prefetch_related(
GenericPrefetch(
'assigned_object',
[
models.FHRPGroup.objects.all(),
Interface.objects.select_related('cable', 'device'),
VMInterface.objects.select_related('virtual_machine'),
],
),
)
return queryset
@strawberry_django.field(only=['assigned_object_type', 'assigned_object_id'])
def assigned_object(self) -> ...:
return self.assigned_object
Measured on the same ~5,600-row query: 51,191 → 34,264 queries (−33%), ~20s → ~15s. Isolating the two changes shows they address distinct per-row costs and compound rather than overlap:
| Variant |
Queries |
| Baseline (unpatched) |
51,191 |
only=[...] alone |
45,546 |
GenericPrefetch alone |
45,550 |
| Both combined |
34,264 |
The dcim_device row disappears entirely (5,642 → 0) once assigned_object is prefetched with select_related('device'), and the ipam_ipaddress refetch drops from 16,898 to 5,612. Further nested fields (device.site, device.device_type, device.contacts, etc.) remain N+1, same as today.
In my particular use, it would be helpful to prefetch even more fields (like device.site, device.device_type, and device.contacts), but I left those out because that's not a common query.
I have a working patch and can open a PR.
NetBox Version
v4.6.5
Python Version
3.12
Area(s) of Concern
Observations
Summary
Querying
ip_address_listover GraphQL withassigned_object(and nested fields likedevice { site, location, device_type }) issues roughly 8 SQL queries per row instead of a handful of batched queries. On ~5,600 IP addresses in prod, this takes over 2 minutes and ~51000 SQL statements.Reproduction
Counted via
django.db.connection.execute_wrapperon v4.6.5 (CaptureQueriesContextcaps at 9000 queries, too low to observe the full blowup here). Breakdown by table
for ~5,600 rows:
ipam_ipaddress(assigned_object_type/id refetch + device.primary_ip4)dcim_manufacturerdcim_devicedcim_sitedcim_devicetypetenancy_contactassignmentdcim_locationTotal: 51,191. Expected: a small constant independent of row count.
Root cause
IPAddressType.assigned_object(ipam/graphql/types.py) resolves aGenericForeignKey:Two problems:
assigned_object_type/assigned_object_idare excluded from the schema (exclude=[...]on the type), so the optimizer defers those columns — but the GFK descriptor still needs them, triggering a per-row refetch.ConfigContextMixinalready works around this identical issue withonly=['local_context_data'];assigned_objectneedsonly=['assigned_object_type', 'assigned_object_id'].prefetch_related='assigned_object'batches the GFK itself per content type, but can't customize per-type querysets, so nested fields (device.site,device.device_type.manufacturer,device.contacts, etc.) aren't covered and resolve one query per row. The REST API'sIPAddressViewSetalready solves this withGenericPrefetch, supplying aselect_related-customized queryset per type — the GraphQL type needs the same, extended one level deeper to cover commonly requested device fields.Proposed Changes
Proposed fix
Override
get_querysetusingGenericPrefetch, gated onassigned_objectbeing selected, plusonly=[...]on the field itself:Measured on the same ~5,600-row query: 51,191 → 34,264 queries (−33%), ~20s → ~15s. Isolating the two changes shows they address distinct per-row costs and compound rather than overlap:
only=[...]aloneGenericPrefetchaloneThe
dcim_devicerow disappears entirely (5,642 → 0) onceassigned_objectis prefetched withselect_related('device'), and theipam_ipaddressrefetch drops from 16,898 to 5,612. Further nested fields (device.site,device.device_type,device.contacts, etc.) remain N+1, same as today.In my particular use, it would be helpful to prefetch even more fields (like device.site, device.device_type, and device.contacts), but I left those out because that's not a common query.
I have a working patch and can open a PR.