⚡️ Disable pagination when requesting form definitions in the admin

[Viicos]

Fixes #3491

I'm a bit frustrated by this issue, as what I wanted to do initially was:

• Be able to specify specific fields to include via query parameters, e.g. path?include=url,name,internal_name (as only these three fields are required by the NewStepFormDefinitionPicker component)
• Disable pagination in some way

First one would be doable with https://github.com/wimglenn/djangorestframework-queryfields but it would add an extra dependency not updated in 7 years.

Or another solution suggested by @joeribekker:

• Define a custom endpoint (e.g. via @action) to only fetch the required fields, but it would not inherit from the currently defined filterset_class.

I ended up using a custom pagination class, that would understand page_size=0 to mean no pagination. We will have to see if this is viable with a lot of form definitions in the db

[Viicos]

I think we can get rid of this?

open-forms/src/openforms/forms/api/filters.py

Lines 34 to 42 in cc8fe70

	def filter_queryset(self, queryset):
	"""
	Override base implementation to add OR-behaviour instead of the standard AND.
	See https://github.com/carltongibson/django-filter/discussions/1426
	"""
	_or = Q()
	for name in self.Meta.or_fields:

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (dd45e7f) 95.92% compared to head (91a4dd4) 95.92%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3542   +/-   ##
=======================================
  Coverage   95.92%   95.92%           
=======================================
  Files         670      670           
  Lines       21527    21529    +2     
  Branches     2485     2485           
=======================================
+ Hits        20649    20651    +2     
  Misses        609      609           
  Partials      269      269           

Files	Coverage Δ
src/openforms/forms/api/filters.py	94.44% <100.00%> (+0.32%)	⬆️
src/openforms/forms/api/viewsets.py	99.14% <ø> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sergei-maertens]

I think we can get rid of this?

open-forms/src/openforms/forms/api/filters.py

Lines 34 to 42 in cc8fe70

	def filter_queryset(self, queryset):
	"""
	Override base implementation to add OR-behaviour instead of the standard AND.
	See https://github.com/carltongibson/django-filter/discussions/1426
	"""
	_or = Q()
	for name in self.Meta.or_fields:

we can deprecate it, but removing it is a breaking change and requires a new major version of the API

[Viicos]

Couldn't get e2e tests to work. My best guess (and I was able to reproduce only once locally but unfortunately the timeout closed the test browser) is that this happens when form definitions are loaded, and the e2e test clicks on the select element on a precise timing where the select options HTML elements don't have time to load, which kind of breaks React's reactivity.

Edit: I'm able to reproduce ~ 1/5th of the time, but can't really do much as console isn't opened on startup and I can't see the requests made.

Dirty solution: page.wait_for_timeout

[sergei-maertens]

Can you also deprecate that filter_queryset thing doing the OR filter? A warnings.warn(..., DeprecationWarning) is fine & add it to the list of things to remove in #3283

[Viicos]

Can you also deprecate that filter_queryset thing doing the OR filter? A warnings.warn(..., DeprecationWarning) is fine & add it to the list of things to remove in #3283

Shouldn't we deprecate the OpenAPI query parameters instead? Or you only want the OR filter to have a warnings.warn to not forgot removing it when #3283 gets tackled?

[sergei-maertens]

Can you also deprecate that filter_queryset thing doing the OR filter? A warnings.warn(..., DeprecationWarning) is fine & add it to the list of things to remove in #3283

Shouldn't we deprecate the OpenAPI query parameters instead? Or you only want the OR filter to have a warnings.warn to not forgot removing it when #3283 gets tackled?

Only the OR filter behaviour, the query params themselves are obviously still used, just not at the same time (anymore).

[sergei-maertens]

Please rebase your PR in a couple of atomic commits :) then it's good to merge!