Does using Redis for task queuing guarantee transaction safety for custom fields?

[michaelniemand]

We want to use NAutobot to store a machine state in a custom field.

To prevent 2 services updating this custom field at the same time, it needs proper transaction safety.
The docs say: "This is used to provide a concurrent write lock for preventing race conditions when allocating IP address objects"

So does it work for custom fields as well?

[jathanism]

Redis itself does not guarantee this, but the library we are currently using (RQ) for tasks does guarantee order of operations to some degree since each worker can only process a single job at a time.

For the majority of cases, including custom fields, this is not a concern because all database writes happen within transactions, making race conditions for updating the same object at the same time practically impossible.

The concurrent write lock for IP address objects is different. This is because of the feature within Nautobot to allow for a simultaneous request for free address space from a network prefix (e.g. "give me the next 10 available IPs in 192.168.0.0/24..."), and immediate allocation of any number of those free addresses returned by the request (e.g. "... and allocate them"). To do this, the list of IPs returned by the query are immediately written back to the database, creating new IPAddress objects, and therefore removing them from the free pool.

IPAddress objects do not have a uniqueness constraint, so without this explicit write lock, it would be possible for two callers to request IPs from the same block, receive the same free addresses, and attempt to allocate duplicates. This violates the principle of least astonishment, but not the data model, so the Redis write lock here is the key to preserving reasonable expectations of the first caller to receive the lock being the one to allocate the IPs, while the following caller either receiving a correct list, or being unable to allocate the addresses because they are no longer free.

I hope this answers your question!

[michaelniemand]

thank you for the comprehensive answer, appreciate it.

[michaelniemand]

I got some feedback and follow up question though: The state we want to persist using Nautobot must, like IPs, be requested and reserved in one call ("give me 1 free server and reserve it").
Is this possible?