update_docs command is failing

[bmispelon]

Here is the latest log file from update_docs on the production server:

[2024-10-29 08:15:03] Starting update for en/dev at 2024-10-29 03:15:03.252015...
[2024-10-29 08:15:03] HEAD is now at b5669f08bc Improved the writing documentation contributing guide.
[2024-10-29 08:15:04] remote: Enumerating objects: 6, done.
remote: Counting objects: 100% (6/6), done.
[2024-10-29 08:15:04] remote: Total 6 (delta 5), reused 6 (delta 5), pack-reused 0 (from 0)
Unpacking objects: 100% (6/6), 598 bytes | 119.00 KiB/s, done.
[2024-10-29 08:15:04] From https://github.com/django/django
[2024-10-29 08:15:04]    555f2412cb..738e0601d5  main       -> origin/main
[2024-10-29 08:15:04] Updating b5669f08bc..738e0601d5
[2024-10-29 08:15:04] Fast-forward
[2024-10-29 08:15:04]  django/contrib/admin/templates/admin/auth/user/add_form.html | 2 +-
[2024-10-29 08:15:04]  django/core/cache/backends/filebased.py                      | 2 +-
[2024-10-29 08:15:04]  docs/ref/forms/api.txt                                       | 4 ++--
[2024-10-29 08:15:04]  3 files changed, 4 insertions(+), 4 deletions(-)
[2024-10-29 08:15:04]   building json (/home/www/www/data/docbuilds/sources/dev/docs -> /home/www/www/data/docbuilds/en/dev/_build/json)
[2024-10-29 08:15:41]   building djangohtml (/home/www/www/data/docbuilds/sources/dev/docs -> /home/www/www/data/docbuilds/en/dev/_build/djangohtml)
[2024-10-29 08:16:18]   build zip (into /home/www/www/media/docs/django-docs-dev-en.zip)
[2024-10-29 08:16:19]   reindexing...
[2024-10-29 08:16:20] Traceback (most recent call last):
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/backends/utils.py", line 89, in _execute
[2024-10-29 08:16:20]     return self.cursor.execute(sql, params)
[2024-10-29 08:16:20]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20] psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "docs_document_release_id_31f73b05439839fa_uniq"
[2024-10-29 08:16:20] DETAIL:  Key (release_id, path)=(1, contents) already exists.
[2024-10-29 08:16:20] 
[2024-10-29 08:16:20] 
[2024-10-29 08:16:20] The above exception was the direct cause of the following exception:
[2024-10-29 08:16:20] 
[2024-10-29 08:16:20] Traceback (most recent call last):
[2024-10-29 08:16:20]   File "<frozen runpy>", line 198, in _run_module_as_main
[2024-10-29 08:16:20]   File "<frozen runpy>", line 88, in _run_code
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/__main__.py", line 9, in <module>
[2024-10-29 08:16:20]     management.execute_from_command_line()
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
[2024-10-29 08:16:20]     utility.execute()
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
[2024-10-29 08:16:20]     self.fetch_command(subcommand).run_from_argv(self.argv)
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/core/management/base.py", line 412, in run_from_argv
[2024-10-29 08:16:20]     self.execute(*args, **cmd_options)
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/core/management/base.py", line 458, in execute
[2024-10-29 08:16:20]     output = self.handle(*args, **options)
[2024-10-29 08:16:20]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20]   File "/usr/src/app/docs/management/commands/update_docs.py", line 85, in handle
[2024-10-29 08:16:20]     self.build_doc_release(release, force=kwargs["force"])
[2024-10-29 08:16:20]   File "/usr/src/app/docs/management/commands/update_docs.py", line 257, in build_doc_release
[2024-10-29 08:16:20]     release.sync_to_db(documents)
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/contextlib.py", line 81, in inner
[2024-10-29 08:16:20]     return func(*args, **kwds)
[2024-10-29 08:16:20]            ^^^^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20]   File "/usr/src/app/docs/models.py", line 188, in sync_to_db
[2024-10-29 08:16:20]     Document.objects.create(
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
[2024-10-29 08:16:20]     return getattr(self.get_queryset(), name)(*args, **kwargs)
[2024-10-29 08:16:20]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/models/query.py", line 658, in create
[2024-10-29 08:16:20]     obj.save(force_insert=True, using=self.db)
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/models/base.py", line 814, in save
[2024-10-29 08:16:20]     self.save_base(
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/models/base.py", line 877, in save_base
[2024-10-29 08:16:20]     updated = self._save_table(
[2024-10-29 08:16:20]               ^^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/models/base.py", line 1020, in _save_table
[2024-10-29 08:16:20]     results = self._do_insert(
[2024-10-29 08:16:20]               ^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/models/base.py", line 1061, in _do_insert
[2024-10-29 08:16:20]     return manager._insert(
[2024-10-29 08:16:20]            ^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
[2024-10-29 08:16:20]     return getattr(self.get_queryset(), name)(*args, **kwargs)
[2024-10-29 08:16:20]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/models/query.py", line 1805, in _insert
[2024-10-29 08:16:20]     return query.get_compiler(using=using).execute_sql(returning_fields)
[2024-10-29 08:16:20]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/models/sql/compiler.py", line 1822, in execute_sql
[2024-10-29 08:16:20]     cursor.execute(sql, params)
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/sentry_sdk/utils.py", line 1761, in runner
[2024-10-29 08:16:20]     return sentry_patched_function(*args, **kwargs)
[2024-10-29 08:16:20]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/sentry_sdk/integrations/django/__init__.py", line 651, in execute
[2024-10-29 08:16:20]     result = real_execute(self, sql, params)
[2024-10-29 08:16:20]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/backends/utils.py", line 67, in execute
[2024-10-29 08:16:20]     return self._execute_with_wrappers(
[2024-10-29 08:16:20]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
[2024-10-29 08:16:20]     return executor(sql, params, many, context)
[2024-10-29 08:16:20]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django_read_only/__init__.py", line 79, in blocker
[2024-10-29 08:16:20]     return execute(sql, params, many, context)
[2024-10-29 08:16:20]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/backends/utils.py", line 84, in _execute
[2024-10-29 08:16:20]     with self.db.wrap_database_errors:
[2024-10-29 08:16:20]          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/utils.py", line 91, in __exit__
[2024-10-29 08:16:20]     raise dj_exc_value.with_traceback(traceback) from exc_value
[2024-10-29 08:16:20]   File "/usr/local/lib/python3.12/site-packages/django/db/backends/utils.py", line 89, in _execute
[2024-10-29 08:16:20]     return self.cursor.execute(sql, params)
[2024-10-29 08:16:20]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-29 08:16:20] django.db.utils.IntegrityError: duplicate key value violates unique constraint "docs_document_release_id_31f73b05439839fa_uniq"
[2024-10-29 08:16:20] DETAIL:  Key (release_id, path)=(1, contents) already exists.
[2024-10-29 08:16:20] 

[pauloxnet]

After loading all doc release [1] locally, I'm running $ time python -m manage update_docs -v 2 on my local copy of the main branch. I'll report to you if it breaks.

[1] #1693 (comment)

[pauloxnet]

The update_docs run with any errors with this timing.

real    15m53,591s
user    76m27,003s
sys     2m14,644s

Can be possible that the issue in the production server is related to the database instance in production?

[marksweb]

I'm not seeing any issues building locally.

python -m manage update_docs -v 2  414.02s user 28.75s system 285% cpu 2:35.11 total