ocp13 replacement

[RobHooper]

Resolves #494 and #504.

[jpmckinney]

I have noted that scrapyd needs a project configuring, on ocp13 this is configured under the datlab user (/home/datlab/.config/scrapy.cfg).

@RobHooper Is it that you observed that e.g. collect.data.open-contracting.org lists no projects? That's because we need to do something like scrapyd-deploy registry from our local computers. Not sure why Datlab has that config.

• We can document this on a new page for the registry.

[jpmckinney]

I will enable mod_md again when DNS is live (sending too many failed requests, as we are at the moment, causes Lets Encrypt to block us).

Yeah, I have a note about that in the docs https://ocdsdeploy.readthedocs.io/en/latest/develop/update/apache.html

Let’s Encrypt will reach a Failed Validation limit if DNS is not propagated.

I've now also added:

In the meantime, you can use Let’s Encrypt’s staging environment.

[jpmckinney]

• An additional manual step is to copy the exchange_rates table from the pelican_backend database. There are docs here, for the data support server.
• We also need to run the migrations for Kingfisher Process and Pelican frontend, like at https://ocdsdeploy.readthedocs.io/en/latest/deploy/servers/data-support.html#docker-apps
• Upgrade ocp13 (registry) to Ubuntu 24.04 LTS #494 mentions copying over /home/collect/scrapyd/logs

---

James when you have a moment, please can you confirm what data needs migrating to the new server from /data/.

332M	/data/storage/spoonbill

Yes, we also need to move /data/storage/spoonbill (looks like you already have).

It contains the Django media files for that project. It has a var subdirectory which is not referenced by anything and that has no changes since 2021, so I deleted it on both the old and new server (I assume it was from an old version).

---

Alongside this work, we can remove state ntp disabling the NTP service. This was only required for older systems (Ubuntu 20.04).
https://github.com/open-contracting/deploy/blob/main/salt/core/systemd/ntp.sls#L31-L34

• @RobHooper Did you want to do this, or you decided not to?

---

For the data support server, we have:

Adjust reserved disk space to 1% for large disks:

tune2fs -m 1 /dev/md2

• Do we want to do this for the registry server?

[jpmckinney]

In terms of migration process, is this the plan?

• Do we adjust the TTLs to speed up the DNS switchover?
• ocp13: Remove/comment out DATA_REGISTRY_CBOM so that new jobs aren't started
• ocp13: Delete /etc/cron.d/postgres_backups
• ocp27: We should re-copy any new files (data/storage/exporter) and databases from ocp13
• ocp27: Can we copy the mod_md files for data.open-contracting.org from ocp13? If so, we want to also restore the Apache conf files and reload Apache.
• GoDaddy: Change the DNS over for data.open-contracting.org
• Run scrapyd-deploy once DNS has propagated
• Deploy ocp27 with Salt, to install the deployer's crontab and the postgres_backups
• When satisfied, decommission ocp13

[jpmckinney]

Looks good! Just one comment.

[jpmckinney]

Can you add a comment about why this option is necessary?

[RobHooper]

This is required on systems with Python 3.11 and newer.
I have added a comment and a Jinja if statement so it only runs where needed.

[jpmckinney]

Is this flag also needed on the pip.installed state in kingfisher/init.sls?

[jpmckinney]

I added a documentation page that covers most of what I wrote above (except where I still have a question). Please add the steps to restore the data_registry and spoonbill databases from backups.

We should add a subheading on this other page, and explain in general how to restore from individual database backups versus the existing docs about pgBackrest backups: https://ocdsdeploy.readthedocs.io/en/latest/maintain/databases.html#restore-from-backup We can then just link to that from the registry page.

[RobHooper]

In the meantime, you can use Let’s Encrypt’s staging environment.

The Lets Encrypt staging environment won't generate SSL certificates without live DNS, so this doesn't help in this situation.

@RobHooper Did you want to [remove the disable NTP state], or you decided not to?

This was already done in a previous PR :)

Do we want [modify reserved disk bytes] for the registry server?

Yes, I have ran tune2fs -m 1 /dev/md2 now.
I think we should run this by default on any disk larger than ~50GBs.
Where would be best to document this?

[jpmckinney]

Where would be best to document this?

Probably at an appropriate step in create_server.rst, and maybe as a note at https://ocdsdeploy.readthedocs.io/en/latest/maintain/hosting.html#rescale-a-server

[jpmckinney]

The Lets Encrypt staging environment won't generate SSL certificates without live DNS, so this doesn't help in this situation.

It helps to not get temporarily blocked by Let's Encrypt :)