gitlab backup restore

gitlab backup and restore

• backup: https://docs.gitlab.com/omnibus/settings/backups.html
• restore: https://docs.gitlab.com/ce/raketasks/backup_restore.html#restore-for-omnibus-installations

version

备份与恢复，必须保持两个服务的版本完全一致

gitlab-rake gitlab:env:info

backup

backup过程除了application的数据之外，也要备份config&secret数据

restore

issue: restore repository, rake abort!

方法：

1. 删除projects表中，namespace_id不存在于namespaces表中的数据
2. 删除routes表中，source_type='Project'且source_id不存在于projects表中的数据
3. 删除routes表中，source_type='Namespace'且source_id不存在于namespaces表中的数据

于2017-09-05-10-14的操作记录

select (select count(*) from projects) as n_pro, (select count(*) from routes where source_type='Project') as n_rou_pro, (select count(*) from namespaces) as n_name, (select count(*) from routes where source_type='Namespace') as n_rou_name, (select count(*) from routes) as n_rou;

  5739 |      5739 |   4755 |       4769 | 10508

projects表与routes表保持一致，routes多承载了14条多余的namespace数据

1. 删除projects表中，namespace_id不存在于namespaces表中的数据

2. 删除routes表中，source_type='Project'且source_id不存在于projects表中的数据

   select * from projects where namespace_id not in (select id from namespaces);

目前有7条数据，这些数据可以说是无用数据

将其从routes表和projects表一并删除

删除routes

select * from routes where source_type='Project' and source_id in (select id from projects where namespace_id not in (select id from namespaces));

-- 删除要谨慎
delete from routes where source_type='Project' and source_id in (select id from projects where namespace_id not in (select id from namespaces));

删除projects

select * from projects where namespace_id not in (select id from namespaces));

-- 删除要谨慎
delete from projects where namespace_id not in (select id from namespaces));

3. 删除routes表中，source_type='Namespace'且source_id不存在于namespaces表中的数据

   select * from routes where source_type='Namespace' and source_id not in (select id from namespaces);

有14条数据，可以说是无用数据

-- 删除要谨慎
delete from routes where source_type='Namespace' and source_id not in (select id from namespaces);

issue: restore output ERRORS at first

log below

Unpacking backup ... done
Before restoring the database we recommend removing all existing
tables to avoid future upgrade problems. Be aware that if you have
custom tables in the GitLab database these tables and all data will be
removed.
Do you want to continue (yes/no)? Removing all tables. Press `Ctrl-C` within 5 seconds to abort
Cleaning the database ...
done
Restoring database ...
Restoring PostgreSQL database gitlabhq_production ... SET
SET
SET
SET
SET
SET
SET
SET
SET
ERROR:  relation "public.timelogs" does not exist
ERROR:  relation "public.timelogs" does not exist
ERROR:  relation "public.merge_requests_closing_issues" does not exist
ERROR:  relation "public.protected_tag_create_access_levels" does not exist
ERROR:  relation "public.label_priorities" does not exist
ERROR:  relation "public.merge_request_metrics" does not exist
ERROR:  relation "public.label_priorities" does not exist
ERROR:  relation "public.subscriptions" does not exist
ERROR:  relation "public.labels" does not exist
ERROR:  relation "public.u2f_registrations" does not exist
ERROR:  relation "public.protected_tag_create_access_levels" does not exist
ERROR:  relation "public.web_hook_logs" does not exist
ERROR:  relation "public.merge_request_diff_files" does not exist
ERROR:  relation "public.issue_metrics" does not exist
ERROR:  relation "public.merge_requests_closing_issues" does not exist
ERROR:  relation "public.chat_teams" does not exist
ERROR:  relation "public.merge_request_metrics" does not exist
ERROR:  relation "public.container_repositories" does not exist
ERROR:  relation "public.merge_request_diff_commits" does not exist
ERROR:  relation "public.protected_tag_create_access_levels" does not exist
.....
.....

ref: https://gitlab.com/gitlab-org/gitlab-ce/issues/30755

原因在于，在restore之前，先清空了所有的表，导致对relation, index的删除出现了报错。

In sum, the warnings above are just noise because we clean the database before the restore, and the SQL restore doesn't know that.

issue：restore最后一步，会有一个问题，rebuild authorized_keys，continue or not?

ref: https://gitlab.com/gitlab-org/gitlab-ce/issues/16343

this will rebuild authorized_keys file, you will lose data stored in authorized_keys file, want to continue?

choose yes!

git用户的根目录为/var/opt/gitlab，其下有一个.ssh/authorized_keys文件，保存着用户主动保存的公钥

当我们从server-master备份，从server-slave恢复，可能会覆盖server-slave的authorized_keys。从恢复的角度来讲是合理的，保持一致性

sanitize

备份恢复之后，进行完整性检测，有没有什么问题

gitlab-rake gitlab:check SANITIZE=true