Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use canonized storage path as tier id #12833

Merged
merged 10 commits into from
Jan 16, 2025

Conversation

swalrus1
Copy link
Collaborator

@swalrus1 swalrus1 commented Dec 20, 2024

Changelog entry

Canonize storage path in CS Tiers

Changelog category

  • Bugfix

Additional information

Один путь до схемного объекта может описываться строками с разным количеством слешей. Чтобы CS корректно идентифицировал тир по пути в произвольном формате, в качестве id тиров используются канонизированные пути.

Copy link

github-actions bot commented Dec 20, 2024

2024-12-20 12:24:51 UTC Pre-commit check linux-x86_64-release-asan for d2ff431 has started.
2024-12-20 12:25:02 UTC Artifacts will be uploaded here
2024-12-20 12:28:11 UTC ya make is running...
🔴 2024-12-20 12:53:44 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Dec 20, 2024

2024-12-20 12:25:32 UTC Pre-commit check linux-x86_64-relwithdebinfo for d2ff431 has started.
2024-12-20 12:25:43 UTC Artifacts will be uploaded here
2024-12-20 12:28:45 UTC ya make is running...
🔴 2024-12-20 12:40:46 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Dec 20, 2024

2024-12-20 18:34:49 UTC Pre-commit check linux-x86_64-relwithdebinfo for 36df012 has started.
2024-12-20 18:35:02 UTC Artifacts will be uploaded here
2024-12-20 18:37:59 UTC ya make is running...
🟡 2024-12-20 19:40:07 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
29504 26519 0 42 2831 112

2024-12-20 19:43:42 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-20 19:55:52 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
234 (only retried tests) 85 0 40 0 109

2024-12-20 19:56:04 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-12-20 20:07:30 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
228 (only retried tests) 80 0 40 0 108

🟢 2024-12-20 20:07:39 UTC Build successful.
🟢 2024-12-20 20:07:59 UTC ydbd size 2.1 GiB changed* by -32.8 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: d2e4bb8 merge: 36df012 diff diff %
ydbd size 2 235 568 584 Bytes 2 235 534 952 Bytes -32.8 KiB -0.002%
ydbd stripped size 477 930 768 Bytes 477 930 256 Bytes -512 Bytes -0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 20, 2024

2024-12-20 18:35:11 UTC Pre-commit check linux-x86_64-release-asan for 36df012 has started.
2024-12-20 18:35:24 UTC Artifacts will be uploaded here
2024-12-20 18:38:28 UTC ya make is running...
🟡 2024-12-20 19:52:12 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12571 12444 0 76 5 46

🟢 2024-12-20 19:53:52 UTC Build successful.
🟢 2024-12-20 19:54:18 UTC ydbd size 3.6 GiB changed* by +9.8 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: d2e4bb8 merge: 36df012 diff diff %
ydbd size 3 866 288 592 Bytes 3 866 298 672 Bytes +9.8 KiB +0.000%
ydbd stripped size 1 350 523 280 Bytes 1 350 541 008 Bytes +17.3 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 22, 2024

2024-12-22 22:14:05 UTC Pre-commit check linux-x86_64-release-asan for 6fc5c69 has started.
2024-12-22 22:14:17 UTC Artifacts will be uploaded here
2024-12-22 22:17:22 UTC ya make is running...
2024-12-22 22:36:32 UTC Check cancelled

Copy link

github-actions bot commented Dec 22, 2024

2024-12-22 22:15:13 UTC Pre-commit check linux-x86_64-relwithdebinfo for 6fc5c69 has started.
2024-12-22 22:15:24 UTC Artifacts will be uploaded here
2024-12-22 22:18:24 UTC ya make is running...
2024-12-22 22:36:32 UTC Check cancelled

Copy link

github-actions bot commented Dec 22, 2024

2024-12-22 22:38:11 UTC Pre-commit check linux-x86_64-relwithdebinfo for 9b01e13 has started.
2024-12-22 22:39:58 UTC Artifacts will be uploaded here
2024-12-22 22:43:36 UTC ya make is running...
🟡 2024-12-22 23:42:47 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
29504 26555 0 1 2831 117

2024-12-22 23:45:11 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-12-22 23:56:46 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
166 (only retried tests) 54 0 0 0 112

🟢 2024-12-22 23:56:53 UTC Build successful.
🟢 2024-12-22 23:57:11 UTC ydbd size 2.1 GiB changed* by -33.5 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: 12789bd merge: 9b01e13 diff diff %
ydbd size 2 235 522 680 Bytes 2 235 488 424 Bytes -33.5 KiB -0.002%
ydbd stripped size 477 900 048 Bytes 477 899 344 Bytes -704 Bytes -0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 22, 2024

2024-12-22 22:40:24 UTC Pre-commit check linux-x86_64-release-asan for 9b01e13 has started.
2024-12-22 22:40:36 UTC Artifacts will be uploaded here
2024-12-22 22:43:35 UTC ya make is running...
🟡 2024-12-22 23:51:20 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12571 12492 0 20 9 50

🟢 2024-12-22 23:52:22 UTC Build successful.
🟢 2024-12-22 23:52:44 UTC ydbd size 3.6 GiB changed* by +4.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 12789bd merge: 9b01e13 diff diff %
ydbd size 3 866 166 992 Bytes 3 866 171 480 Bytes +4.4 KiB +0.000%
ydbd stripped size 1 350 424 112 Bytes 1 350 438 192 Bytes +13.8 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@swalrus1 swalrus1 marked this pull request as ready for review January 9, 2025 12:33
@swalrus1 swalrus1 requested a review from a team as a code owner January 9, 2025 12:33
@swalrus1 swalrus1 linked an issue Jan 15, 2025 that may be closed by this pull request
@swalrus1 swalrus1 requested a review from a team as a code owner January 16, 2025 09:46
Copy link

github-actions bot commented Jan 16, 2025

2025-01-16 09:54:11 UTC Pre-commit check linux-x86_64-relwithdebinfo for 12550da has started.
2025-01-16 09:54:53 UTC Artifacts will be uploaded here
2025-01-16 09:58:18 UTC ya make is running...
🔴 2025-01-16 10:02:53 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Jan 16, 2025

2025-01-16 09:55:30 UTC Pre-commit check linux-x86_64-release-asan for 12550da has started.
2025-01-16 09:55:42 UTC Artifacts will be uploaded here
2025-01-16 09:58:50 UTC ya make is running...
🔴 2025-01-16 10:05:57 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Jan 16, 2025

2025-01-16 10:47:57 UTC Pre-commit check linux-x86_64-relwithdebinfo for 23b458c has started.
2025-01-16 10:48:09 UTC Artifacts will be uploaded here
2025-01-16 10:51:11 UTC ya make is running...
2025-01-16 11:42:44 UTC Check cancelled

Copy link

github-actions bot commented Jan 16, 2025

2025-01-16 10:50:15 UTC Pre-commit check linux-x86_64-release-asan for 23b458c has started.
2025-01-16 10:50:26 UTC Artifacts will be uploaded here
2025-01-16 10:53:36 UTC ya make is running...
2025-01-16 11:42:47 UTC Check cancelled

@github-actions github-actions bot added bugfix and removed bugfix labels Jan 16, 2025
@@ -68,7 +68,7 @@ class TTestEvictionBase {
UNIT_ASSERT_GT(columnRawBytes, 0);
}

TestHelper->SetTiering("/Root/olapStore/olapTable", "/Root/tier1", "timestamp");
TestHelper->SetTiering("/Root/olapStore/olapTable", "Root/tier1", "timestamp");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Хотелось бы тест, который работал проверял работу в двух режима, где ставится "/" и нет.

@@ -72,7 +73,8 @@ class TTierInfo {

TString GetDebugString() const {
TStringBuilder sb;
sb << "name=" << Name << ";duration=" << EvictDuration << ";column=" << EvictColumnName << ";serializer=";
sb << "storage=" << (ExternalStorageId ? ExternalStorageId->GetConfigPath() : "") << ";duration=" << EvictDuration << ";column=" << EvictColumnName
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

По старой логике, как я понимаю, писался бы __DELETE, поэтому хотелось бы сохранить предыдущий вывод

@@ -48,49 +49,53 @@ class TTiersManager: public ITiersManager {
friend class TTierRef;
class TTierRefGuard: public TMoveOnly {
private:
YDB_READONLY_DEF(TString, TierName);
NTiers::TExternalStorageId TierId;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Почему убрал YDB_READONLY_DEF?

Copy link

github-actions bot commented Jan 16, 2025

2025-01-16 11:44:18 UTC Pre-commit check linux-x86_64-release-asan for 597cdab has started.
2025-01-16 11:45:18 UTC Check cancelled

Copy link

github-actions bot commented Jan 16, 2025

2025-01-16 11:44:32 UTC Pre-commit check linux-x86_64-relwithdebinfo for 597cdab has started.
2025-01-16 11:44:45 UTC Artifacts will be uploaded here
2025-01-16 11:45:19 UTC Check cancelled

Copy link

github-actions bot commented Jan 16, 2025

2025-01-16 11:49:03 UTC Pre-commit check linux-x86_64-relwithdebinfo for 2b3fc28 has started.
2025-01-16 11:49:14 UTC Artifacts will be uploaded here
2025-01-16 11:52:19 UTC ya make is running...
🟢 2025-01-16 13:04:28 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
25965 23459 0 0 2380 126

🟢 2025-01-16 13:06:43 UTC Build successful.
🟢 2025-01-16 13:07:01 UTC ydbd size 2.1 GiB changed* by -40.2 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: 1b5955d merge: 2b3fc28 diff diff %
ydbd size 2 252 477 168 Bytes 2 252 436 008 Bytes -40.2 KiB -0.002%
ydbd stripped size 473 399 760 Bytes 473 396 944 Bytes -2.8 KiB -0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jan 16, 2025

2025-01-16 11:49:13 UTC Pre-commit check linux-x86_64-release-asan for 2b3fc28 has started.
2025-01-16 11:49:24 UTC Artifacts will be uploaded here
2025-01-16 11:52:23 UTC ya make is running...
🟡 2025-01-16 13:09:02 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11727 11685 0 15 1 26

2025-01-16 13:10:03 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-01-16 13:22:05 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
75 (only retried tests) 50 0 0 2 23

🟢 2025-01-16 13:22:12 UTC Build successful.
🟢 2025-01-16 13:22:35 UTC ydbd size 3.6 GiB changed* by -21.3 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: 1b5955d merge: 2b3fc28 diff diff %
ydbd size 3 902 828 488 Bytes 3 902 806 640 Bytes -21.3 KiB -0.001%
ydbd stripped size 1 362 513 232 Bytes 1 362 521 872 Bytes +8.4 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@swalrus1 swalrus1 merged commit be05ea4 into ydb-platform:main Jan 16, 2025
12 checks passed
@swalrus1 swalrus1 deleted the canonize-tier-id branch January 16, 2025 13:25
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use canonized path to EDS as tier ID on CS
3 participants