Enable patch releases by fetching some components from existing distribution. #789
Conversation
dblock
force-pushed
the
build-from-dist
branch
2 times, most recently
from
ad14bd8 to
c1f48bd
Compare
dblock
changed the title
dblock
force-pushed
the
build-from-dist
branch
from
c1f48bd to
21cd158
Compare
dblock
force-pushed
the
build-from-dist
branch
from
21cd158 to
25e0584
Compare
|
@saratvemulapalli @peternied Bump, this is ready for review please. |
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
dblock
force-pushed
the
build-from-dist
branch
from
99a4ae3 to
6b38d06
Compare
Codecov Report
@@ Coverage Diff @@
## main #789 +/- ##
==========================================
- Coverage 91.60% 91.18% -0.42%
==========================================
Files 81 87 +6
Lines 2250 2370 +120
==========================================
+ Hits 2061 2161 +100
- Misses 189 209 +20
Continue to review full report at Codecov.
|
| - 1.1.0 | ||
| components: | ||
| - name: OpenSearch | ||
| dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405 |
There was a problem hiding this comment.
The path will need to be inferred/discovered which will have been broken with the recent url restructuring that only impacted the 1.2.0+ builds. Should we add parameters to help the build system construct the correct path to the manifest?
There was a problem hiding this comment.
Either in a separate PR or maybe we should just copy the manifest with bad URLs to a corrected copy in S3 and use that instead.
There was a problem hiding this comment.
The manifest is oriented around the components of with a platform / architecture, do we need another kind of manifest that is oriented around a releases artifacts. Note, we can move this out to another PR
There was a problem hiding this comment.
I don't see this as necessary. This is a bundle (assembly/distribution) manifest that can now include references to another bundle, and picks up components with matching platform/architecture.
| def build(self, build_recorder): | ||
| pass | ||
|
|
||
| def export_artifacts(self, build_recorder): |
There was a problem hiding this comment.
export doesn't seem to align with the action what about import or download + "_artifacts"?
There was a problem hiding this comment.
I thought about this. The "from source" code exports those artifacts, while the "from distribution" code downloads and exports those artifacts. They should really be spit up into download_artifacts and export_artifacts. I have another PR with more changes along those lines if that's OK?
| - 1.1.0 | ||
| components: | ||
| - name: OpenSearch | ||
| dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405 |
There was a problem hiding this comment.
The manifest is oriented around the components of with a platform / architecture, do we need another kind of manifest that is oriented around a releases artifacts. Note, we can move this out to another PR
Signed-off-by: dblock dblock@amazon.com
Description
This allows build to produce a mixed distribution with multiple versions of components, e.g. 1.1.0 + some components built from source = 1.1.1.
In the provided example this build fails trying to build
anomaly-detection, which expects OpenSearch core 1.1.1, but we're giving it 1.1.0. The builds for those patched components will need to pickup the released dependency from maven explicitly.Issues Resolved
Closes #373.
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.