[Feature/Identity] Merge remote-tracking branch 'origin/main' into fe…

…ature/identity (#5581)

* Fix flaky ShardIndexingPressureConcurrentExecutionTests (#5439)

Add conditional check on assertNull to fix flaky tests.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Fix bwc for cluster manager throttling settings (#5305)

Signed-off-by: Dhwanil Patel <dhwanip@amazon.com>

* Update ingest-attachment plugin dependencies: Apache Tika 3.6.0, Apache Mime4j 0.8.8, Apache Poi 5.2.3, Apache PdfBox 2.0.27 (#5448)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Enhance CheckpointState to support no-op replication (#5282)

* CheckpointState enhanced to support no-op replication

Signed-off-by: Ashish Singh <ssashish@amazon.com>
Co-authored-by: Bukhtawar Khan<bukhtawa@amazon.com>

* [BUG] org.opensearch.repositories.s3.RepositoryS3ClientYamlTestSuiteIT/test {yaml=repository_s3/20_repository_permanent_credentials/Snapshot and Restore with repository-s3 using permanent credentials} flaky: randomizing basePath (#5482)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* [Bug] fix case sensitivity for wildcard queries (#5462)

Fixes the wildcard query to not normalize the pattern when case_insensitive is
set by the user. This is achieved by creating a new normalizedWildcardQuery
method so that query_string queries (which do not support case sensitivity) can
still normalize the pattern when the default analyzer is used; maintaining
existing behavior.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* Support OpenSSL Provider with default Netty allocator (#5460)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Revert "build no-jdk distributions as part of release build (#4902)" (#5465)

This reverts commit 8c9ca4e.

It seems that this wasn't entirely the correct way and is currently
blocking us from removing the `build.sh` from the `opensearch-build`
repository (i.e. this `build.sh` here is not yet being used).
See the discussion in opensearch-project/opensearch-build#2835 for
further details.

Signed-off-by: Ralph Ursprung <Ralph.Ursprung@avaloq.com>

Signed-off-by: Ralph Ursprung <Ralph.Ursprung@avaloq.com>

* Add max_shard_size parameter for Shrink API (fix supported version after backport) (#5503)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Sync CODEOWNERS with MAINTAINERS. (#5501)

Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>

Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>

* Added jackson dependency to server (#5366)

* Added jackson dependency to server

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Updated CHANGELOG

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Update build.gradle files

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Add RuntimePermission to fix errors

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Fix flaky test BulkIntegrationIT.testDeleteIndexWhileIndexing (#5491)

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

* Add release notes for 2.4.1 (#5488)

Signed-off-by: Xue Zhou <xuezhou@amazon.com>

Signed-off-by: Xue Zhou <xuezhou@amazon.com>

* Properly skip OnDemandBlockSnapshotIndexInputTests.testVariousBlockSize on Windows. (#5511)

PR #5397 skipped this test in @before block but still
frequently throws a TestCouldNotBeSkippedException.  This is caused by the after block still executing and throwing  an exception
while cleaning the directory created at the path in @before.  Moving the assumption to the individual test prevents this exception by ensuring the path exists.

Signed-off-by: Marc Handalian <handalm@amazon.com>

Signed-off-by: Marc Handalian <handalm@amazon.com>

* Merge first batch of feature/extensions into main (#5347)

* Merge first batch of feature/extensions into main

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Fixed CHANGELOG

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Fixed newline errors

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Renaming and CHANGELOG fixes

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Refactor extension loading into private method

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Removed skipValidation and added connectToExtensionNode method

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Remove unnecessary feature flag calls

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Renaming and exception handling

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Change latches to CompletableFuture

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Removed unnecessary validateSettingKey call

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Fix azure-core dependency

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Update SHAs

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Remove unintended dependency changes

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Removed dynamic settings regitration, removed info() method, and added NoopExtensionsManager

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Add javadoc

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Fixed spotless failure

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Removed NoopExtensionsManager

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Added functioning NoopExtensionsManager

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Added missing javadoc

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Remove forbiddenAPI

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Fix spotless

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Change logger.info to logger.error in handleException

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Fix ExtensionsManagerTests

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Removing unrelated change

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Update SHAs

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Bump commons-compress from 1.21 to 1.22 (#5520)

Bumps commons-compress from 1.21 to 1.22.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-compress
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [Segment Replication] Trigger a round of replication for replica shards during peer recovery when segment replication is enabled (#5332)

* Fix new added replica shards falling behind primary.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Trigger a round of replication during peer recovery when segment replication is enabled.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Remove unnecessary start replication overloaded method.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Add test for failure case and refactor some code.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Apply spotless check.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Addressing comments on the PR.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Remove unnecessary condition check.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Apply spotless check.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Add step listeners to resolve forcing round of segment replication.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Adding support to register settings dynamically (#5495)

* Adding support to register settings dynamically

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Update CHANGELOG

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Removed unnecessary registerSetting methods

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Change setting registration order

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Add unregisterSettings method

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Remove unnecessary feature flag

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Updated 1.3.7 release notes date (#5536)

Signed-off-by: owaiskazi19 <owaiskazi19@gmail.com>

Signed-off-by: owaiskazi19 <owaiskazi19@gmail.com>

* Pre conditions check before updating weighted routing metadata (#4955)

* Pre conditions check to allow weight updates for non decommissioned attribute

Signed-off-by: Rishab Nahata <rnnahata@amazon.com>

* Atomically update cluster state with decommission status and corresponding action (#5093)

* Atomically update the cluster state with decommission status and its corresponding action in the same execute call

Signed-off-by: Rishab Nahata <rnnahata@amazon.com>

* Update Netty to 4.1.86.Final (#5529)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Update release date in 2.4.1 release notes (#5549)

Signed-off-by: Suraj Singh <surajrider@gmail.com>

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Update 2.4.1 release notes (#5552)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Refactor fuzziness interface on query builders (#5433)

* Refactor Object to Fuzziness type for all query builders

Signed-off-by: noCharger <lingzhichu.clz@gmail.com>

* Revise on bwc

Signed-off-by: noCharger <lingzhichu.clz@gmail.com>

* Update change log

Signed-off-by: noCharger <lingzhichu.clz@gmail.com>

Signed-off-by: noCharger <lingzhichu.clz@gmail.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>

* Upgrade lucene version (#5570)

* Added bwc version 2.4.2

Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>

* Added 2.4.2.

Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>

* Update Lucene snapshot to 9.5.0-snapshot-d5cef1c

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Update changelog entry

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Add 2.4.2 bwc version

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Internal changes post lucene upgrade

Signed-off-by: Suraj Singh <surajrider@gmail.com>

Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Suraj Singh <surajrider@gmail.com>
Co-authored-by: opensearch-ci-bot <opensearch-ci-bot@users.noreply.github.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>

* Add CI bundle pattern to distribution download (#5348)

* Add CI bundle pattern for ivy repo

Signed-off-by: Zelin Hao <zelinhao@amazon.com>

* Gradle update

Signed-off-by: Zelin Hao <zelinhao@amazon.com>

* Extract path

Signed-off-by: Zelin Hao <zelinhao@amazon.com>

* Change with customDistributionDownloadType

Signed-off-by: Zelin Hao <zelinhao@amazon.com>

* Add default for exception handle

Signed-off-by: Zelin Hao <zelinhao@amazon.com>

* Add documentations

Signed-off-by: Zelin Hao <zelinhao@amazon.com>

Signed-off-by: Zelin Hao <zelinhao@amazon.com>

* Bump protobuf-java from 3.21.9 to 3.21.11 in /plugins/repository-hdfs (#5519)

* Bump protobuf-java from 3.21.9 to 3.21.11 in /plugins/repository-hdfs

Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.21.9 to 3.21.11.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py)
- [Commits](protocolbuffers/protobuf@v3.21.9...v3.21.11)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Updated changelog

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Owais Kazi <owaiskazi19@gmail.com>
Co-authored-by: Suraj Singh <surajrider@gmail.com>

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
Signed-off-by: Dhwanil Patel <dhwanip@amazon.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Signed-off-by: Ralph Ursprung <Ralph.Ursprung@avaloq.com>
Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Ryan Bogan <rbogan@amazon.com>
Signed-off-by: Poojita Raj <poojiraj@amazon.com>
Signed-off-by: Xue Zhou <xuezhou@amazon.com>
Signed-off-by: Marc Handalian <handalm@amazon.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: owaiskazi19 <owaiskazi19@gmail.com>
Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
Signed-off-by: Suraj Singh <surajrider@gmail.com>
Signed-off-by: noCharger <lingzhichu.clz@gmail.com>
Signed-off-by: Zelin Hao <zelinhao@amazon.com>
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
Co-authored-by: Rishikesh Pasham <62345295+Rishikesh1159@users.noreply.github.com>
Co-authored-by: Dhwanil Patel <dhwanip@amazon.com>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: Ashish <ssashish@amazon.com>
Co-authored-by: Nick Knize <nknize@apache.org>
Co-authored-by: Ralph Ursprung <39383228+rursprung@users.noreply.github.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: Ryan Bogan <10944539+ryanbogan@users.noreply.github.com>
Co-authored-by: Poojita Raj <poojiraj@amazon.com>
Co-authored-by: Xue Zhou <85715413+xuezhou25@users.noreply.github.com>
Co-authored-by: Marc Handalian <handalm@amazon.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Owais Kazi <owaiskazi19@gmail.com>
Co-authored-by: Rishab Nahata <rnnahata@amazon.com>
Co-authored-by: Suraj Singh <surajrider@gmail.com>
Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com>
Co-authored-by: opensearch-ci-bot <opensearch-ci-bot@users.noreply.github.com>
Co-authored-by: Zelin Hao <87548827+zelinh@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

.ci/bwcVersions

@@ -11,4 +11,5 @@ BWC_VERSION:
   - "2.3.1"
   - "2.4.0"
   - "2.4.1"
+  - "2.4.2"
   - "2.5.0"

.github/CODEOWNERS

@@ -1,3 +1 @@
-# This should match the owning team set up in https://github.com/orgs/opensearch-project/teams
-*   @opensearch-project/opensearch-core @reta
-
+*   @reta @anasalkouz @andrross @reta @Bukhtawar @CEHENKLE @dblock @setiah @kartg @kotwanikunal @mch2 @nknize @owaiskazi19 @adnapibar @Rishikesh1159 @ryanbogan @saratvemulapalli @shwetathareja @dreamer-89 @tlfeng @VachaShah @xuezhou25

MAINTAINERS.md

@@ -5,7 +5,6 @@
 
 | Maintainer | GitHub ID | Affiliation |
 | --------------- | --------- | ----------- |
-| Abbas Hussain | [abbashus](https://github.com/abbashus) | Amazon |
 | Anas Alkouz | [anasalkouz](https://github.com/anasalkouz) | Amazon |
 | Andrew Ross   | [andrross](https://github.com/andrross)| Amazon |
 | Andriy Redko | [reta](https://github.com/reta) | Aiven |
@@ -22,8 +21,8 @@
 | Rishikesh Pasham | [Rishikesh1159](https://github.com/Rishikesh1159) | Amazon|
 | Ryan Bogan | [ryanbogan](https://github.com/ryanbogan) | Amazon |
 | Sarat Vemulapalli | [saratvemulapalli](https://github.com/saratvemulapalli) | Amazon |
-| Shweta Thareja |[shwetathareja](https://github.com/shwetathareja) | Amazon |
-| Suraj Singh |[dreamer-89](https://github.com/dreamer-89) | Amazon |
+| Shweta Thareja | [shwetathareja](https://github.com/shwetathareja) | Amazon |
+| Suraj Singh | [dreamer-89](https://github.com/dreamer-89) | Amazon |
 | Tianli Feng | [tlfeng](https://github.com/tlfeng) | Amazon |
 | Vacha Shah | [VachaShah](https://github.com/VachaShah) | Amazon |
 | Xue Zhou | [xuezhou25](https://github.com/xuezhou25) | Amazon |
@@ -32,6 +31,7 @@
 
 | Maintainer | GitHub ID | Affiliation |
 | --------------- | --------- | ----------- |
+| Abbas Hussain | [abbashus](https://github.com/abbashus) | Amazon |
 | Megha Sai Kavikondala | [meghasaik](https://github.com/meghasaik) | Amazon |
 
 [This document](https://github.com/opensearch-project/.github/blob/main/MAINTAINERS.md) explains what maintainers do in this repo, and how they should be doing it. If you're interested in contributing, see [CONTRIBUTING](CONTRIBUTING.md).

TESTING.md

@@ -383,6 +383,10 @@ Use -Dtest.class and -Dtests.method to run a specific bwcTest test. For example
      -Dtests.class=org.opensearch.upgrades.RecoveryIT \
      -Dtests.method=testHistoryUUIDIsGenerated
 
+Use `-PcustomDistributionDownloadType=bundle` to run the bwcTest against the test cluster with latest CI distribution bundle set up for the specified version; this property is default to min and exclusive choices between `bundle` and `min`:
+
+    ./gradlew bwcTest -PcustomDistributionDownloadType=bundle
+
 When running `./gradlew check`, minimal bwc checks are also run against compatible versions that are not yet released.
 
 ## BWC Testing against a specific remote/branch

buildSrc/build.gradle

@@ -103,7 +103,7 @@ dependencies {
   api localGroovy()
 
   api 'commons-codec:commons-codec:1.15'
-  api 'org.apache.commons:commons-compress:1.21'
+  api 'org.apache.commons:commons-compress:1.22'
   api 'org.apache.ant:ant:1.10.12'
   api 'com.netflix.nebula:gradle-extra-configurations-plugin:8.0.0'
   api 'com.netflix.nebula:nebula-publishing-plugin:4.6.0'

buildSrc/src/main/java/org/opensearch/gradle/DistributionDownloadPlugin.java

@@ -73,6 +73,8 @@ public class DistributionDownloadPlugin implements Plugin<Project> {
     private static final String RELEASE_PATTERN_LAYOUT = "/core/opensearch/[revision]/[module]-min-[revision](-[classifier]).[ext]";
     private static final String SNAPSHOT_PATTERN_LAYOUT =
         "/snapshots/core/opensearch/[revision]/[module]-min-[revision](-[classifier])-latest.[ext]";
+    private static final String BUNDLE_PATTERN_LAYOUT =
+        "/ci/dbc/distribution-build-opensearch/[revision]/latest/linux/x64/tar/dist/opensearch/[module]-[revision](-[classifier]).[ext]";
 
     private NamedDomainObjectContainer<OpenSearchDistribution> distributionsContainer;
     private NamedDomainObjectContainer<DistributionResolution> distributionsResolutionStrategiesContainer;
@@ -174,20 +176,39 @@ private static void setupDownloadServiceRepo(Project project) {
             return;
         }
         Object customDistributionUrl = project.findProperty("customDistributionUrl");
-        // checks if custom Distribution Url has been passed by user from plugins
+        Object customDistributionDownloadType = project.findProperty("customDistributionDownloadType");
+        // distributionDownloadType is default min if is not specified; download the distribution from CI if is bundle
+        String distributionDownloadType = customDistributionDownloadType != null
+            && customDistributionDownloadType.toString().equals("bundle") ? "bundle" : "min";
         if (customDistributionUrl != null) {
             addIvyRepo(project, DOWNLOAD_REPO_NAME, customDistributionUrl.toString(), FAKE_IVY_GROUP, "");
             addIvyRepo(project, SNAPSHOT_REPO_NAME, customDistributionUrl.toString(), FAKE_SNAPSHOT_IVY_GROUP, "");
-        } else {
-            addIvyRepo(
-                project,
-                DOWNLOAD_REPO_NAME,
-                "https://artifacts.opensearch.org",
-                FAKE_IVY_GROUP,
-                "/releases" + RELEASE_PATTERN_LAYOUT,
-                "/release-candidates" + RELEASE_PATTERN_LAYOUT
-            );
-            addIvyRepo(project, SNAPSHOT_REPO_NAME, "https://artifacts.opensearch.org", FAKE_SNAPSHOT_IVY_GROUP, SNAPSHOT_PATTERN_LAYOUT);
+            return;
+        }
+        switch (distributionDownloadType) {
+            case "bundle":
+                addIvyRepo(project, DOWNLOAD_REPO_NAME, "https://ci.opensearch.org", FAKE_IVY_GROUP, BUNDLE_PATTERN_LAYOUT);
+                addIvyRepo(project, SNAPSHOT_REPO_NAME, "https://ci.opensearch.org", FAKE_SNAPSHOT_IVY_GROUP, BUNDLE_PATTERN_LAYOUT);
+                break;
+            case "min":
+                addIvyRepo(
+                    project,
+                    DOWNLOAD_REPO_NAME,
+                    "https://artifacts.opensearch.org",
+                    FAKE_IVY_GROUP,
+                    "/releases" + RELEASE_PATTERN_LAYOUT,
+                    "/release-candidates" + RELEASE_PATTERN_LAYOUT
+                );
+                addIvyRepo(
+                    project,
+                    SNAPSHOT_REPO_NAME,
+                    "https://artifacts.opensearch.org",
+                    FAKE_SNAPSHOT_IVY_GROUP,
+                    SNAPSHOT_PATTERN_LAYOUT
+                );
+                break;
+            default:
+                throw new IllegalArgumentException("Unsupported property argument: " + distributionDownloadType);
         }
     }
 

buildSrc/version.properties

@@ -1,5 +1,5 @@
 opensearch        = 3.0.0
-lucene            = 9.5.0-snapshot-a4ef70f
+lucene            = 9.5.0-snapshot-d5cef1c
 
 bundled_jdk_vendor = adoptium
 bundled_jdk = 19.0.1+10
@@ -24,7 +24,7 @@ kotlin            = 1.7.10
 # when updating the JNA version, also update the version in buildSrc/build.gradle
 jna               = 5.12.1
 
-netty             = 4.1.84.Final
+netty             = 4.1.86.Final
 joda              = 2.10.13
 
 # client dependencies

modules/ingest-geoip/build.gradle

@@ -41,8 +41,6 @@ opensearchplugin {
 dependencies {
   api('com.maxmind.geoip2:geoip2:3.0.2')
   // geoip2 dependencies:
-  api("com.fasterxml.jackson.core:jackson-annotations:${versions.jackson}")
-  api("com.fasterxml.jackson.core:jackson-databind:${versions.jackson_databind}")
   api('com.maxmind.db:maxmind-db:2.1.0')
 
   testImplementation 'org.elasticsearch:geolite2-databases:20191119'

modules/lang-expression/licenses/lucene-expressions-9.5.0-snapshot-d5cef1c.jar.sha1

@@ -0,0 +1 @@
+afe877ebf5ec4be7d17636b695015c449a523a3b

modules/transport-netty4/build.gradle

@@ -125,7 +125,8 @@ thirdPartyAudit {
     'com.aayushatharva.brotli4j.Brotli4jLoader',
     'com.aayushatharva.brotli4j.decoder.DecoderJNI$Status',
     'com.aayushatharva.brotli4j.decoder.DecoderJNI$Wrapper',
-    'com.aayushatharva.brotli4j.encoder.Encoders',
+    'com.aayushatharva.brotli4j.encoder.BrotliEncoderChannel',
+    'com.aayushatharva.brotli4j.encoder.Encoder',
     'com.aayushatharva.brotli4j.encoder.Encoder$Mode',
     'com.aayushatharva.brotli4j.encoder.Encoder$Parameters',
     // classes are missing

modules/transport-netty4/licenses/netty-buffer-4.1.86.Final.jar.sha1

@@ -0,0 +1 @@
+a66fa0ed2687eb33a2e53a17a6df61bfe3b3f2bd

modules/transport-netty4/licenses/netty-codec-4.1.86.Final.jar.sha1

@@ -0,0 +1 @@
+ee126da926ea202da3b21eb737788ef83b1db772

modules/transport-netty4/licenses/netty-codec-http-4.1.86.Final.jar.sha1

@@ -0,0 +1 @@
+23674593f004959ae002ec348626eecf677191ae

modules/transport-netty4/licenses/netty-codec-http2-4.1.86.Final.jar.sha1

@@ -0,0 +1 @@
+ac0ca067e4118533ad1038776fcd9d5f3058b7d4

modules/transport-netty4/licenses/netty-common-4.1.86.Final.jar.sha1

@@ -0,0 +1 @@
+1dceab4662a9cc93faf87b237bb41103b1bc7f0e

modules/transport-netty4/licenses/netty-handler-4.1.86.Final.jar.sha1

@@ -0,0 +1 @@
+0bcb65230218286e6456b5d085cb42e67776eb70

modules/transport-netty4/licenses/netty-resolver-4.1.86.Final.jar.sha1

@@ -0,0 +1 @@
+bad83d479f7bd8ea84eefd77c316435be4c97270

modules/transport-netty4/licenses/netty-transport-4.1.86.Final.jar.sha1

@@ -0,0 +1 @@
+7c7739c41fd110c3576e9faace332ee957f27203

modules/transport-netty4/licenses/netty-transport-native-unix-common-4.1.86.Final.jar.sha1

@@ -0,0 +1 @@
+5e0e7fc1c337485cabcf7971faefe692b76f93a2

modules/transport-netty4/src/main/java/org/opensearch/transport/Netty4NioServerSocketChannel.java

@@ -0,0 +1,62 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.transport;
+
+import io.netty.channel.socket.InternetProtocolFamily;
+import io.netty.channel.socket.nio.NioServerSocketChannel;
+import io.netty.util.internal.SocketUtils;
+import io.netty.util.internal.logging.InternalLogger;
+import io.netty.util.internal.logging.InternalLoggerFactory;
+
+import java.nio.channels.ServerSocketChannel;
+import java.nio.channels.SocketChannel;
+import java.nio.channels.spi.SelectorProvider;
+import java.util.List;
+
+public class Netty4NioServerSocketChannel extends NioServerSocketChannel {
+    private static final InternalLogger logger = InternalLoggerFactory.getInstance(Netty4NioServerSocketChannel.class);
+
+    public Netty4NioServerSocketChannel() {
+        super();
+    }
+
+    public Netty4NioServerSocketChannel(SelectorProvider provider) {
+        super(provider);
+    }
+
+    public Netty4NioServerSocketChannel(SelectorProvider provider, InternetProtocolFamily family) {
+        super(provider, family);
+    }
+
+    public Netty4NioServerSocketChannel(ServerSocketChannel channel) {
+        super(channel);
+    }
+
+    @Override
+    protected int doReadMessages(List<Object> buf) throws Exception {
+        SocketChannel ch = SocketUtils.accept(javaChannel());
+
+        try {
+            if (ch != null) {
+                buf.add(new Netty4NioSocketChannel(this, ch));
+                return 1;
+            }
+        } catch (Throwable t) {
+            logger.warn("Failed to create a new channel from an accepted socket.", t);
+
+            try {
+                ch.close();
+            } catch (Throwable t2) {
+                logger.warn("Failed to close a socket.", t2);
+            }
+        }
+
+        return 0;
+    }
+}

modules/transport-netty4/src/main/java/org/opensearch/transport/NettyAllocator.java

@@ -39,7 +39,6 @@
 import io.netty.buffer.UnpooledByteBufAllocator;
 import io.netty.channel.Channel;
 import io.netty.channel.ServerChannel;
-import io.netty.channel.socket.nio.NioServerSocketChannel;
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 import org.opensearch.common.Booleans;
@@ -181,7 +180,7 @@ public static Class<? extends ServerChannel> getServerChannelType() {
         if (ALLOCATOR instanceof NoDirectBuffers) {
             return CopyBytesServerSocketChannel.class;
         } else {
-            return NioServerSocketChannel.class;
+            return Netty4NioServerSocketChannel.class;
         }
     }
 

plugins/analysis-icu/licenses/lucene-analysis-icu-9.5.0-snapshot-d5cef1c.jar.sha1

@@ -0,0 +1 @@
+670d8f48ea9cba542e263d3ec6c3e2a33accc561

plugins/analysis-kuromoji/licenses/lucene-analysis-kuromoji-9.5.0-snapshot-d5cef1c.jar.sha1

@@ -0,0 +1 @@
+b1f42bad26470c8ef88096e0c8564a74223c52c9

plugins/analysis-nori/licenses/lucene-analysis-nori-9.5.0-snapshot-d5cef1c.jar.sha1

@@ -0,0 +1 @@
+f3cf74fa91da5133667f8916f93071fed231f2ee

plugins/analysis-phonetic/licenses/lucene-analysis-phonetic-9.5.0-snapshot-d5cef1c.jar.sha1

@@ -0,0 +1 @@
+e633c2d0cd677e4f1cef5aadc6bdc65e8e898d98

plugins/analysis-smartcn/licenses/lucene-analysis-smartcn-9.5.0-snapshot-d5cef1c.jar.sha1

@@ -0,0 +1 @@
+c12a2943e6f4977f15d489ac3e9802c5dfb3c4cc

plugins/analysis-stempel/licenses/lucene-analysis-stempel-9.5.0-snapshot-d5cef1c.jar.sha1

@@ -0,0 +1 @@
+5d257928a34e586a7de9fc7d4a013868f7a1db74

plugins/analysis-ukrainian/licenses/lucene-analysis-morfologik-9.5.0-snapshot-d5cef1c.jar.sha1

@@ -0,0 +1 @@
+3d88f80ad07421b9470cb44a6f5b67dd47047b13

plugins/discovery-ec2/build.gradle

@@ -46,8 +46,6 @@ dependencies {
   api "commons-logging:commons-logging:${versions.commonslogging}"
   api "org.apache.logging.log4j:log4j-1.2-api:${versions.log4j}"
   api "commons-codec:commons-codec:${versions.commonscodec}"
-  api "com.fasterxml.jackson.core:jackson-databind:${versions.jackson_databind}"
-  api "com.fasterxml.jackson.core:jackson-annotations:${versions.jackson}"
 }
 
 restResources {

plugins/ingest-attachment/build.gradle

@@ -38,10 +38,10 @@ opensearchplugin {
 }
 
 versions << [
-  'tika'  : '2.5.0',
-  'pdfbox': '2.0.25',
-  'poi'   : '5.2.2',
-  'mime4j': '0.8.3'
+  'tika'  : '2.6.0',
+  'pdfbox': '2.0.27',
+  'poi'   : '5.2.3',
+  'mime4j': '0.8.8'
 ]
 
 dependencies {

plugins/ingest-attachment/licenses/apache-mime4j-core-0.8.8.jar.sha1

@@ -0,0 +1 @@
+7330de23c52f71617cbec7f1d2760dae32e687cd

plugins/ingest-attachment/licenses/apache-mime4j-dom-0.8.8.jar.sha1

@@ -0,0 +1 @@
+e76715563a6bd150f84ccb0adb920aec8faf4779

plugins/ingest-attachment/licenses/fontbox-2.0.27.jar.sha1

@@ -0,0 +1 @@
+d08c064d18b2b149da937d15c0d1708cba03f29d

plugins/ingest-attachment/licenses/pdfbox-2.0.27.jar.sha1

@@ -0,0 +1 @@
+416a9dfce3714116bfdf793b15368df04266845f

plugins/ingest-attachment/licenses/poi-5.2.3.jar.sha1

@@ -0,0 +1 @@
+2fb22ae74ad5aea6af1a9c64b9542f2ccf348604

plugins/ingest-attachment/licenses/poi-ooxml-5.2.3.jar.sha1

@@ -0,0 +1 @@
+02efd11c940adb18c03eb9ce7ad88fc40ee6a196