Skip to content

Commit

Permalink
Merge branch 'main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
clun authored Jun 14, 2024
2 parents c4a0b93 + 25e4d6d commit f9c8fe3
Show file tree
Hide file tree
Showing 597 changed files with 23,265 additions and 5,529 deletions.
42 changes: 33 additions & 9 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,45 @@
<!-- Thank you so much for your contribution! -->
<!-- Please fill in all the sections below. -->

<!-- Please open the PR as a draft initially. Once it is reviewed and approved, we will ask you to add documentation and examples. -->
<!-- Please note that PRs with breaking changes will be rejected. -->
<!-- Please note that PRs without tests will be rejected. -->

## Context
<!-- Please provide some context so that it is clear why this change is required. -->
<!-- Please note that PRs will be reviewed based on the priority of the issues they address. -->
<!-- We ask for your patience. We are doing our best to review your PR as quickly as possible. -->
<!-- Please refrain from pinging and asking when it will be reviewed. Thank you for understanding! -->


## Issue
<!-- Please paste the link to the issue this PR is addressing. For example: https://github.com/langchain4j/langchain4j/issues/1012 -->


## Change
<!-- Please describe the changed you made. -->
<!-- Please describe the changes you made. -->


## Checklist
Before submitting this PR, please check the following points:
## General checklist
<!-- Please double-check the following points and mark them like this: [X] -->
- [ ] There are no breaking changes
- [ ] I have added unit and integration tests for my change
- [ ] All unit and integration tests in the module I have added/changed are green
- [ ] All unit and integration tests in the [core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core) and [main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j) modules are green
- [ ] I have manually run all the unit and integration tests in the module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the [core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core) and [main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j) modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait until the PR is reviewed and approved. -->
- [ ] I have added/updated the [documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples repo](https://github.com/langchain4j/langchain4j-examples) (only for "big" features)
- [ ] I have added my new module in the [BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml) (only when a new module is added)
- [ ] I have added/updated [Spring Boot starter(s)](https://github.com/langchain4j/langchain4j-spring) (if applicable)

## Checklist for adding new model integration
<!-- Please double-check the following points and mark them like this: [X] -->
- [ ] I have added my new module in the [BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for adding new embedding store integration
- [ ] I have added a {NameOfIntegration}EmbeddingStoreIT that extends from either EmbeddingStoreIT or EmbeddingStoreWithFilteringIT
<!-- Please double-check the following points and mark them like this: [X] -->
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [ ] I have added my new module in the [BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this: [X] -->
- [ ] I have manually verified that the `{NameOfIntegration}EmbeddingStore` works correctly with the data persisted using the latest released version of LangChain4j
77 changes: 77 additions & 0 deletions .github/workflows/nightly.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
name: Nightly Build

on:
schedule:
- cron: '0 0 * * *' # daily at midnight UTC
workflow_dispatch:

jobs:
java_build:
strategy:
matrix:
java_version: [ 8, 11, 17, 21 ]
include:
- java_version: '8'
included_modules: '-pl !langchain4j-local-ai,!langchain4j-milvus,!code-execution-engines/langchain4j-code-execution-engine-graalvm-polyglot,!langchain4j-cassandra,!langchain4j-infinispan,!langchain4j-neo4j,!langchain4j-opensearch,!langchain4j-azure-ai-search'
- java_version: '11'
included_modules: '-pl !langchain4j-local-ai,!langchain4j-milvus,!code-execution-engines/langchain4j-code-execution-engine-graalvm-polyglot,!langchain4j-infinispan,!langchain4j-neo4j'
- java_version: '17'
included_modules: '-pl !langchain4j-local-ai,!langchain4j-milvus,!code-execution-engines/langchain4j-code-execution-engine-graalvm-polyglot'
- java_version: '21'
included_modules: '-pl !langchain4j-local-ai,!langchain4j-milvus'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up JDK ${{ matrix.java_version }}
uses: actions/setup-java@v4
with:
java-version: ${{ matrix.java_version }}
distribution: 'temurin'
cache: 'maven'

- name: Authenticate to Google Cloud
# Needed for langchain4j-vertex-ai and langchain4j-vertex-ai-gemini modules
uses: 'google-github-actions/auth@v2'
with:
project_id: ${{ secrets.GCP_PROJECT_ID }}
credentials_json: ${{ secrets.GCP_CREDENTIALS_JSON }}

- name: Setup Testcontainers Cloud Client
# Needed for langchain4j-ollama and other modules using testcontainers
uses: atomicjar/testcontainers-cloud-setup-action@v1
with:
token: ${{ secrets.TC_CLOUD_TOKEN }}

- name: Build with JDK ${{ matrix.java_version }}
run: mvn -B -U --fail-at-end ${{ matrix.included_modules }} verify
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AZURE_OPENAI_ENDPOINT: ${{ secrets.AZURE_OPENAI_ENDPOINT }}
AZURE_OPENAI_KEY: ${{ secrets.AZURE_OPENAI_KEY }}
AZURE_SEARCH_ENDPOINT: ${{ secrets.AZURE_SEARCH_ENDPOINT }}
AZURE_SEARCH_KEY: ${{ secrets.AZURE_SEARCH_KEY }}
COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
ELASTICSEARCH_CLOUD_API_KEY: ${{ secrets.ELASTICSEARCH_CLOUD_API_KEY }}
ELASTICSEARCH_CLOUD_URL: ${{ secrets.ELASTICSEARCH_CLOUD_URL }}
GCP_CREDENTIALS_JSON: ${{ secrets.GCP_CREDENTIALS_JSON }}
GCP_LOCATION: ${{ secrets.GCP_LOCATION }}
GCP_PROJECT_ID: ${{ secrets.GCP_PROJECT_ID }}
GCP_VERTEXAI_ENDPOINT: ${{ secrets.GCP_VERTEXAI_ENDPOINT }}
HF_API_KEY: ${{ secrets.HF_API_KEY }}
JINA_API_KEY: ${{ secrets.JINA_API_KEY }}
MILVUS_API_KEY: ${{ secrets.MILVUS_API_KEY }}
MILVUS_URI: ${{ secrets.MILVUS_URI }}
MISTRAL_AI_API_KEY: ${{ secrets.MISTRAL_AI_API_KEY }}
MONGODB_ATLAS_USERNAME: ${{ secrets.MONGODB_ATLAS_USERNAME }}
MONGODB_ATLAS_PASSWORD: ${{ secrets.MONGODB_ATLAS_PASSWORD }}
MONGODB_ATLAS_HOST: ${{ secrets.MONGODB_ATLAS_HOST }}
NOMIC_API_KEY: ${{ secrets.NOMIC_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
OPENAI_BASE_URL: ${{ secrets.OPENAI_BASE_URL }}
PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
TAVILY_API_KEY: ${{ secrets.TAVILY_API_KEY }}
WEAVIATE_API_KEY: ${{ secrets.WEAVIATE_API_KEY }}
WEAVIATE_HOST: ${{ secrets.WEAVIATE_HOST }}
6 changes: 5 additions & 1 deletion .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,11 @@ jobs:
token: ${{ secrets.TC_CLOUD_TOKEN }}

- name: release
run: mvn -B -U --fail-at-end -DskipTests -DskipAnthropicITs -DskipLocalAiITs -DskipMilvusITs -DskipMongoDbAtlasITs -DskipOllamaITs -DskipVearchITs -DskipVertexAiGeminiITs -pl !langchain4j-core,!langchain4j-parent -Psign clean deploy
run: mvn -B -U --fail-at-end -DskipTests -DskipITs -DskipAnthropicITs -DskipLocalAiITs -DskipMilvusITs -DskipMongoDbAtlasITs -DskipOllamaITs -DskipVearchITs -DskipVertexAiGeminiITs -pl !langchain4j-core,!langchain4j-parent -Psign clean deploy
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AZURE_OPENAI_ENDPOINT: ${{ secrets.AZURE_OPENAI_ENDPOINT }}
AZURE_OPENAI_KEY: ${{ secrets.AZURE_OPENAI_KEY }}
AZURE_SEARCH_ENDPOINT: ${{ secrets.AZURE_SEARCH_ENDPOINT }}
Expand All @@ -50,6 +52,7 @@ jobs:
GCP_PROJECT_ID: ${{ secrets.GCP_PROJECT_ID }}
GCP_VERTEXAI_ENDPOINT: ${{ secrets.GCP_VERTEXAI_ENDPOINT }}
HF_API_KEY: ${{ secrets.HF_API_KEY }}
JINA_API_KEY: ${{ secrets.JINA_API_KEY }}
MILVUS_API_KEY: ${{ secrets.MILVUS_API_KEY }}
MILVUS_URI: ${{ secrets.MILVUS_URI }}
MISTRAL_AI_API_KEY: ${{ secrets.MISTRAL_AI_API_KEY }}
Expand All @@ -60,6 +63,7 @@ jobs:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
OPENAI_BASE_URL: ${{ secrets.OPENAI_BASE_URL }}
PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
TAVILY_API_KEY: ${{ secrets.TAVILY_API_KEY }}
WEAVIATE_API_KEY: ${{ secrets.WEAVIATE_API_KEY }}
WEAVIATE_HOST: ${{ secrets.WEAVIATE_HOST }}
GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }}
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,4 @@ build/

### .env files contain local environment variables ###
.env
langchain4j-core/target_test-classes/
Binary file removed .idea/icon.png
Binary file not shown.
40 changes: 28 additions & 12 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
Thank you for investing your time and effort in contributing to our project, we appreciate it a lot! 🤗

# General Guidelines

- If you want to contribute a bug fix or a new feature that isn't listed in the [issues](https://github.com/langchain4j/langchain4j/issues) yet, please open a new issue for it and link it to your PR.
# Current situation (25 April 2024)
There are over 60 open PRs. Please help us by reviewing them first, before opening new ones. 🙏


# General guidelines
- If you want to contribute a bug fix or a new feature that isn't listed in the [issues](https://github.com/langchain4j/langchain4j/issues) yet, please open a new issue for it. We will prioritize is shortly.
- Follow [Google's Best Practices for Java Libraries](https://jlbp.dev/)
- Keep the code compatible with Java 8. We plan to increase the baseline to Java 17 a bit later.
- Avoid adding new dependencies as much as possible. If absolutely necessary, try to use the same libraries which are already used in the project.
Expand All @@ -13,42 +17,54 @@ Thank you for investing your time and effort in contributing to our project, we
- Follow existing code style present in the project.
- Large features should be discussed with maintainers before implementation. Please ping @langchain4j in the comments on the issue.


# Priorities
All [issues](https://github.com/langchain4j/langchain4j/issues) are prioritized by maintainers. There are 4 priorities: [P1](https://github.com/langchain4j/langchain4j/issues?q=is%3Aissue+is%3Aopen+label%3AP1), [P2](https://github.com/langchain4j/langchain4j/issues?q=is%3Aissue+is%3Aopen+label%3AP2), [P3](https://github.com/langchain4j/langchain4j/issues?q=is%3Aissue+is%3Aopen+label%3AP3) and [P4](https://github.com/langchain4j/langchain4j/issues?q=is%3Aissue+is%3Aopen+label%3AP4).

Please start with the higher priorities. PRs will be reviewed in order of priority, with bugs being a higher priority than new features.

Please note that we do not have the capacity to review all PRs immediately.
Please note that we do not have the capacity to review PRs immediately. We ask for your patience. We are doing our best to review your PR as quickly as possible.


# Opening an issue
- Please fill in all sections of the issue template.

# Opening a PR
- Link an [issue](https://github.com/langchain4j/langchain4j/issues) to your PR. If there is no issue yet, open one.

# Opening a draft PR
- Please open the PR as a draft initially. Once it is reviewed and approved, we will then ask you to finalize it (see section below).
- Fill in all the sections of the PR template.
- Make sure you've added tests.
- Make sure you've added documentation where required.
- For new big features, make sure you've added an example in the [examples repository](https://github.com/langchain4j/langchain4j-examples) (as a separate PR, linked to the main one).
- Please make it easier to review your PR:
- Keep changes as small as possible.
- Do not combine refactoring with changes in a single PR.
- Avoid reformatting existing code.


# Finalizing the draft PR
- Add [documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs) (if required).
- Add an example to the [examples repository](https://github.com/langchain4j/langchain4j-examples) (if required).
- [Mark a PR as ready for review](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-stage-of-a-pull-request#marking-a-pull-request-as-ready-for-review)


# Guidelines on adding a new model integration
- [Integration with Anthropic](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-anthropic) is a good example.
- Use the official SDK if available.
- If the official SDK is not available, use Retrofit and Gson to implement the client.
- If the official SDK is not available, use Retrofit and Jackson to implement the client.
- Document the new integration [here](https://github.com/langchain4j/langchain4j/blob/main/README.md), [here](https://github.com/langchain4j/langchain4j/tree/main/docs/docs/integrations/language-models) and [here](https://github.com/langchain4j/langchain4j/blob/main/docs/docs/integrations/language-models/index.md).
- Add an example to the [examples repository](https://github.com/langchain4j/langchain4j-examples), similar to [this](https://github.com/langchain4j/langchain4j-examples/tree/main/anthropic-examples).
- Add a new module to the appropriate section of the [BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml).
- It would be great if you could add a [Spring Boot starter](https://github.com/langchain4j/langchain4j-spring).


# Guidelines on adding a new embedding store integration
- [Integration with Chroma](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-chroma) is a good example.
- Use the official SDK if available.
- If the official SDK is not available, use Retrofit and Gson to implement the client.
- `{IntegrationName}EmbeddingStoreIT` should extend from `EmbeddingStoreWithFilteringIT` or `EmbeddingStoreIT` and pass all tests.
- If the official SDK is not available, use Retrofit and Jackson to implement the client.
- Add a `{IntegrationName}EmbeddingStoreIT`. It should extend from `EmbeddingStoreWithFilteringIT` or `EmbeddingStoreIT` and pass all tests.
- Document the new integration [here](https://github.com/langchain4j/langchain4j/blob/main/README.md), [here](https://github.com/langchain4j/langchain4j/tree/main/docs/docs/integrations/embedding-stores) and [here](https://github.com/langchain4j/langchain4j/blob/main/docs/docs/integrations/embedding-stores/index.md).
- Add an example to the [examples repository](https://github.com/langchain4j/langchain4j-examples), similar to [this](https://github.com/langchain4j/langchain4j-examples/tree/main/chroma-example).
- Add a new module to the appropriate section of the [BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml).
- It would be great if you could add a [Spring Boot starter](https://github.com/langchain4j/langchain4j-spring).
- It would be great if you could add a [Spring Boot starter](https://github.com/langchain4j/langchain4j-spring). (after


# Guidelines on changing an existing embedding store integration
- Ensure that your changes are backwards compatible. `Embedding`s and `TextSegment`s persisted with the latest released version of LangChain4j should still work.
Loading

0 comments on commit f9c8fe3

Please sign in to comment.