Skip to content
This repository has been archived by the owner on Apr 1, 2024. It is now read-only.

Bump pulsar version to 3.1.0-SNAPSHOT #5864

Closed
wants to merge 44 commits into from

Commits on Jul 20, 2023

  1. Configuration menu
    Copy the full SHA
    f5553a2 View commit details
    Browse the repository at this point in the history
  2. [improve] [ws] add cryptoKeyReaderFactoryClassName into the file webs…

    …ocket.conf (apache#20840)
    
    Motivation: Since the PR apache#16234 add the prop `cryptoKeyReaderFactoryClassName` for the WebSocket Proxy, but did not add this prop to `websocket.conf`.  This can make the script which try to replacement attribute a bit difficult to write
    
    Modifications: add the conf `cryptoKeyReaderFactoryClassName` into the file `websocket.conf`
    poorbarcode authored Jul 20, 2023
    Configuration menu
    Copy the full SHA
    5d0aa56 View commit details
    Browse the repository at this point in the history
  3. [fix][sec] Suppress addressed CVE-2023-2976 in clickhouse-jdbc-0.4.6-…

    …all and canal.client-1.1.5 (apache#20792)
    JooHyukKim authored Jul 20, 2023
    Configuration menu
    Copy the full SHA
    514fa60 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    aa4fef2 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    ac94296 View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2023

  1. Configuration menu
    Copy the full SHA
    d5f97bf View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    9b6a123 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    b8e6948 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2023

  1. Configuration menu
    Copy the full SHA
    9ccc562 View commit details
    Browse the repository at this point in the history
  2. [optimize][admin]Enhancing Transaction Buffer Stats and Introducing T…

    …ransactionBufferInternalStats API (apache#20330)
    
    master apache#20291
    
    ### Motivation
    
    Our primary goal is to improve the visibility and troubleshooting capabilities of the Pulsar Transaction Buffer by providing more detailed information about the snapshot stats and system topic internal status.
    ### Modifications
    
    1. Enhance the existing TransactionBufferStats by adding information about snapshot stats, including the capital of the current segment, unseal aborted transaction ID size, and other related data. This will provide better visibility and troubleshooting capabilities for the Pulsar Transaction Buffer.
    2. Introduce a new API for obtaining TransactionBufferInternalStats, allowing users to access the state of the system topic used for storing snapshots. This will facilitate problem investigation and resolution when issues arise with the transaction buffer.
    liangyepianzhou authored Jul 22, 2023
    Configuration menu
    Copy the full SHA
    ebfef6d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    54f0ca1 View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2023

  1. [fix][broker] Fix potential OOM due to client use sasl authentication (

    …apache#20713)
    
    Sasl authentication has two phases. In the first phase, the broker will temporarily cache the authentication info, and clean the cache at the end of the second phase.
    When the client crashes after phase 1, or someone deliberately attacks, the auth info will always remain in the broker's memory.
    If the client keeps reconnecting like this, the broker will soon OOM.
    315157973 authored Jul 23, 2023
    Configuration menu
    Copy the full SHA
    e7ae9e2 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    3c82d27 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    9256407 View commit details
    Browse the repository at this point in the history
  4. [improve][broker] Add the MessageExpirer interface to make code clear (

    …apache#20800)
    
    Co-authored-by: tison <wander4096@gmail.com>
    BewareMyPower and tisonkun authored Jul 23, 2023
    Configuration menu
    Copy the full SHA
    4ccb5bb View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2023

  1. Configuration menu
    Copy the full SHA
    69d7a2b View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2023

  1. Configuration menu
    Copy the full SHA
    563f929 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    3f63768 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2023

  1. [improve][cli] Better error message for pulsar-admin schema upload (a…

    …pache#20876)
    
    Signed-off-by: tison <wander4096@gmail.com>
    tisonkun authored Jul 26, 2023
    Configuration menu
    Copy the full SHA
    b12dbe7 View commit details
    Browse the repository at this point in the history
  2. [improve][broker] Add annotation for topic compaction strategy (apach…

    …e#20858)
    
    ### Motivation
    
    Add `Audience.Private` annotation for topic compaction strategy. Since this interface only uses for load manager, other users should not use this interface.
    Demogorgon314 authored Jul 26, 2023
    Configuration menu
    Copy the full SHA
    e55ed58 View commit details
    Browse the repository at this point in the history
  3. [fix][broker] Gracefully shutdown does not work with admin cli in sta…

    …ndalone (apache#20709)
    
    Fixes : apache#20617
    
    ### Motivation
    
    Currently, clients' shutdown API does not behave consistently in sense that asynchronicity is not handled explicitly. So issue apache#20617 happens.
    
    This PR will allow clients know that shutdown is actually triggered.
    
    ### Modifications
    
    - Synchronize call to `POST /shutdown` on client side
    - Asynchronize explicitly `pulsar().closeAsync()` invocation
    JooHyukKim authored Jul 26, 2023
    Configuration menu
    Copy the full SHA
    362c4f4 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    ee47b92 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    ffb9d30 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    a0372f4 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2023

  1. Configuration menu
    Copy the full SHA
    5d72615 View commit details
    Browse the repository at this point in the history
  2. [fix][broker] In replication scenario, remote consumer could not be r…

    …egistered if there has no message was sent (apache#20888)
    
    Motivation: In the replication scenario, we want to produce messages on the native cluster and consume messages on the remote cluster, the producer and consumer both use a same schema, but the consumer cannot be registered if there has no messages in the topic yet.The root cause is that for the remote cluster, there is a producer who has been registered with `AUTO_PRODUCE_BYTES` schema, so there is no schema to check the compatibility.
    
    Modifications: If there is no schema and only the replicator producer was registered, skip the compatibility check.
    poorbarcode authored Jul 28, 2023
    Configuration menu
    Copy the full SHA
    9be0b52 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2023

  1. [improve][txn]Add log when can't send TcClientConnectRequest to broke…

    …r. (apache#20880)
    
    Main Issue: apache#20851
    ### Motivation
    When the Proto version does not allow us to send TcClientConnectRequest to the broker, we should add a log to debug it.
    
    ### Modifications
    
    Add a waining log.
    liangyepianzhou authored Jul 31, 2023
    Configuration menu
    Copy the full SHA
    ca01447 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    001b134 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2023

  1. [fix][broker] fix MessageDeduplication throw NPE when enable broker d…

    …edup and set namespace disable deduplication. (apache#20905)
    lifepuzzlefun authored Aug 1, 2023
    Configuration menu
    Copy the full SHA
    4349e9f View commit details
    Browse the repository at this point in the history
  2. [fix] [ml] fix discontinuous ledger deletion (apache#20898)

    ### Motivation
    
    - The task `trim ledgers` runs in the thread `BkMainThreadPool.choose(ledgerName)`
    - The task `write entries to BK` runs in the thread `BkMainThreadPool.choose(ledgerId)`
    
    So the two tasks above may run concurrently/
    
    The task `trim ledgers` work as the flow below:
    - find the ledgers which are no longer to read, the result is `{Ledgers before the slowest read}`.
    - check if the `{Ledgers before the slowest read}` is out of retention policy, the result is `{Ledgers to be deleted}`.
      - if the create time of the ledger is lower than the earliest retention time, mark it should be deleted
      - if after deleting this ledger, the rest ledgers are still larger than the retention size, mark it should be deleted
    - delete the`{Ledgers to be deleted}`
    
    **(Highlight)** There is a scenario that causes the task `trim ledgers` did  discontinuous ledger deletion, resulting consume messages discontinuous:
    - context: 
      - ledgers: `[{id=1, size=100}, {id=2,size=100}]`
      - retention size: 150
      - no cursor there
    - Check `ledger 1`, skip by retention check `(200 - 100) < 150`
    - One in-flight writing is finished, the `calculateTotalSizeWrited()` would return `300` now.
    - Check `ledger 2`, retention check `(300 - 100) > 150`, mark the ledger-2 should be deleted.
    - Delete the `ledger 2`.
    - Create a new consumer. It will receive messages from `[ledger-1, ledegr-3]`, but the `ledger-2` will be skipped.
    
    ### Modifications
    
    Once the retention constraint has been met, break the loop.
    poorbarcode authored Aug 1, 2023
    Configuration menu
    Copy the full SHA
    782e91f View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2023

  1. Configuration menu
    Copy the full SHA
    c0f23eb View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    69298da View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2023

  1. Configuration menu
    Copy the full SHA
    2ab184e View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2023

  1. Configuration menu
    Copy the full SHA
    9862884 View commit details
    Browse the repository at this point in the history
  2. [fix][broker]Check that the super user role is in the MultiRolesToken…

    …AuthorizationProvider plugin (apache#20939)
    
    Fixed apache#20938
    tuteng authored Aug 11, 2023
    Configuration menu
    Copy the full SHA
    e41883e View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2023

  1. [improve][broker] Improve performance of StatsOutputStream by using…

    … `ArrayDeque` instead of `Stack` (apache#20808)
    JooHyukKim authored Aug 12, 2023
    Configuration menu
    Copy the full SHA
    d9538ea View commit details
    Browse the repository at this point in the history
  2. [improve][bk] Improve getIsolationGroup by avoid creating arrayList (a…

    …pache#20952)
    
    ### Motivation
    
    Minor improve `getIsolationGroup` by avoid creating arrayList. And it'  beneficial to GC
    
    ### Modifications
    
    `new HashSet<>(Arrays.asList(...))` -> `Sets.newHashSet(...)`
    AnonHxy authored Aug 12, 2023
    Configuration menu
    Copy the full SHA
    64cd753 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2023

  1. [feat][build] Use git-archive to generate source release (apache#20981)

    Signed-off-by: tison <wander4096@gmail.com>
    tisonkun authored Aug 13, 2023
    Configuration menu
    Copy the full SHA
    499eef6 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2023

  1. Configuration menu
    Copy the full SHA
    980dfc8 View commit details
    Browse the repository at this point in the history
  2. [fix][broker] Fix http error message of Bookie rack placement config …

    …API when bookie-rack is not configured. (apache#20949)
    1Jack2 authored Aug 14, 2023
    Configuration menu
    Copy the full SHA
    bafc827 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    75d4d82 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    63d9eaf View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    3ab420c View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    7977172 View commit details
    Browse the repository at this point in the history