diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 7adb1c2c2..4991a5968 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -16,5 +16,8 @@ jobs: - name: Setup Gradle uses: gradle/gradle-build-action@v2 + - name: Setup Android SDK + uses: android-actions/setup-android@v2 + - name: Execute Gradle build run: ./gradlew -s build dokkaJar \ No newline at end of file diff --git a/README.md b/README.md index 593dadd51..a61e2b443 100644 --- a/README.md +++ b/README.md @@ -6,58 +6,45 @@ ![Build Status](https://github.com/libp2p/jvm-libp2p/actions/workflows/build.yml/badge.svg?branch=master) [![Discourse posts](https://img.shields.io/discourse/https/discuss.libp2p.io/posts.svg)](https://discuss.libp2p.io) -> a libp2p implementation for the JVM, written in Kotlin 🔥 - -**⚠️ This is heavy work in progress! ⚠️** - -## Roadmap - -The endeavour to build jvm-libp2p is split in two phases: - -* **minimal phase (v0.x):** aims to provide the bare minimum stack that will - allow JVM-based Ethereum 2.0 clients to interoperate with other clients that - rely on fully-fledged libp2p stacks written in other languages. - * To achieve this, we have to be wire-compliant, but don't need to fulfill - the complete catalogue of libp2p abstractions. - * This effort will act as a starting point to evolve this project into a - fully-fledged libp2p stack for JVM environments, including Android - runtimes. - * We are shooting for Aug/early Sept 2019. - * Only Java-friendly façade. - -* **maturity phase (v1.x):** upgrades the minimal version to a flexible and - versatile stack adhering to the key design principles of modularity and - pluggability that define the libp2p project. It adds features present in - mature implementations like go-libp2p, rust-libp2p, js-libp2p. - * will offer: pluggable peerstore, connection manager, QUIC transport, - circuit relay, AutoNAT, AutoRelay, NAT traversal, etc. - * Android-friendly. - * Kotlin coroutine-based façade, possibly a Reactive Streams façade too. - * work will begin after the minimal phase concludes. - -## minimal phase (v0.x): Definition of Done - -We have identified the following components on the path to attaining a minimal -implementation: - -- [X] multistream-select 1.0 -- [X] multiformats: [multiaddr](https://github.com/multiformats/multiaddr) -- [X] crypto (RSA, ed25519, secp256k1) -- [X] [secio](https://github.com/libp2p/specs/pull/106) -- [X] [connection bootstrapping](https://github.com/libp2p/specs/pull/168) -- [X] mplex as a multiplexer -- [X] stream multiplexing -- [X] TCP transport (dialing and listening) -- [X] Identify protocol -- [X] Ping protocol -- [X] [peer ID](https://github.com/libp2p/specs/pull/100) -- [X] noise security protocol -- [X] MDNS -- [X] Gossip 1.1 pubsub - -We are explicitly leaving out the peerstore, DHT, pubsub, connection manager, -etc. and other subsystems or concepts that are internal to implementations and -do not impact the ability to hold communications with other libp2p processes. +[Libp2p](https://libp2p.io/) implementation for the JVM, written in Kotlin 🔥 + +## Components + +List of components in the Libp2p spec and their JVM implementation status + +| | Component | Status | +|--------------------------|-------------------------------------------------------------------------------------------------|:----------------:| +| **Transport** | tcp | :green_apple: | +| | [quic](https://github.com/libp2p/specs/tree/master/quic) | :tomato: | +| | websocket | :lemon: | +| | [webtransport](https://github.com/libp2p/specs/tree/master/webtransport) | | +| | [webrtc-browser-to-server](https://github.com/libp2p/specs/blob/master/webrtc/webrtc-direct.md) | | +| | [webrtc-private-to-private](https://github.com/libp2p/specs/blob/master/webrtc/webrtc.md) | | +| **Secure Communication** | [noise](https://github.com/libp2p/specs/blob/master/noise/) | :green_apple: | +| | [tls](https://github.com/libp2p/specs/blob/master/tls/tls.md) | :lemon: | +| | [plaintext](https://github.com/libp2p/specs/blob/master/plaintext/README.md) | :lemon: | +| | [secio](https://github.com/libp2p/specs/blob/master/secio/README.md) **(deprecated)** | :green_apple: | +| **Protocol Select** | [multistream](https://github.com/multiformats/multistream-select) | :green_apple: | +| **Stream Multiplexing** | [yamux](https://github.com/libp2p/specs/blob/master/yamux/README.md) | :lemon: | +| | [mplex](https://github.com/libp2p/specs/blob/master/mplex/README.md) | :green_apple: | +| **NAT Traversal** | [circuit-relay-v2](https://github.com/libp2p/specs/blob/master/relay/circuit-v2.md) | | +| | [autonat](https://github.com/libp2p/specs/tree/master/autonat) | | +| | [hole-punching](https://github.com/libp2p/specs/blob/master/connections/hole-punching.md) | | +| **Discovery** | [bootstrap](https://github.com/libp2p/specs/blob/master/kad-dht/README.md#bootstrap-process) | | +| | random-walk | | +| | [mdns-discovery](https://github.com/libp2p/specs/blob/master/discovery/mdns.md) | :lemon: | +| | [rendezvous](https://github.com/libp2p/specs/blob/master/rendezvous/README.md) | | +| **Peer Routing** | [kad-dht](https://github.com/libp2p/specs/blob/master/kad-dht/README.md) | | +| **Publish/Subscribe** | floodsub | :lemon: | +| | [gossipsub](https://github.com/libp2p/specs/tree/master/pubsub/gossipsub) | :green_apple: | +| **Storage** | record | | +| **Other protocols** | [ping](https://github.com/libp2p/specs/blob/master/ping/ping.md) | :green_apple: | +| | [identify](https://github.com/libp2p/specs/blob/master/identify/README.md) | :green_apple: | + +Legend: +- :green_apple: - tested in production +- :lemon: - prototype or beta, not tested in production +- :tomato: - in progress ## Gossip simulator @@ -65,11 +52,23 @@ Deterministic Gossip simulator which may simulate networks as large as 10000 of Please check the Simulator [README](tools/simulator/README.md) for more details +## Android support + +The library is basically being developed with Android compatibility in mind. +However we are not aware of anyone using it in production. + +The `examples/android-chatter` module contains working sample Android application. This module is ignored by the Gradle +build when no Android SDK is installed. +To include the Android module define a valid SDK location with an `ANDROID_HOME` environment variable +or by setting the `sdk.dir` path in your project's local properties file local.properties. + +Importing the project into Android Studio should work out of the box. + ## Adding as a dependency to your project Hosting of artefacts is graciously provided by [Cloudsmith](https://cloudsmith.com). -[![Latest version of 'jvm-libp2p-minimal' @ Cloudsmith](https://api-prd.cloudsmith.io/v1/badges/version/libp2p/jvm-libp2p/maven/jvm-libp2p-minimal/latest/a=noarch;xg=io.libp2p/?render=true&show_latest=true)](https://cloudsmith.io/~libp2p/repos/jvm-libp2p/packages/detail/maven/jvm-libp2p-minimal/latest/a=noarch;xg=io.libp2p/) +[![Latest version of 'jvm-libp2p' @ Cloudsmith](https://api-prd.cloudsmith.io/v1/badges/version/libp2p/jvm-libp2p/maven/jvm-libp2p/latest/a=noarch;xg=io.libp2p/?render=true&show_latest=true)](https://cloudsmith.io/~libp2p/repos/jvm-libp2p/packages/detail/maven/jvm-libp2p/latest/a=noarch;xg=io.libp2p/) As an alternative, artefacts are also available on [JitPack](https://jitpack.io/). @@ -87,7 +86,7 @@ Add the library to the `implementation` part of your Gradle file. ```groovy dependencies { // ... - implementation 'io.libp2p:jvm-libp2p-minimal:X.Y.Z-RELEASE' + implementation 'io.libp2p:jvm-libp2p:X.Y.Z-RELEASE' } ``` ### Using Maven @@ -113,7 +112,7 @@ And then add jvm-libp2p as a dependency: ``` xml io.libp2p - jvm-libp2p-minimal + jvm-libp2p X.Y.Z-RELEASE pom @@ -138,7 +137,15 @@ To build the library from the `jvm-libp2p` folder, run: ./gradlew build ``` -After the build is complete you may find the library `.jar` file here: `jvm-libp2p/build/libs/jvm-libp2p-minimal-0.x.y-RELEASE.jar` +After the build is complete you may find the library `.jar` file here: `jvm-libp2p/build/libs/jvm-libp2p-X.Y.Z-RELEASE.jar` + +## Notable users + +- [Teku](https://github.com/Consensys/teku) - Ethereum Consensus Layer client +- [Nabu](https://github.com/peergos/nabu) - minimal Java implementation of IPFS +- [Peergos](https://github.com/peergos/peergos) - peer-to-peer encrypted global filesystem + +(Please open a pull request if you want your project to be added here) ## License diff --git a/build.gradle.kts b/build.gradle.kts index 2b09f3ab5..64ce83673 100644 --- a/build.gradle.kts +++ b/build.gradle.kts @@ -5,10 +5,12 @@ import java.net.URL // To publish the release artifact to CloudSmith repo run the following : // ./gradlew publish -PcloudsmithUser= -PcloudsmithApiKey= -description = "a minimal implementation of libp2p for the jvm" +description = "an implementation of libp2p for the jvm" plugins { - kotlin("jvm").version("1.6.21") + val kotlinVersion = "1.6.21" + + id("org.jetbrains.kotlin.jvm") version kotlinVersion apply false id("com.github.ben-manes.versions").version("0.44.0") id("idea") @@ -19,16 +21,28 @@ plugins { id("org.jmailen.kotlinter").version("3.10.0") id("java-test-fixtures") id("io.spring.dependency-management").version("1.1.0") + + id("org.jetbrains.kotlin.android") version kotlinVersion apply false + id("com.android.application") version "7.4.2" apply false } -allprojects { +fun Project.getBooleanPropertyOrFalse(propName: String) = + (this.properties[propName] as? String)?.toBoolean() ?: false + +configure( + allprojects + .filterNot { + it.getBooleanPropertyOrFalse("libp2p.gradle.custom") + } +) { group = "io.libp2p" version = "develop" apply(plugin = "kotlin") apply(plugin = "idea") - apply(plugin = "io.gitlab.arturbosch.detekt") apply(plugin = "java") + + apply(plugin = "io.gitlab.arturbosch.detekt") apply(plugin = "maven-publish") apply(plugin = "org.jetbrains.dokka") apply(plugin = "org.jmailen.kotlinter") @@ -36,27 +50,23 @@ allprojects { apply(plugin = "io.spring.dependency-management") apply(from = "$rootDir/versions.gradle") - repositories { - mavenCentral() - maven("https://artifacts.consensys.net/public/maven/maven/") - } - dependencies { implementation(kotlin("stdlib-jdk8")) implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core") implementation("com.google.guava:guava") - implementation("org.apache.logging.log4j:log4j-api") + implementation("org.slf4j:slf4j-api") + implementation("com.github.multiformats:java-multibase:v1.1.1") - testFixturesImplementation("org.apache.logging.log4j:log4j-api") testFixturesImplementation("com.google.guava:guava") + testFixturesImplementation("org.slf4j:slf4j-api") testImplementation("org.junit.jupiter:junit-jupiter") testImplementation("org.junit.jupiter:junit-jupiter-params") testImplementation("io.mockk:mockk") testImplementation("org.assertj:assertj-core") - testImplementation("org.apache.logging.log4j:log4j-core") + testRuntimeOnly("org.apache.logging.log4j:log4j-slf4j2-impl") } java { diff --git a/examples/android-chatter/build.gradle b/examples/android-chatter/build.gradle index 9d4c46bcf..fdf829839 100644 --- a/examples/android-chatter/build.gradle +++ b/examples/android-chatter/build.gradle @@ -1,32 +1,16 @@ -buildscript { - ext.kotlin_version = '1.3.50' - repositories { - google() - jcenter() +apply plugin: "com.android.application" +apply plugin: "org.jetbrains.kotlin.android" - } - dependencies { - classpath 'com.android.tools.build:gradle:3.5.3' - classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version" - } -} - -repositories { - google() -} - -apply plugin: 'com.android.application' - -apply plugin: 'kotlin-android' - -apply plugin: 'kotlin-android-extensions' +apply plugin: "io.spring.dependency-management" +apply from: "$rootDir/versions.gradle" android { - compileSdkVersion 28 + namespace = "io.libp2p.example.chat" + compileSdkVersion 30 defaultConfig { - applicationId "io.libp2p.example.chatter" - minSdkVersion 28 - targetSdkVersion 28 + applicationId "io.libp2p.example.chat" + minSdkVersion 26 + targetSdkVersion 30 versionCode 1 versionName "1.0" testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner" @@ -38,20 +22,24 @@ android { } } packagingOptions { - exclude 'META-INF/DEPENDENCIES' + exclude 'META-INF/io.netty.versions.properties' + exclude 'META-INF/INDEX.LIST' + } + kotlinOptions { + jvmTarget = "11" } compileOptions { - sourceCompatibility = 1.8 - targetCompatibility = 1.8 + sourceCompatibility = 1.11 + targetCompatibility = 1.11 + } + lint { + abortOnError = false } } dependencies { - implementation fileTree(dir: 'libs', include: ['*.jar']) - implementation"org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version" - implementation 'androidx.appcompat:appcompat:1.1.0' - implementation 'androidx.core:core-ktx:1.1.0' - implementation 'androidx.constraintlayout:constraintlayout:1.1.3' - implementation project(':chatter') - annotationProcessor 'org.apache.logging.log4j:log4j-core:2.11.2' + implementation("androidx.appcompat:appcompat:1.2.0") + implementation("com.google.android.material:material:1.2.0") + implementation("androidx.constraintlayout:constraintlayout:2.0.4") + implementation project(':examples:chatter') } diff --git a/examples/android-chatter/gradle.properties b/examples/android-chatter/gradle.properties new file mode 100644 index 000000000..5ba6d5df0 --- /dev/null +++ b/examples/android-chatter/gradle.properties @@ -0,0 +1 @@ +libp2p.gradle.custom = true diff --git a/examples/android-chatter/src/main/AndroidManifest.xml b/examples/android-chatter/src/main/AndroidManifest.xml index 0ac48f735..e1cd80406 100644 --- a/examples/android-chatter/src/main/AndroidManifest.xml +++ b/examples/android-chatter/src/main/AndroidManifest.xml @@ -1,6 +1,5 @@ - + '} - case $link in #( - /*) app_path=$link ;; #( - *) app_path=$APP_HOME$link ;; - esac +PRG="$0" +# Need this for relative symlinks. +while [ -h "$PRG" ] ; do + ls=`ls -ld "$PRG"` + link=`expr "$ls" : '.*-> \(.*\)$'` + if expr "$link" : '/.*' > /dev/null; then + PRG="$link" + else + PRG=`dirname "$PRG"`"/$link" + fi done +SAVED="`pwd`" +cd "`dirname \"$PRG\"`/" >/dev/null +APP_HOME="`pwd -P`" +cd "$SAVED" >/dev/null -# This is normally unused -# shellcheck disable=SC2034 -APP_BASE_NAME=${0##*/} -APP_HOME=$( cd "${APP_HOME:-./}" && pwd -P ) || exit +APP_NAME="Gradle" +APP_BASE_NAME=`basename "$0"` # Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"' # Use the maximum available, or set MAX_FD != -1 to use that value. -MAX_FD=maximum +MAX_FD="maximum" warn () { echo "$*" -} >&2 +} die () { echo echo "$*" echo exit 1 -} >&2 +} # OS specific support (must be 'true' or 'false'). cygwin=false msys=false darwin=false nonstop=false -case "$( uname )" in #( - CYGWIN* ) cygwin=true ;; #( - Darwin* ) darwin=true ;; #( - MSYS* | MINGW* ) msys=true ;; #( - NONSTOP* ) nonstop=true ;; +case "`uname`" in + CYGWIN* ) + cygwin=true + ;; + Darwin* ) + darwin=true + ;; + MINGW* ) + msys=true + ;; + NONSTOP* ) + nonstop=true + ;; esac CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar @@ -121,9 +87,9 @@ CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar if [ -n "$JAVA_HOME" ] ; then if [ -x "$JAVA_HOME/jre/sh/java" ] ; then # IBM's JDK on AIX uses strange locations for the executables - JAVACMD=$JAVA_HOME/jre/sh/java + JAVACMD="$JAVA_HOME/jre/sh/java" else - JAVACMD=$JAVA_HOME/bin/java + JAVACMD="$JAVA_HOME/bin/java" fi if [ ! -x "$JAVACMD" ] ; then die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME @@ -132,7 +98,7 @@ Please set the JAVA_HOME variable in your environment to match the location of your Java installation." fi else - JAVACMD=java + JAVACMD="java" which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. Please set the JAVA_HOME variable in your environment to match the @@ -140,105 +106,80 @@ location of your Java installation." fi # Increase the maximum file descriptors if we can. -if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then - case $MAX_FD in #( - max*) - # In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked. - # shellcheck disable=SC3045 - MAX_FD=$( ulimit -H -n ) || - warn "Could not query maximum file descriptor limit" - esac - case $MAX_FD in #( - '' | soft) :;; #( - *) - # In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked. - # shellcheck disable=SC3045 - ulimit -n "$MAX_FD" || - warn "Could not set maximum file descriptor limit to $MAX_FD" - esac +if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then + MAX_FD_LIMIT=`ulimit -H -n` + if [ $? -eq 0 ] ; then + if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then + MAX_FD="$MAX_FD_LIMIT" + fi + ulimit -n $MAX_FD + if [ $? -ne 0 ] ; then + warn "Could not set maximum file descriptor limit: $MAX_FD" + fi + else + warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT" + fi fi -# Collect all arguments for the java command, stacking in reverse order: -# * args from the command line -# * the main class name -# * -classpath -# * -D...appname settings -# * --module-path (only if needed) -# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables. +# For Darwin, add options to specify how the application appears in the dock +if $darwin; then + GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\"" +fi # For Cygwin or MSYS, switch paths to Windows format before running java -if "$cygwin" || "$msys" ; then - APP_HOME=$( cygpath --path --mixed "$APP_HOME" ) - CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" ) - - JAVACMD=$( cygpath --unix "$JAVACMD" ) - +if [ "$cygwin" = "true" -o "$msys" = "true" ] ; then + APP_HOME=`cygpath --path --mixed "$APP_HOME"` + CLASSPATH=`cygpath --path --mixed "$CLASSPATH"` + + JAVACMD=`cygpath --unix "$JAVACMD"` + + # We build the pattern for arguments to be converted via cygpath + ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null` + SEP="" + for dir in $ROOTDIRSRAW ; do + ROOTDIRS="$ROOTDIRS$SEP$dir" + SEP="|" + done + OURCYGPATTERN="(^($ROOTDIRS))" + # Add a user-defined pattern to the cygpath arguments + if [ "$GRADLE_CYGPATTERN" != "" ] ; then + OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)" + fi # Now convert the arguments - kludge to limit ourselves to /bin/sh - for arg do - if - case $arg in #( - -*) false ;; # don't mess with options #( - /?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath - [ -e "$t" ] ;; #( - *) false ;; - esac - then - arg=$( cygpath --path --ignore --mixed "$arg" ) + i=0 + for arg in "$@" ; do + CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -` + CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option + + if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition + eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"` + else + eval `echo args$i`="\"$arg\"" fi - # Roll the args list around exactly as many times as the number of - # args, so each arg winds up back in the position where it started, but - # possibly modified. - # - # NB: a `for` loop captures its iteration list before it begins, so - # changing the positional parameters here affects neither the number of - # iterations, nor the values presented in `arg`. - shift # remove old arg - set -- "$@" "$arg" # push replacement arg + i=`expr $i + 1` done + case $i in + 0) set -- ;; + 1) set -- "$args0" ;; + 2) set -- "$args0" "$args1" ;; + 3) set -- "$args0" "$args1" "$args2" ;; + 4) set -- "$args0" "$args1" "$args2" "$args3" ;; + 5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;; + 6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;; + 7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;; + 8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;; + 9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;; + esac fi -# Collect all arguments for the java command; -# * $DEFAULT_JVM_OPTS, $JAVA_OPTS, and $GRADLE_OPTS can contain fragments of -# shell script including quotes and variable substitutions, so put them in -# double quotes to make sure that they get re-expanded; and -# * put everything else in single quotes, so that it's not re-expanded. - -set -- \ - "-Dorg.gradle.appname=$APP_BASE_NAME" \ - -classpath "$CLASSPATH" \ - org.gradle.wrapper.GradleWrapperMain \ - "$@" - -# Stop when "xargs" is not available. -if ! command -v xargs >/dev/null 2>&1 -then - die "xargs is not available" -fi - -# Use "xargs" to parse quoted args. -# -# With -n1 it outputs one arg per line, with the quotes and backslashes removed. -# -# In Bash we could simply go: -# -# readarray ARGS < <( xargs -n1 <<<"$var" ) && -# set -- "${ARGS[@]}" "$@" -# -# but POSIX shell has neither arrays nor command substitution, so instead we -# post-process each arg (as a line of input to sed) to backslash-escape any -# character that might be a shell metacharacter, then use eval to reverse -# that process (while maintaining the separation between arguments), and wrap -# the whole thing up as a single "set" statement. -# -# This will of course break if any of these variables contains a newline or -# an unmatched quote. -# +# Escape application args +save () { + for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done + echo " " +} +APP_ARGS=`save "$@"` -eval "set -- $( - printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" | - xargs -n1 | - sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' | - tr '\n' ' ' - )" '"$@"' +# Collect all arguments for the java command, following the shell quoting and substitution rules +eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS" exec "$JAVACMD" "$@" diff --git a/gradlew.bat b/gradlew.bat index 93e3f59f1..ac1b06f93 100644 --- a/gradlew.bat +++ b/gradlew.bat @@ -1,92 +1,89 @@ -@rem -@rem Copyright 2015 the original author or authors. -@rem -@rem Licensed under the Apache License, Version 2.0 (the "License"); -@rem you may not use this file except in compliance with the License. -@rem You may obtain a copy of the License at -@rem -@rem https://www.apache.org/licenses/LICENSE-2.0 -@rem -@rem Unless required by applicable law or agreed to in writing, software -@rem distributed under the License is distributed on an "AS IS" BASIS, -@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -@rem See the License for the specific language governing permissions and -@rem limitations under the License. -@rem - -@if "%DEBUG%"=="" @echo off -@rem ########################################################################## -@rem -@rem Gradle startup script for Windows -@rem -@rem ########################################################################## - -@rem Set local scope for the variables with windows NT shell -if "%OS%"=="Windows_NT" setlocal - -set DIRNAME=%~dp0 -if "%DIRNAME%"=="" set DIRNAME=. -@rem This is normally unused -set APP_BASE_NAME=%~n0 -set APP_HOME=%DIRNAME% - -@rem Resolve any "." and ".." in APP_HOME to make it shorter. -for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi - -@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. -set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" - -@rem Find java.exe -if defined JAVA_HOME goto findJavaFromJavaHome - -set JAVA_EXE=java.exe -%JAVA_EXE% -version >NUL 2>&1 -if %ERRORLEVEL% equ 0 goto execute - -echo. -echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. -echo. -echo Please set the JAVA_HOME variable in your environment to match the -echo location of your Java installation. - -goto fail - -:findJavaFromJavaHome -set JAVA_HOME=%JAVA_HOME:"=% -set JAVA_EXE=%JAVA_HOME%/bin/java.exe - -if exist "%JAVA_EXE%" goto execute - -echo. -echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% -echo. -echo Please set the JAVA_HOME variable in your environment to match the -echo location of your Java installation. - -goto fail - -:execute -@rem Setup the command line - -set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar - - -@rem Execute Gradle -"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %* - -:end -@rem End local scope for the variables with windows NT shell -if %ERRORLEVEL% equ 0 goto mainEnd - -:fail -rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of -rem the _cmd.exe /c_ return code! -set EXIT_CODE=%ERRORLEVEL% -if %EXIT_CODE% equ 0 set EXIT_CODE=1 -if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE% -exit /b %EXIT_CODE% - -:mainEnd -if "%OS%"=="Windows_NT" endlocal - -:omega +@rem +@rem Copyright 2015 the original author or authors. +@rem +@rem Licensed under the Apache License, Version 2.0 (the "License"); +@rem you may not use this file except in compliance with the License. +@rem You may obtain a copy of the License at +@rem +@rem https://www.apache.org/licenses/LICENSE-2.0 +@rem +@rem Unless required by applicable law or agreed to in writing, software +@rem distributed under the License is distributed on an "AS IS" BASIS, +@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +@rem See the License for the specific language governing permissions and +@rem limitations under the License. +@rem + +@if "%DEBUG%" == "" @echo off +@rem ########################################################################## +@rem +@rem Gradle startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables with windows NT shell +if "%OS%"=="Windows_NT" setlocal + +set DIRNAME=%~dp0 +if "%DIRNAME%" == "" set DIRNAME=. +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME% + +@rem Resolve any "." and ".." in APP_HOME to make it shorter. +for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi + +@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if "%ERRORLEVEL%" == "0" goto execute + +echo. +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto execute + +echo. +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:execute +@rem Setup the command line + +set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar + + +@rem Execute Gradle +"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %* + +:end +@rem End local scope for the variables with windows NT shell +if "%ERRORLEVEL%"=="0" goto mainEnd + +:fail +rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of +rem the _cmd.exe /c_ return code! +if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1 +exit /b 1 + +:mainEnd +if "%OS%"=="Windows_NT" endlocal + +:omega diff --git a/jitpack.yml b/jitpack.yml new file mode 100644 index 000000000..46c852919 --- /dev/null +++ b/jitpack.yml @@ -0,0 +1,2 @@ +jdk: + - openjdk11 \ No newline at end of file diff --git a/libp2p/build.gradle.kts b/libp2p/build.gradle.kts index 246b41836..49f8833ae 100644 --- a/libp2p/build.gradle.kts +++ b/libp2p/build.gradle.kts @@ -5,17 +5,28 @@ plugins { } dependencies { - api("io.netty:netty-all") + api("io.netty:netty-common") + api("io.netty:netty-buffer") + api("io.netty:netty-transport") + implementation("io.netty:netty-handler") + implementation("io.netty:netty-codec-http") + api("com.google.protobuf:protobuf-java") - implementation("commons-codec:commons-codec") implementation("tech.pegasys:noise-java") implementation("org.bouncycastle:bcprov-jdk15on") implementation("org.bouncycastle:bcpkix-jdk15on") + implementation("org.bouncycastle:bctls-jdk15on") testImplementation(project(":tools:schedulers")) + testFixturesApi("org.apache.logging.log4j:log4j-core") + testFixturesImplementation(project(":tools:schedulers")) + testFixturesImplementation("io.netty:netty-transport-classes-epoll") + testFixturesImplementation("io.netty:netty-handler") + testFixturesImplementation("org.junit.jupiter:junit-jupiter-api") + jmhImplementation(project(":tools:schedulers")) jmhImplementation("org.openjdk.jmh:jmh-core") jmhAnnotationProcessor("org.openjdk.jmh:jmh-generator-annprocess") diff --git a/libp2p/gradle.properties b/libp2p/gradle.properties index 0ef1e083b..3d6bf87a8 100644 --- a/libp2p/gradle.properties +++ b/libp2p/gradle.properties @@ -1 +1 @@ -mavenArtifactId=jvm-libp2p-minimal \ No newline at end of file +mavenArtifactId=jvm-libp2p \ No newline at end of file diff --git a/libp2p/src/main/java/io/libp2p/core/dsl/HostBuilder.java b/libp2p/src/main/java/io/libp2p/core/dsl/HostBuilder.java index f32724268..069d5701a 100644 --- a/libp2p/src/main/java/io/libp2p/core/dsl/HostBuilder.java +++ b/libp2p/src/main/java/io/libp2p/core/dsl/HostBuilder.java @@ -3,7 +3,7 @@ import io.libp2p.core.Host; import io.libp2p.core.crypto.PrivKey; import io.libp2p.core.multistream.ProtocolBinding; -import io.libp2p.core.mux.StreamMuxerProtocol; +import io.libp2p.core.mux.*; import io.libp2p.core.security.SecureChannel; import io.libp2p.core.transport.Transport; import io.libp2p.transport.ConnectionUpgrader; @@ -11,8 +11,7 @@ import java.util.ArrayList; import java.util.Arrays; import java.util.List; -import java.util.function.Function; -import java.util.function.Supplier; +import java.util.function.*; public class HostBuilder { public HostBuilder() { this(DefaultMode.Standard); } @@ -41,7 +40,7 @@ public final HostBuilder transport( @SafeVarargs public final HostBuilder secureChannel( - Function... secureChannels) { + BiFunction, SecureChannel>... secureChannels) { secureChannels_.addAll(Arrays.asList(secureChannels)); return this; } @@ -76,7 +75,7 @@ public Host build() { b.getTransports().add(t::apply) ); secureChannels_.forEach(sc -> - b.getSecureChannels().add(sc::apply) + b.getSecureChannels().add((k, m) -> sc.apply(k, (List)m)) ); muxers_.forEach(m -> b.getMuxers().add(m.get()) @@ -91,7 +90,7 @@ public Host build() { private DefaultMode defaultMode_; private List> transports_ = new ArrayList<>(); - private List> secureChannels_ = new ArrayList<>(); + private List, SecureChannel>> secureChannels_ = new ArrayList<>(); private List> muxers_ = new ArrayList<>(); private List> protocols_ = new ArrayList<>(); private List listenAddresses_ = new ArrayList<>(); diff --git a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/DNSIncoming.java b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/DNSIncoming.java index 351da96d5..f71bf1e29 100644 --- a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/DNSIncoming.java +++ b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/DNSIncoming.java @@ -14,8 +14,8 @@ import io.libp2p.discovery.mdns.impl.constants.DNSRecordClass; import io.libp2p.discovery.mdns.impl.constants.DNSRecordType; import io.libp2p.discovery.mdns.impl.constants.DNSResultCode; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.LogManager; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import io.libp2p.discovery.mdns.impl.constants.DNSConstants; import io.libp2p.discovery.mdns.impl.constants.DNSLabel; @@ -27,14 +27,14 @@ * @author Arthur van Hoff, Werner Randelshofer, Pierre Frisch, Daniel Bobbert */ public final class DNSIncoming extends DNSMessage { - private static Logger logger = LogManager.getLogger(DNSIncoming.class.getName()); + private static Logger logger = LoggerFactory.getLogger(DNSIncoming.class.getName()); // This is a hack to handle a bug in the BonjourConformanceTest // It is sending out target strings that don't follow the "domain name" format. public static boolean USE_DOMAIN_NAME_FORMAT_FOR_SRV_TARGET = true; public static class MessageInputStream extends ByteArrayInputStream { - private static Logger logger1 = LogManager.getLogger(MessageInputStream.class.getName()); + private static Logger logger1 = LoggerFactory.getLogger(MessageInputStream.class.getName()); final Map _names; diff --git a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/DNSQuestion.java b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/DNSQuestion.java index eccba86c4..789aaf418 100644 --- a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/DNSQuestion.java +++ b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/DNSQuestion.java @@ -8,8 +8,8 @@ import io.libp2p.discovery.mdns.impl.constants.DNSRecordClass; import io.libp2p.discovery.mdns.impl.constants.DNSRecordType; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.LogManager; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import io.libp2p.discovery.mdns.ServiceInfo; import io.libp2p.discovery.mdns.impl.constants.DNSConstants; @@ -20,7 +20,7 @@ * @author Arthur van Hoff, Pierre Frisch */ public class DNSQuestion extends DNSEntry { - private static Logger logger = LogManager.getLogger(DNSQuestion.class.getName()); + private static Logger logger = LoggerFactory.getLogger(DNSQuestion.class.getName()); /** * Pointer question. diff --git a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/DNSRecord.java b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/DNSRecord.java index bdc3e9ddd..6e75a6c47 100644 --- a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/DNSRecord.java +++ b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/DNSRecord.java @@ -8,8 +8,8 @@ import io.libp2p.discovery.mdns.impl.constants.DNSRecordClass; import io.libp2p.discovery.mdns.impl.constants.DNSRecordType; import io.libp2p.discovery.mdns.impl.util.ByteWrangler; -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.io.DataOutputStream; import java.io.IOException; @@ -26,7 +26,7 @@ * @author Arthur van Hoff, Rick Blair, Werner Randelshofer, Pierre Frisch */ public abstract class DNSRecord extends DNSEntry { - private static Logger logger = LogManager.getLogger(DNSRecord.class.getName()); + private static Logger logger = LoggerFactory.getLogger(DNSRecord.class.getName()); private int _ttl; private long _created; diff --git a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/HostInfo.java b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/HostInfo.java index dd47bb6dd..c93d6d983 100644 --- a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/HostInfo.java +++ b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/HostInfo.java @@ -8,8 +8,8 @@ import java.net.*; import java.util.*; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.LogManager; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * HostInfo information on the local host to be able to cope with change of addresses. @@ -17,7 +17,7 @@ * @author Pierre Frisch, Werner Randelshofer */ public class HostInfo { - private static Logger logger = LogManager.getLogger(HostInfo.class.getName()); + private static Logger logger = LoggerFactory.getLogger(HostInfo.class.getName()); protected String _name; diff --git a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/JmDNSImpl.java b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/JmDNSImpl.java index d8ad47121..d050053c4 100644 --- a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/JmDNSImpl.java +++ b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/JmDNSImpl.java @@ -12,8 +12,8 @@ import io.libp2p.discovery.mdns.impl.tasks.Responder; import io.libp2p.discovery.mdns.impl.tasks.ServiceResolver; import io.libp2p.discovery.mdns.impl.util.NamedThreadFactory; -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.io.IOException; import java.net.DatagramPacket; @@ -46,7 +46,7 @@ * @author Arthur van Hoff, Rick Blair, Jeff Sonstein, Werner Randelshofer, Pierre Frisch, Scott Lewis, Kai Kreuzer, Victor Toni */ public class JmDNSImpl extends JmDNS { - private static Logger logger = LogManager.getLogger(JmDNSImpl.class.getName()); + private static Logger logger = LoggerFactory.getLogger(JmDNSImpl.class.getName()); /** * This is the multicast group, we are listening to for multicast DNS messages. diff --git a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/ServiceInfoImpl.java b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/ServiceInfoImpl.java index 27ddfdd78..cc0c94dbf 100644 --- a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/ServiceInfoImpl.java +++ b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/ServiceInfoImpl.java @@ -7,8 +7,8 @@ import io.libp2p.discovery.mdns.ServiceInfo; import io.libp2p.discovery.mdns.impl.constants.DNSRecordClass; import io.libp2p.discovery.mdns.impl.util.ByteWrangler; -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.io.IOException; import java.net.Inet4Address; @@ -30,7 +30,7 @@ * @author Arthur van Hoff, Jeff Sonstein, Werner Randelshofer, Victor Toni */ public class ServiceInfoImpl extends ServiceInfo { - private static Logger logger = LogManager.getLogger(ServiceInfoImpl.class.getName()); + private static Logger logger = LoggerFactory.getLogger(ServiceInfoImpl.class.getName()); private String _domain; private String _protocol; diff --git a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/SocketListener.java b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/SocketListener.java index cf349253b..3edbf9acc 100644 --- a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/SocketListener.java +++ b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/SocketListener.java @@ -11,8 +11,8 @@ import java.util.concurrent.Future; import io.libp2p.discovery.mdns.impl.util.NamedThreadFactory; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.LogManager; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import io.libp2p.discovery.mdns.impl.constants.DNSConstants; @@ -20,7 +20,7 @@ * Listen for multicast packets. */ class SocketListener implements Runnable { - static Logger logger = LogManager.getLogger(SocketListener.class.getName()); + static Logger logger = LoggerFactory.getLogger(SocketListener.class.getName()); private final JmDNSImpl _jmDNSImpl; private final String _name; diff --git a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/constants/DNSRecordClass.java b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/constants/DNSRecordClass.java index 41c8ff49d..359279f88 100644 --- a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/constants/DNSRecordClass.java +++ b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/constants/DNSRecordClass.java @@ -3,8 +3,8 @@ */ package io.libp2p.discovery.mdns.impl.constants; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.LogManager; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * DNS Record Class @@ -41,7 +41,7 @@ public enum DNSRecordClass { */ CLASS_ANY("any", 255); - private static Logger logger = LogManager.getLogger(DNSRecordClass.class.getName()); + private static Logger logger = LoggerFactory.getLogger(DNSRecordClass.class.getName()); /** * Multicast DNS uses the bottom 15 bits to identify the record class...
diff --git a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/constants/DNSRecordType.java b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/constants/DNSRecordType.java index 3f2e36a26..0ad1e8e9f 100644 --- a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/constants/DNSRecordType.java +++ b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/constants/DNSRecordType.java @@ -3,8 +3,8 @@ */ package io.libp2p.discovery.mdns.impl.constants; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.LogManager; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * DNS Record Type @@ -249,7 +249,7 @@ public enum DNSRecordType { */ TYPE_ANY("any", 255); - private static Logger logger = LogManager.getLogger(DNSRecordType.class.getName()); + private static Logger logger = LoggerFactory.getLogger(DNSRecordType.class.getName()); private final String _externalName; diff --git a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/tasks/Responder.java b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/tasks/Responder.java index 1a576384c..77dab738e 100644 --- a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/tasks/Responder.java +++ b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/tasks/Responder.java @@ -11,8 +11,8 @@ import java.util.Set; import java.util.concurrent.TimeUnit; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.LogManager; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import io.libp2p.discovery.mdns.impl.DNSIncoming; import io.libp2p.discovery.mdns.impl.DNSOutgoing; @@ -25,7 +25,7 @@ * The Responder sends a single answer for the specified service infos and for the host name. */ public class Responder extends DNSTask { - static Logger logger = LogManager.getLogger(Responder.class.getName()); + static Logger logger = LoggerFactory.getLogger(Responder.class.getName()); private final DNSIncoming _in; diff --git a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/tasks/ServiceResolver.java b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/tasks/ServiceResolver.java index 895a7c71d..4b15a5544 100644 --- a/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/tasks/ServiceResolver.java +++ b/libp2p/src/main/java/io/libp2p/discovery/mdns/impl/tasks/ServiceResolver.java @@ -10,8 +10,8 @@ import io.libp2p.discovery.mdns.impl.constants.DNSConstants; import io.libp2p.discovery.mdns.impl.constants.DNSRecordClass; import io.libp2p.discovery.mdns.impl.constants.DNSRecordType; -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.io.IOException; import java.util.concurrent.Future; @@ -22,7 +22,7 @@ * The ServiceResolver queries three times consecutively for services of a given type, and then removes itself from the timer. */ public class ServiceResolver extends DNSTask { - private static Logger logger = LogManager.getLogger(ServiceResolver.class.getName()); + private static Logger logger = LoggerFactory.getLogger(ServiceResolver.class.getName()); private final String _type; private final int _queryInterval; diff --git a/libp2p/src/main/kotlin/io/libp2p/core/Host.kt b/libp2p/src/main/kotlin/io/libp2p/core/Host.kt index cfdb3586d..f419787ad 100644 --- a/libp2p/src/main/kotlin/io/libp2p/core/Host.kt +++ b/libp2p/src/main/kotlin/io/libp2p/core/Host.kt @@ -81,6 +81,8 @@ interface Host { */ fun addProtocolHandler(protocolBinding: ProtocolBinding) + fun getProtocols(): List> + /** * Removes the handler added with [addProtocolHandler] */ diff --git a/libp2p/src/main/kotlin/io/libp2p/core/dsl/Builders.kt b/libp2p/src/main/kotlin/io/libp2p/core/dsl/Builders.kt index 203aa476f..67dd02fcd 100644 --- a/libp2p/src/main/kotlin/io/libp2p/core/dsl/Builders.kt +++ b/libp2p/src/main/kotlin/io/libp2p/core/dsl/Builders.kt @@ -27,15 +27,16 @@ import io.libp2p.host.HostImpl import io.libp2p.host.MemoryAddressBook import io.libp2p.network.NetworkImpl import io.libp2p.protocol.IdentifyBinding -import io.libp2p.security.secio.SecIoSecureChannel +import io.libp2p.security.noise.NoiseXXSecureChannel import io.libp2p.transport.ConnectionUpgrader import io.libp2p.transport.tcp.TcpTransport import io.netty.channel.ChannelHandler import io.netty.handler.logging.LogLevel import io.netty.handler.logging.LoggingHandler +import java.util.concurrent.CopyOnWriteArrayList typealias TransportCtor = (ConnectionUpgrader) -> Transport -typealias SecureChannelCtor = (PrivKey) -> SecureChannel +typealias SecureChannelCtor = (PrivKey, List) -> SecureChannel typealias IdentityFactory = () -> PrivKey class HostConfigurationException(message: String) : RuntimeException(message) @@ -131,7 +132,7 @@ open class Builder { if (def == Defaults.Standard) { if (identity.factory == null) identity.random() if (transports.values.isEmpty()) transports { add(::TcpTransport) } - if (secureChannels.values.isEmpty()) secureChannels { add(::SecIoSecureChannel) } + if (secureChannels.values.isEmpty()) secureChannels { add(::NoiseXXSecureChannel) } if (muxers.values.isEmpty()) muxers { add(StreamMuxerProtocol.Mplex) } } @@ -160,8 +161,6 @@ open class Builder { val privKey = identity.factory!!() - val secureChannels = secureChannels.values.map { it(privKey) } - protocols.values.mapNotNull { (it as? IdentifyBinding) }.map { it.protocol }.find { it.idMessage == null }?.apply { // initializing Identify with appropriate values IdentifyOuterClass.Identify.newBuilder().apply { @@ -175,7 +174,10 @@ open class Builder { } } - val muxers = muxers.map { it.createMuxer(streamMultistreamProtocol, protocols.values) } + val updatableProtocols: MutableList> = CopyOnWriteArrayList(protocols.values) + val muxers = muxers.map { it.createMuxer(streamMultistreamProtocol, updatableProtocols) } + + val secureChannels = secureChannels.values.map { it(privKey, muxers) } if (debug.muxFramesHandler.handlers.isNotEmpty()) { val broadcast = ChannelVisitor.createBroadcast(*debug.muxFramesHandler.handlers.toTypedArray()) @@ -201,7 +203,7 @@ open class Builder { networkImpl, addressBook, network.listen.map { Multiaddr(it) }, - protocols.values, + updatableProtocols, broadcastConnHandler, streamVisitors ) diff --git a/libp2p/src/main/kotlin/io/libp2p/core/multiformats/MultiaddrDns.kt b/libp2p/src/main/kotlin/io/libp2p/core/multiformats/MultiaddrDns.kt index 61e0fcd4a..294fc5a8a 100644 --- a/libp2p/src/main/kotlin/io/libp2p/core/multiformats/MultiaddrDns.kt +++ b/libp2p/src/main/kotlin/io/libp2p/core/multiformats/MultiaddrDns.kt @@ -1,6 +1,6 @@ package io.libp2p.core.multiformats -import org.apache.logging.log4j.LogManager +import org.slf4j.LoggerFactory import java.net.Inet4Address import java.net.Inet6Address import java.net.InetAddress @@ -13,7 +13,7 @@ class MultiaddrDns { } companion object { - private val log = LogManager.getLogger(MultiaddrDns::class.java) + private val log = LoggerFactory.getLogger(MultiaddrDns::class.java) private val dnsProtocols = arrayOf(Protocol.DNS4, Protocol.DNS6, Protocol.DNSADDR) fun resolve(addr: Multiaddr, resolver: Resolver = DefaultResolver): List { @@ -54,7 +54,7 @@ class MultiaddrDns { } } } catch (e: UnknownHostException) { - log.debug(e) + log.debug("Unknown error", e) return emptyList() // squash, as this might not be fatal, // and if it is we'll handle this higher up the call chain diff --git a/libp2p/src/main/kotlin/io/libp2p/core/multiformats/Protocol.kt b/libp2p/src/main/kotlin/io/libp2p/core/multiformats/Protocol.kt index c190823a3..f097591a1 100644 --- a/libp2p/src/main/kotlin/io/libp2p/core/multiformats/Protocol.kt +++ b/libp2p/src/main/kotlin/io/libp2p/core/multiformats/Protocol.kt @@ -2,6 +2,8 @@ package io.libp2p.core.multiformats import com.google.common.base.Utf8 import com.google.common.net.InetAddresses +import io.ipfs.multibase.Multibase +import io.ipfs.multibase.binary.Base32 import io.libp2p.core.PeerId import io.libp2p.etc.encode.Base58 import io.libp2p.etc.types.readUvarint @@ -9,7 +11,6 @@ import io.libp2p.etc.types.toByteArray import io.libp2p.etc.types.toByteBuf import io.libp2p.etc.types.writeUvarint import io.netty.buffer.ByteBuf -import org.apache.commons.codec.binary.Base32 import java.net.Inet4Address import java.net.Inet6Address import java.net.InetAddress @@ -36,6 +37,7 @@ enum class Protocol( DCCP(33, 16, "dccp", UINT16_PARSER, UINT16_STRINGIFIER), IP6(41, 128, "ip6", IP6_PARSER, IP6_STRINGIFIER), IP6ZONE(42, LENGTH_PREFIXED_VAR_SIZE, "ip6zone", UTF8_PARSER, UTF8_STRINGIFIER, UTF8_VALIDATOR), + DNS(53, LENGTH_PREFIXED_VAR_SIZE, "dns", UTF8_PARSER, UTF8_STRINGIFIER, UTF8_VALIDATOR), DNS4(54, LENGTH_PREFIXED_VAR_SIZE, "dns4", UTF8_PARSER, UTF8_STRINGIFIER, UTF8_VALIDATOR), DNS6(55, LENGTH_PREFIXED_VAR_SIZE, "dns6", UTF8_PARSER, UTF8_STRINGIFIER, UTF8_VALIDATOR), DNSADDR(56, LENGTH_PREFIXED_VAR_SIZE, "dnsaddr", UTF8_PARSER, UTF8_STRINGIFIER, UTF8_VALIDATOR), @@ -48,7 +50,11 @@ enum class Protocol( HTTPS(443, 0, "https"), ONION(444, 96, "onion", ONION_PARSER, ONION_STRINGIFIER), QUIC(460, 0, "quic"), + QUICV1(461, 0, "quic-v1"), + WEBTRANSPORT(465, 0, "webtransport"), + CERTHASH(466, LENGTH_PREFIXED_VAR_SIZE, "certhash", MULTIBASE_PARSER, MULTIBASE_BASE64_STRINGIFIER), WS(477, 0, "ws"), + WSS(478, 0, "wss"), P2PCIRCUIT(290, 0, "p2p-circuit"), HTTP(480, 0, "http"); @@ -180,6 +186,12 @@ private val BASE58_PARSER: (Protocol, String) -> ByteArray = { _, addr -> private val BASE58_STRINGIFIER: (Protocol, ByteArray) -> String = { _, bytes -> Base58.encode(bytes) } +private val MULTIBASE_PARSER: (Protocol, String) -> ByteArray = { _, addr -> + Multibase.decode(addr) +} +private val MULTIBASE_BASE64_STRINGIFIER: (Protocol, ByteArray) -> String = { _, bytes -> + Multibase.encode(Multibase.Base.Base64Url, bytes) +} private val ONION_PARSER: (Protocol, String) -> ByteArray = { _, addr -> val split = addr.split(":") if (split.size != 2) throw IllegalArgumentException("Onion address needs a port: $addr") @@ -188,7 +200,6 @@ private val ONION_PARSER: (Protocol, String) -> ByteArray = { _, addr -> val base32 = Base32() val base32Text = split[0].uppercase() - if (!base32.isInAlphabet(base32Text)) throw IllegalArgumentException("Invalid Base32 string in the Onion address: $base32Text") val onionHostBytes = base32.decode(base32Text) val port = split[1].toInt() if (port > 65535) throw IllegalArgumentException("Port is > 65535: $port") diff --git a/libp2p/src/main/kotlin/io/libp2p/core/multistream/Multistream.kt b/libp2p/src/main/kotlin/io/libp2p/core/multistream/Multistream.kt index 2a7101334..1851086aa 100644 --- a/libp2p/src/main/kotlin/io/libp2p/core/multistream/Multistream.kt +++ b/libp2p/src/main/kotlin/io/libp2p/core/multistream/Multistream.kt @@ -25,5 +25,5 @@ interface Multistream : P2PChannelHandler { * For _initiator_ role this is the list of protocols the initiator wants to instantiate. * Basically this is either a single protocol or a protocol versions */ - val bindings: MutableList> + val bindings: List> } diff --git a/libp2p/src/main/kotlin/io/libp2p/core/multistream/NegotiatedProtocol.kt b/libp2p/src/main/kotlin/io/libp2p/core/multistream/NegotiatedProtocol.kt new file mode 100644 index 000000000..e0b04d655 --- /dev/null +++ b/libp2p/src/main/kotlin/io/libp2p/core/multistream/NegotiatedProtocol.kt @@ -0,0 +1,15 @@ +package io.libp2p.core.multistream + +import io.libp2p.core.P2PChannel +import java.util.concurrent.CompletableFuture + +/** + * Represents [ProtocolBinding] with exact protocol version which was agreed on + */ +open class NegotiatedProtocol> ( + val binding: TBinding, + val protocol: ProtocolId +) { + open fun initChannel(ch: P2PChannel): CompletableFuture = + binding.initChannel(ch, protocol) +} diff --git a/libp2p/src/main/kotlin/io/libp2p/core/multistream/ProtocolBinding.kt b/libp2p/src/main/kotlin/io/libp2p/core/multistream/ProtocolBinding.kt index 054b3c0f3..6e0b1224a 100644 --- a/libp2p/src/main/kotlin/io/libp2p/core/multistream/ProtocolBinding.kt +++ b/libp2p/src/main/kotlin/io/libp2p/core/multistream/ProtocolBinding.kt @@ -44,7 +44,7 @@ interface ProtocolBinding { /** * Returns initializer for this protocol on the provided channel, together with an optional controller object. */ - fun initChannel(ch: P2PChannel, selectedProtocol: String): CompletableFuture + fun initChannel(ch: P2PChannel, selectedProtocol: ProtocolId): CompletableFuture /** * If the [matcher] of this binding is not [Mode.STRICT] then it can't play initiator role since @@ -56,7 +56,7 @@ interface ProtocolBinding { val srcBinding = this return object : ProtocolBinding { override val protocolDescriptor = ProtocolDescriptor(protocols, srcBinding.protocolDescriptor.protocolMatcher) - override fun initChannel(ch: P2PChannel, selectedProtocol: String): CompletableFuture = + override fun initChannel(ch: P2PChannel, selectedProtocol: ProtocolId): CompletableFuture = srcBinding.initChannel(ch, selectedProtocol) } } @@ -68,7 +68,7 @@ interface ProtocolBinding { fun createSimple(protocolName: ProtocolId, handler: P2PChannelHandler): ProtocolBinding { return object : ProtocolBinding { override val protocolDescriptor = ProtocolDescriptor(protocolName) - override fun initChannel(ch: P2PChannel, selectedProtocol: String): CompletableFuture { + override fun initChannel(ch: P2PChannel, selectedProtocol: ProtocolId): CompletableFuture { return handler.initChannel(ch) } } diff --git a/libp2p/src/main/kotlin/io/libp2p/core/mux/NegotiatedStreamMuxer.kt b/libp2p/src/main/kotlin/io/libp2p/core/mux/NegotiatedStreamMuxer.kt new file mode 100644 index 000000000..4053b16df --- /dev/null +++ b/libp2p/src/main/kotlin/io/libp2p/core/mux/NegotiatedStreamMuxer.kt @@ -0,0 +1,5 @@ +package io.libp2p.core.mux + +import io.libp2p.core.multistream.NegotiatedProtocol + +typealias NegotiatedStreamMuxer = NegotiatedProtocol diff --git a/libp2p/src/main/kotlin/io/libp2p/core/mux/StreamMuxerProtocol.kt b/libp2p/src/main/kotlin/io/libp2p/core/mux/StreamMuxerProtocol.kt index 0c961c0cb..879cd60cd 100644 --- a/libp2p/src/main/kotlin/io/libp2p/core/mux/StreamMuxerProtocol.kt +++ b/libp2p/src/main/kotlin/io/libp2p/core/mux/StreamMuxerProtocol.kt @@ -3,6 +3,7 @@ package io.libp2p.core.mux import io.libp2p.core.multistream.MultistreamProtocol import io.libp2p.core.multistream.ProtocolBinding import io.libp2p.mux.mplex.MplexStreamMuxer +import io.libp2p.mux.yamux.YamuxStreamMuxer fun interface StreamMuxerProtocol { @@ -18,5 +19,15 @@ fun interface StreamMuxerProtocol { multistreamProtocol ) } + + @JvmStatic + val Yamux = StreamMuxerProtocol { multistreamProtocol, protocols -> + YamuxStreamMuxer( + multistreamProtocol.createMultistream( + protocols + ).toStreamHandler(), + multistreamProtocol + ) + } } } diff --git a/libp2p/src/main/kotlin/io/libp2p/core/security/SecureChannel.kt b/libp2p/src/main/kotlin/io/libp2p/core/security/SecureChannel.kt index fb14f039c..abd0b69cd 100644 --- a/libp2p/src/main/kotlin/io/libp2p/core/security/SecureChannel.kt +++ b/libp2p/src/main/kotlin/io/libp2p/core/security/SecureChannel.kt @@ -3,11 +3,13 @@ package io.libp2p.core.security import io.libp2p.core.PeerId import io.libp2p.core.crypto.PubKey import io.libp2p.core.multistream.ProtocolBinding +import io.libp2p.core.mux.NegotiatedStreamMuxer /** * The SecureChannel interface is implemented by all security channels, such as SecIO, TLS 1.3, Noise, and so on. */ interface SecureChannel : ProtocolBinding { + open class Session( /** * The peer ID of the local peer. @@ -20,8 +22,15 @@ interface SecureChannel : ProtocolBinding { val remoteId: PeerId, /** - * The public key of the + * The public key of the remote peer. + */ + val remotePubKey: PubKey, + + /** + * Contains muxer if security protocol supports + * [Early Multiplexer Negotiation](https://docs.libp2p.io/concepts/multiplex/early-negotiation/) + * and the protocol was successfully negotiated. Else contains `null` */ - val remotePubKey: PubKey + val earlyMuxer: NegotiatedStreamMuxer? ) } diff --git a/libp2p/src/main/kotlin/io/libp2p/crypto/keys/Ecdsa.kt b/libp2p/src/main/kotlin/io/libp2p/crypto/keys/Ecdsa.kt index b8f3e124e..3ff0de363 100644 --- a/libp2p/src/main/kotlin/io/libp2p/crypto/keys/Ecdsa.kt +++ b/libp2p/src/main/kotlin/io/libp2p/crypto/keys/Ecdsa.kt @@ -91,6 +91,10 @@ class EcdsaPublicKey(val pub: JavaECPublicKey) : PubKey(Crypto.KeyType.ECDSA) { override fun raw(): ByteArray = pub.encoded + fun javaKey(): JavaECPublicKey { + return pub + } + fun toUncompressedBytes(): ByteArray = byteArrayOf(0x04) + pub.w.affineX.toBytes(32) + pub.w.affineY.toBytes(32) diff --git a/libp2p/src/main/kotlin/io/libp2p/etc/util/P2PService.kt b/libp2p/src/main/kotlin/io/libp2p/etc/util/P2PService.kt index f48f05b50..13c2d3f38 100644 --- a/libp2p/src/main/kotlin/io/libp2p/etc/util/P2PService.kt +++ b/libp2p/src/main/kotlin/io/libp2p/etc/util/P2PService.kt @@ -8,11 +8,11 @@ import io.libp2p.etc.types.toVoidCompletableFuture import io.netty.channel.ChannelHandlerContext import io.netty.channel.ChannelInboundHandlerAdapter import io.netty.util.ReferenceCountUtil -import org.apache.logging.log4j.LogManager +import org.slf4j.LoggerFactory import java.util.concurrent.CompletableFuture import java.util.concurrent.ScheduledExecutorService -private val logger = LogManager.getLogger(P2PService::class.java) +private val logger = LoggerFactory.getLogger(P2PService::class.java) /** * Base class for a service which manages many streams from different peers diff --git a/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/AbstractMuxHandler.kt b/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/AbstractMuxHandler.kt index e6433d1a3..f50c3a088 100644 --- a/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/AbstractMuxHandler.kt +++ b/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/AbstractMuxHandler.kt @@ -7,13 +7,13 @@ import io.libp2p.etc.types.completedExceptionally import io.libp2p.etc.types.hasCauseOfType import io.netty.channel.ChannelHandlerContext import io.netty.channel.ChannelInboundHandlerAdapter -import org.apache.logging.log4j.LogManager +import org.slf4j.LoggerFactory import java.util.concurrent.CompletableFuture import java.util.function.Function typealias MuxChannelInitializer = (MuxChannel) -> Unit -private val log = LogManager.getLogger(AbstractMuxHandler::class.java) +private val log = LoggerFactory.getLogger(AbstractMuxHandler::class.java) abstract class AbstractMuxHandler() : ChannelInboundHandlerAdapter() { @@ -50,13 +50,26 @@ abstract class AbstractMuxHandler() : } fun getChannelHandlerContext(): ChannelHandlerContext { - return ctx ?: throw InternalErrorException("Internal error: handler context should be initialized at this stage") + return ctx + ?: throw InternalErrorException("Internal error: handler context should be initialized at this stage") } protected fun childRead(id: MuxId, msg: TData) { - val child = streamMap[id] ?: throw ConnectionClosedException("Channel with id $id not opened") - pendingReadComplete += id - child.pipeline().fireChannelRead(msg) + val child = streamMap[id] + when { + child == null -> { + releaseMessage(msg) + throw ConnectionClosedException("Channel with id $id not opened") + } + child.remoteDisconnected -> { + releaseMessage(msg) + throw ConnectionClosedException("Channel with id $id was closed for sending by remote") + } + else -> { + pendingReadComplete += id + child.pipeline().fireChannelRead(msg) + } + } } override fun channelReadComplete(ctx: ChannelHandlerContext) { @@ -64,6 +77,12 @@ abstract class AbstractMuxHandler() : pendingReadComplete.clear() } + /** + * Needs to be called when message was not passed to the child channel pipeline due to any error. + * (if a message was passed to the child channel it's the child channel's responsibility to release the message) + */ + abstract fun releaseMessage(msg: TData) + abstract fun onChildWrite(child: MuxChannel, data: TData) protected fun onRemoteOpen(id: MuxId) { @@ -96,6 +115,7 @@ abstract class AbstractMuxHandler() : fun onClosed(child: MuxChannel) { streamMap.remove(child.id) + onChildClosed(child) } abstract override fun channelRead(ctx: ChannelHandlerContext, msg: Any) @@ -103,6 +123,7 @@ abstract class AbstractMuxHandler() : protected abstract fun onLocalOpen(child: MuxChannel) protected abstract fun onLocalClose(child: MuxChannel) protected abstract fun onLocalDisconnect(child: MuxChannel) + protected abstract fun onChildClosed(child: MuxChannel) private fun createChild( id: MuxId, @@ -142,5 +163,6 @@ abstract class AbstractMuxHandler() : } } - private fun checkClosed() = if (closed) throw ConnectionClosedException("Can't create a new stream: connection was closed: " + ctx!!.channel()) else Unit + private fun checkClosed() = + if (closed) throw ConnectionClosedException("Can't create a new stream: connection was closed: " + ctx!!.channel()) else Unit } diff --git a/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/MuxChannel.kt b/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/MuxChannel.kt index 6b992b0a2..855046c5a 100644 --- a/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/MuxChannel.kt +++ b/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/MuxChannel.kt @@ -1,5 +1,6 @@ package io.libp2p.etc.util.netty.mux +import io.libp2p.core.ConnectionClosedException import io.libp2p.etc.util.netty.AbstractChildChannel import io.netty.channel.ChannelMetadata import io.netty.channel.ChannelOutboundBuffer @@ -16,8 +17,8 @@ class MuxChannel( val initiator: Boolean ) : AbstractChildChannel(parent.ctx!!.channel(), id) { - private var remoteDisconnected = false - private var localDisconnected = false + var remoteDisconnected = false + var localDisconnected = false override fun metadata(): ChannelMetadata = ChannelMetadata(true) override fun localAddress0() = @@ -35,6 +36,9 @@ class MuxChannel( while (true) { val msg = buf.current() ?: break try { + if (localDisconnected) { + throw ConnectionClosedException("The stream was closed for writing locally: $id") + } // the msg is released by both onChildWrite and buf.remove() so we need to retain // however it is still to be confirmed that no buf leaks happen here TODO ReferenceCountUtil.retain(msg) @@ -55,7 +59,7 @@ class MuxChannel( } fun onRemoteDisconnected() { - pipeline().fireUserEventTriggered(RemoteWriteClosed()) + pipeline().fireUserEventTriggered(RemoteWriteClosed) remoteDisconnected = true closeIfBothDisconnected() } @@ -74,11 +78,6 @@ class MuxChannel( } } -/** - * This Netty user event is fired to the [Stream] channel when remote peer closes its write side of the Stream - */ -class RemoteWriteClosed - data class MultiplexSocketAddress(val parentAddress: SocketAddress, val streamId: MuxId) : SocketAddress() { override fun toString(): String { return "Mux[$parentAddress-$streamId]" diff --git a/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/MuxId.kt b/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/MuxId.kt index 0d7051d93..25bb66f2a 100644 --- a/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/MuxId.kt +++ b/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/MuxId.kt @@ -3,8 +3,8 @@ package io.libp2p.etc.util.netty.mux import io.netty.channel.ChannelId data class MuxId(val parentId: ChannelId, val id: Long, val initiator: Boolean) : ChannelId { - override fun asShortText() = "$parentId/$id/$initiator" - override fun asLongText() = asShortText() + override fun asShortText() = "${parentId.asShortText()}/$id/$initiator" + override fun asLongText() = "${parentId.asLongText()}/$id/$initiator" override fun compareTo(other: ChannelId?) = asShortText().compareTo(other?.asShortText() ?: "") override fun toString() = asLongText() } diff --git a/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/RemoteWriteClosed.kt b/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/RemoteWriteClosed.kt new file mode 100644 index 000000000..fceb10dfa --- /dev/null +++ b/libp2p/src/main/kotlin/io/libp2p/etc/util/netty/mux/RemoteWriteClosed.kt @@ -0,0 +1,6 @@ +package io.libp2p.etc.util.netty.mux + +/** + * This Netty user event is fired to the [Stream] channel when remote peer closes its write side of the Stream + */ +object RemoteWriteClosed diff --git a/libp2p/src/main/kotlin/io/libp2p/host/HostImpl.kt b/libp2p/src/main/kotlin/io/libp2p/host/HostImpl.kt index 328f6018e..40df4e491 100644 --- a/libp2p/src/main/kotlin/io/libp2p/host/HostImpl.kt +++ b/libp2p/src/main/kotlin/io/libp2p/host/HostImpl.kt @@ -78,6 +78,10 @@ class HostImpl( protocolHandlers -= protocolBinding } + override fun getProtocols(): List> { + return protocolHandlers + } + override fun addConnectionHandler(handler: ConnectionHandler) { connectionHandlers += handler } diff --git a/libp2p/src/main/kotlin/io/libp2p/multistream/MultistreamImpl.kt b/libp2p/src/main/kotlin/io/libp2p/multistream/MultistreamImpl.kt index 4b7fc626d..8cf242f68 100644 --- a/libp2p/src/main/kotlin/io/libp2p/multistream/MultistreamImpl.kt +++ b/libp2p/src/main/kotlin/io/libp2p/multistream/MultistreamImpl.kt @@ -6,18 +6,14 @@ import io.libp2p.core.multistream.Multistream import io.libp2p.core.multistream.ProtocolBinding import java.time.Duration import java.util.concurrent.CompletableFuture -import java.util.concurrent.CopyOnWriteArrayList class MultistreamImpl( - initList: List> = listOf(), + override val bindings: List>, val preHandler: P2PChannelHandler<*>? = null, val postHandler: P2PChannelHandler<*>? = null, val negotiationTimeLimit: Duration = DEFAULT_NEGOTIATION_TIME_LIMIT ) : Multistream { - override val bindings: MutableList> = - CopyOnWriteArrayList(initList) - override fun initChannel(ch: P2PChannel): CompletableFuture { return with(ch) { preHandler?.also { diff --git a/libp2p/src/main/kotlin/io/libp2p/multistream/ProtocolSelect.kt b/libp2p/src/main/kotlin/io/libp2p/multistream/ProtocolSelect.kt index 434216fa0..332d8f2b6 100644 --- a/libp2p/src/main/kotlin/io/libp2p/multistream/ProtocolSelect.kt +++ b/libp2p/src/main/kotlin/io/libp2p/multistream/ProtocolSelect.kt @@ -23,7 +23,6 @@ class ProtocolSelect(val protocols: List() var activeFired = false - var userEvent = false override fun channelRead(ctx: ChannelHandlerContext, msg: Any) { // when protocol data immediately follows protocol id in the same packet @@ -43,7 +42,6 @@ class ProtocolSelect(val protocols: List { val protocolBinding = protocols.find { it.protocolDescriptor.protocolMatcher.matches(evt.proto) } diff --git a/libp2p/src/main/kotlin/io/libp2p/mux/MuxFrame.kt b/libp2p/src/main/kotlin/io/libp2p/mux/MuxFrame.kt deleted file mode 100644 index 2d14c3f2e..000000000 --- a/libp2p/src/main/kotlin/io/libp2p/mux/MuxFrame.kt +++ /dev/null @@ -1,23 +0,0 @@ -package io.libp2p.mux - -import io.libp2p.etc.types.toByteArray -import io.libp2p.etc.types.toHex -import io.libp2p.etc.util.netty.mux.MuxId -import io.netty.buffer.ByteBuf -import io.netty.buffer.DefaultByteBufHolder -import io.netty.buffer.Unpooled - -open class MuxFrame(val id: MuxId, val flag: Flag, val data: ByteBuf? = null) : - DefaultByteBufHolder(data ?: Unpooled.EMPTY_BUFFER) { - - enum class Flag { - OPEN, - DATA, - CLOSE, - RESET - } - - override fun toString(): String { - return "MuxFrame(id=$id, flag=$flag, data=${data?.toByteArray()?.toHex()})" - } -} diff --git a/libp2p/src/main/kotlin/io/libp2p/mux/MuxHandler.kt b/libp2p/src/main/kotlin/io/libp2p/mux/MuxHandler.kt index ce10cd67e..08a6bd12b 100644 --- a/libp2p/src/main/kotlin/io/libp2p/mux/MuxHandler.kt +++ b/libp2p/src/main/kotlin/io/libp2p/mux/MuxHandler.kt @@ -9,22 +9,18 @@ import io.libp2p.core.mux.StreamMuxer import io.libp2p.etc.CONNECTION import io.libp2p.etc.STREAM import io.libp2p.etc.types.forward -import io.libp2p.etc.types.sliceMaxSize import io.libp2p.etc.util.netty.mux.AbstractMuxHandler import io.libp2p.etc.util.netty.mux.MuxChannel import io.libp2p.etc.util.netty.mux.MuxChannelInitializer -import io.libp2p.etc.util.netty.mux.MuxId import io.libp2p.transport.implementation.StreamOverNetty import io.netty.buffer.ByteBuf import io.netty.channel.ChannelHandlerContext import java.util.concurrent.CompletableFuture -import java.util.concurrent.atomic.AtomicLong abstract class MuxHandler( private val ready: CompletableFuture?, inboundStreamHandler: StreamHandler<*> ) : AbstractMuxHandler(), StreamMuxer.Session { - private val idGenerator = AtomicLong(0xF) protected abstract val multistreamProtocol: MultistreamProtocol protected abstract val maxFrameDataLength: Int @@ -38,45 +34,6 @@ abstract class MuxHandler( ready?.complete(this) } - override fun channelRead(ctx: ChannelHandlerContext, msg: Any) { - msg as MuxFrame - when (msg.flag) { - MuxFrame.Flag.OPEN -> onRemoteOpen(msg.id) - MuxFrame.Flag.CLOSE -> onRemoteDisconnect(msg.id) - MuxFrame.Flag.RESET -> onRemoteClose(msg.id) - MuxFrame.Flag.DATA -> childRead(msg.id, msg.data!!) - } - } - - override fun onChildWrite(child: MuxChannel, data: ByteBuf) { - val ctx = getChannelHandlerContext() - data.sliceMaxSize(maxFrameDataLength) - .map { frameSliceBuf -> - MuxFrame(child.id, MuxFrame.Flag.DATA, frameSliceBuf) - }.forEach { muxFrame -> - ctx.write(muxFrame) - } - ctx.flush() - } - - override fun onLocalOpen(child: MuxChannel) { - getChannelHandlerContext().writeAndFlush(MuxFrame(child.id, MuxFrame.Flag.OPEN)) - } - - override fun onLocalDisconnect(child: MuxChannel) { - getChannelHandlerContext().writeAndFlush(MuxFrame(child.id, MuxFrame.Flag.CLOSE)) - } - - override fun onLocalClose(child: MuxChannel) { - getChannelHandlerContext().writeAndFlush(MuxFrame(child.id, MuxFrame.Flag.RESET)) - } - - override fun onRemoteCreated(child: MuxChannel) { - } - - override fun generateNextId() = - MuxId(getChannelHandlerContext().channel().id(), idGenerator.incrementAndGet(), true) - private fun createStream(channel: MuxChannel): Stream { val connection = ctx!!.channel().attr(CONNECTION).get() val stream = StreamOverNetty(channel, connection, channel.initiator) @@ -95,4 +52,8 @@ abstract class MuxHandler( }.thenApply { it.attr(STREAM).get() } return StreamPromise(stream, controller) } + + override fun releaseMessage(msg: ByteBuf) { + msg.release() + } } diff --git a/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexFlag.kt b/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexFlag.kt new file mode 100644 index 000000000..6cc4c685b --- /dev/null +++ b/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexFlag.kt @@ -0,0 +1,62 @@ +/* + * Copyright 2019 BLK Technologies Limited (web3labs.com). + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + */ +package io.libp2p.mux.mplex + +import io.libp2p.mux.mplex.MplexFlag.Type.* + +/** + * Contains all the permissible values for flags in the mplex protocol. + */ +enum class MplexFlag( + val value: Int, + val type: Type +) { + NewStream(0, OPEN), + MessageReceiver(1, DATA), + MessageInitiator(2, DATA), + CloseReceiver(3, CLOSE), + CloseInitiator(4, CLOSE), + ResetReceiver(5, RESET), + ResetInitiator(6, RESET); + + enum class Type { + OPEN, + DATA, + CLOSE, + RESET + } + + val isInitiator get() = value % 2 == 0 + + private val initiatorString get() = when (isInitiator) { + true -> "init" + false -> "resp" + } + + override fun toString(): String = "$type($initiatorString)" + + companion object { + private val valueToFlag = MplexFlag.values().associateBy { it.value } + + fun getByValue(flagValue: Int): MplexFlag = + valueToFlag[flagValue] ?: throw IllegalArgumentException("Invalid Mplex stream tag: $flagValue") + + fun getByType(type: Type, initiator: Boolean): MplexFlag = + when (type) { + OPEN -> NewStream + DATA -> if (initiator) MessageInitiator else MessageReceiver + CLOSE -> if (initiator) CloseInitiator else CloseReceiver + RESET -> if (initiator) ResetInitiator else ResetReceiver + } + } +} diff --git a/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexFlags.kt b/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexFlags.kt deleted file mode 100644 index b42431260..000000000 --- a/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexFlags.kt +++ /dev/null @@ -1,52 +0,0 @@ -/* - * Copyright 2019 BLK Technologies Limited (web3labs.com). - * - * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on - * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the - * specific language governing permissions and limitations under the License. - */ -package io.libp2p.mux.mplex - -import io.libp2p.core.Libp2pException -import io.libp2p.mux.MuxFrame -import io.libp2p.mux.MuxFrame.Flag.CLOSE -import io.libp2p.mux.MuxFrame.Flag.DATA -import io.libp2p.mux.MuxFrame.Flag.OPEN -import io.libp2p.mux.MuxFrame.Flag.RESET - -/** - * Contains all the permissible values for flags in the mplex protocol. - */ -object MplexFlags { - const val NewStream = 0 - const val MessageReceiver = 1 - const val MessageInitiator = 2 - const val CloseReceiver = 3 - const val CloseInitiator = 4 - const val ResetReceiver = 5 - const val ResetInitiator = 6 - - fun isInitiator(mplexFlag: Int) = mplexFlag % 2 == 0 - - fun toAbstractFlag(mplexFlag: Int): MuxFrame.Flag = - when (mplexFlag) { - NewStream -> OPEN - MessageReceiver, MessageInitiator -> DATA - CloseReceiver, CloseInitiator -> CLOSE - ResetReceiver, ResetInitiator -> RESET - else -> throw Libp2pException("Unknown mplex flag: $mplexFlag") - } - - fun toMplexFlag(abstractFlag: MuxFrame.Flag, initiator: Boolean): Int = - when (abstractFlag) { - OPEN -> NewStream - DATA -> if (initiator) MessageInitiator else MessageReceiver - CLOSE -> if (initiator) CloseInitiator else CloseReceiver - RESET -> if (initiator) ResetInitiator else ResetReceiver - } -} diff --git a/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexFrame.kt b/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexFrame.kt index b38b52f0c..fb4e11bc0 100644 --- a/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexFrame.kt +++ b/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexFrame.kt @@ -12,11 +12,10 @@ */ package io.libp2p.mux.mplex -import io.libp2p.etc.types.toByteArray -import io.libp2p.etc.types.toHex import io.libp2p.etc.util.netty.mux.MuxId -import io.libp2p.mux.MuxFrame import io.netty.buffer.ByteBuf +import io.netty.buffer.DefaultByteBufHolder +import io.netty.buffer.Unpooled /** * Contains the fields that comprise an mplex frame. @@ -26,11 +25,16 @@ import io.netty.buffer.ByteBuf * @param data the data segment. * @see [mplex documentation](https://github.com/libp2p/specs/tree/master/mplex#opening-a-new-stream) */ -class MplexFrame(channelId: MuxId, val mplexFlag: Int, data: ByteBuf? = null) : - MuxFrame(channelId, MplexFlags.toAbstractFlag(mplexFlag), data) { +data class MplexFrame(val id: MuxId, val flag: MplexFlag, val data: ByteBuf) : DefaultByteBufHolder(data) { - override fun toString(): String { - val init = if (MplexFlags.isInitiator(mplexFlag)) "init" else "resp" - return "MplexFrame(id=$id, flag=$flag ($init), data=${data?.toByteArray()?.toHex()})" + companion object { + fun createDataFrame(id: MuxId, data: ByteBuf) = + MplexFrame(id, MplexFlag.getByType(MplexFlag.Type.DATA, id.initiator), data) + fun createOpenFrame(id: MuxId) = + MplexFrame(id, MplexFlag.getByType(MplexFlag.Type.OPEN, id.initiator), Unpooled.EMPTY_BUFFER) + fun createCloseFrame(id: MuxId) = + MplexFrame(id, MplexFlag.getByType(MplexFlag.Type.CLOSE, id.initiator), Unpooled.EMPTY_BUFFER) + fun createResetFrame(id: MuxId) = + MplexFrame(id, MplexFlag.getByType(MplexFlag.Type.RESET, id.initiator), Unpooled.EMPTY_BUFFER) } } diff --git a/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexFrameCodec.kt b/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexFrameCodec.kt index a31658e38..9abe21ed8 100644 --- a/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexFrameCodec.kt +++ b/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexFrameCodec.kt @@ -16,9 +16,7 @@ import io.libp2p.core.ProtocolViolationException import io.libp2p.etc.types.readUvarint import io.libp2p.etc.types.writeUvarint import io.libp2p.etc.util.netty.mux.MuxId -import io.libp2p.mux.MuxFrame import io.netty.buffer.ByteBuf -import io.netty.buffer.Unpooled import io.netty.channel.ChannelHandlerContext import io.netty.handler.codec.ByteToMessageCodec @@ -29,7 +27,7 @@ const val DEFAULT_MAX_MPLEX_FRAME_DATA_LENGTH = 1 shl 20 */ class MplexFrameCodec( val maxFrameDataLength: Int = DEFAULT_MAX_MPLEX_FRAME_DATA_LENGTH -) : ByteToMessageCodec() { +) : ByteToMessageCodec() { /** * Encodes the given mplex frame into bytes and writes them into the output list. @@ -38,10 +36,10 @@ class MplexFrameCodec( * @param msg the mplex frame. * @param out the list to write the bytes to. */ - override fun encode(ctx: ChannelHandlerContext, msg: MuxFrame, out: ByteBuf) { - out.writeUvarint(msg.id.id.shl(3).or(MplexFlags.toMplexFlag(msg.flag, msg.id.initiator).toLong())) - out.writeUvarint(msg.data?.readableBytes() ?: 0) - out.writeBytes(msg.data ?: Unpooled.EMPTY_BUFFER) + override fun encode(ctx: ChannelHandlerContext, msg: MplexFrame, out: ByteBuf) { + out.writeUvarint(msg.id.id.shl(3).or(msg.flag.value.toLong())) + out.writeUvarint(msg.data.readableBytes()) + out.writeBytes(msg.data) } /** @@ -76,8 +74,8 @@ class MplexFrameCodec( val streamId = header.shr(3) val data = msg.readSlice(lenData.toInt()) data.retain() // MessageToMessageCodec releases original buffer, but it needs to be relayed - val initiator = if (streamTag == MplexFlags.NewStream) false else !MplexFlags.isInitiator(streamTag) - val mplexFrame = MplexFrame(MuxId(ctx.channel().id(), streamId, initiator), streamTag, data) + val flag = MplexFlag.getByValue(streamTag) + val mplexFrame = MplexFrame(MuxId(ctx.channel().id(), streamId, !flag.isInitiator), flag, data) out.add(mplexFrame) } } diff --git a/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexHandler.kt b/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexHandler.kt index 0cff7ffee..b87bdd8e6 100644 --- a/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexHandler.kt +++ b/libp2p/src/main/kotlin/io/libp2p/mux/mplex/MplexHandler.kt @@ -3,12 +3,59 @@ package io.libp2p.mux.mplex import io.libp2p.core.StreamHandler import io.libp2p.core.multistream.MultistreamProtocol import io.libp2p.core.mux.StreamMuxer +import io.libp2p.etc.types.sliceMaxSize +import io.libp2p.etc.util.netty.mux.MuxChannel +import io.libp2p.etc.util.netty.mux.MuxId import io.libp2p.mux.MuxHandler +import io.netty.buffer.ByteBuf +import io.netty.channel.ChannelHandlerContext import java.util.concurrent.CompletableFuture +import java.util.concurrent.atomic.AtomicLong open class MplexHandler( override val multistreamProtocol: MultistreamProtocol, override val maxFrameDataLength: Int, ready: CompletableFuture?, inboundStreamHandler: StreamHandler<*> -) : MuxHandler(ready, inboundStreamHandler) +) : MuxHandler(ready, inboundStreamHandler) { + + private val idGenerator = AtomicLong(0xF) + + override fun generateNextId() = + MuxId(getChannelHandlerContext().channel().id(), idGenerator.incrementAndGet(), true) + + override fun channelRead(ctx: ChannelHandlerContext, msg: Any) { + msg as MplexFrame + when (msg.flag.type) { + MplexFlag.Type.OPEN -> onRemoteOpen(msg.id) + MplexFlag.Type.CLOSE -> onRemoteDisconnect(msg.id) + MplexFlag.Type.RESET -> onRemoteClose(msg.id) + MplexFlag.Type.DATA -> childRead(msg.id, msg.data) + } + } + + override fun onChildWrite(child: MuxChannel, data: ByteBuf) { + val ctx = getChannelHandlerContext() + data.sliceMaxSize(maxFrameDataLength) + .map { frameSliceBuf -> + MplexFrame.createDataFrame(child.id, frameSliceBuf) + }.forEach { muxFrame -> + ctx.write(muxFrame) + } + ctx.flush() + } + + override fun onLocalOpen(child: MuxChannel) { + getChannelHandlerContext().writeAndFlush(MplexFrame.createOpenFrame(child.id)) + } + + override fun onLocalDisconnect(child: MuxChannel) { + getChannelHandlerContext().writeAndFlush(MplexFrame.createCloseFrame(child.id)) + } + + override fun onLocalClose(child: MuxChannel) { + getChannelHandlerContext().writeAndFlush(MplexFrame.createResetFrame(child.id)) + } + + override fun onChildClosed(child: MuxChannel) {} +} diff --git a/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxFlags.kt b/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxFlags.kt new file mode 100644 index 000000000..85499d0dd --- /dev/null +++ b/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxFlags.kt @@ -0,0 +1,11 @@ +package io.libp2p.mux.yamux + +/** + * Contains all the permissible values for flags in the yamux protocol. + */ +object YamuxFlags { + const val SYN = 1 + const val ACK = 2 + const val FIN = 4 + const val RST = 8 +} diff --git a/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxFrame.kt b/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxFrame.kt new file mode 100644 index 000000000..32bd32e6a --- /dev/null +++ b/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxFrame.kt @@ -0,0 +1,25 @@ +package io.libp2p.mux.yamux + +import io.libp2p.etc.types.toByteArray +import io.libp2p.etc.util.netty.mux.MuxId +import io.netty.buffer.ByteBuf +import io.netty.buffer.DefaultByteBufHolder +import io.netty.buffer.Unpooled + +/** + * Contains the fields that comprise a yamux frame. + * @param id the ID of the stream. + * @param flags the flags value for this frame. + * @param length the length field for this frame. + * @param data the data segment. + */ +class YamuxFrame(val id: MuxId, val type: Int, val flags: Int, val length: Long, val data: ByteBuf? = null) : + DefaultByteBufHolder(data ?: Unpooled.EMPTY_BUFFER) { + + override fun toString(): String { + if (data == null) { + return "YamuxFrame(id=$id, type=$type, flags=$flags, length=$length)" + } + return "YamuxFrame(id=$id, type=$type, flags=$flags, length=$length, data=${String(data.toByteArray())})" + } +} diff --git a/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxFrameCodec.kt b/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxFrameCodec.kt new file mode 100644 index 000000000..d8a8e2679 --- /dev/null +++ b/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxFrameCodec.kt @@ -0,0 +1,93 @@ +package io.libp2p.mux.yamux + +import io.libp2p.core.ProtocolViolationException +import io.libp2p.etc.util.netty.mux.MuxId +import io.netty.buffer.ByteBuf +import io.netty.buffer.Unpooled +import io.netty.channel.ChannelHandlerContext +import io.netty.handler.codec.ByteToMessageCodec + +const val DEFAULT_MAX_YAMUX_FRAME_DATA_LENGTH = 1 shl 20 + +/** + * A Netty codec implementation that converts [YamuxFrame] instances to [ByteBuf] and vice-versa. + */ +class YamuxFrameCodec( + val isInitiator: Boolean, + val maxFrameDataLength: Int = DEFAULT_MAX_YAMUX_FRAME_DATA_LENGTH +) : ByteToMessageCodec() { + + /** + * Encodes the given yamux frame into bytes and writes them into the output list. + * @see [https://github.com/hashicorp/yamux/blob/master/spec.md] + * @param ctx the context. + * @param msg the yamux frame. + * @param out the list to write the bytes to. + */ + override fun encode(ctx: ChannelHandlerContext, msg: YamuxFrame, out: ByteBuf) { + out.writeByte(0) // version + out.writeByte(msg.type) + out.writeShort(msg.flags) + out.writeInt(msg.id.id.toInt()) + out.writeInt(msg.data?.readableBytes() ?: msg.length.toInt()) + out.writeBytes(msg.data ?: Unpooled.EMPTY_BUFFER) + } + + /** + * Decodes the bytes in the given byte buffer and constructs a [YamuxFrame] that is written into + * the output list. + * @param ctx the context. + * @param msg the byte buffer. + * @param out the list to write the extracted frame to. + */ + override fun decode(ctx: ChannelHandlerContext, msg: ByteBuf, out: MutableList) { + while (msg.isReadable) { + if (msg.readableBytes() < 12) { + return + } + val readerIndex = msg.readerIndex() + msg.readByte(); // version always 0 + val type = msg.readUnsignedByte() + val flags = msg.readUnsignedShort() + val streamId = msg.readUnsignedInt() + val length = msg.readUnsignedInt() + if (type.toInt() != YamuxType.DATA) { + val yamuxFrame = YamuxFrame( + MuxId(ctx.channel().id(), streamId, isInitiator.xor(streamId.mod(2) == 1).not()), + type.toInt(), + flags, + length + ) + out.add(yamuxFrame) + continue + } + if (length > maxFrameDataLength) { + msg.skipBytes(msg.readableBytes()) + throw ProtocolViolationException("Yamux frame is too large: $length") + } + if (msg.readableBytes() < length) { + // not enough data to read the frame content + // will wait for more ... + msg.readerIndex(readerIndex) + return + } + val data = msg.readSlice(length.toInt()) + data.retain() // MessageToMessageCodec releases original buffer, but it needs to be relayed + val yamuxFrame = YamuxFrame( + MuxId(ctx.channel().id(), streamId, isInitiator.xor(streamId.mod(2) == 1).not()), + type.toInt(), + flags, + length, + data + ) + out.add(yamuxFrame) + } + } + + override fun exceptionCaught(ctx: ChannelHandlerContext, cause: Throwable) { + // notify higher level handlers on the error + ctx.fireExceptionCaught(cause) + // exceptions in [decode] are very likely unrecoverable so just close the connection + ctx.close() + } +} diff --git a/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxHandler.kt b/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxHandler.kt new file mode 100644 index 000000000..072051b74 --- /dev/null +++ b/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxHandler.kt @@ -0,0 +1,211 @@ +package io.libp2p.mux.yamux + +import io.libp2p.core.Libp2pException +import io.libp2p.core.StreamHandler +import io.libp2p.core.multistream.MultistreamProtocol +import io.libp2p.core.mux.StreamMuxer +import io.libp2p.etc.types.sliceMaxSize +import io.libp2p.etc.util.netty.mux.MuxChannel +import io.libp2p.etc.util.netty.mux.MuxId +import io.libp2p.mux.MuxHandler +import io.netty.buffer.ByteBuf +import io.netty.channel.ChannelHandlerContext +import org.slf4j.LoggerFactory +import java.util.concurrent.CompletableFuture +import java.util.concurrent.ConcurrentHashMap +import java.util.concurrent.atomic.AtomicInteger + +const val INITIAL_WINDOW_SIZE = 256 * 1024 +const val MAX_BUFFERED_CONNECTION_WRITES = 1024 * 1024 + +private val log = LoggerFactory.getLogger(YamuxHandler::class.java) + +open class YamuxHandler( + override val multistreamProtocol: MultistreamProtocol, + override val maxFrameDataLength: Int, + ready: CompletableFuture?, + inboundStreamHandler: StreamHandler<*>, + initiator: Boolean +) : MuxHandler(ready, inboundStreamHandler) { + private val idGenerator = AtomicInteger(if (initiator) 1 else 2) // 0 is reserved + private val receiveWindows = ConcurrentHashMap() + private val sendWindows = ConcurrentHashMap() + private val sendBuffers = ConcurrentHashMap() + private val totalBufferedWrites = AtomicInteger() + + inner class SendBuffer(val ctx: ChannelHandlerContext) { + private val buffered = ArrayDeque() + + fun add(data: ByteBuf) { + buffered.add(data.retain()) + } + + fun flush(sendWindow: AtomicInteger, id: MuxId): Int { + var written = 0 + while (!buffered.isEmpty()) { + val buf = buffered.first() + val readableBytes = buf.readableBytes() + if (readableBytes + written < sendWindow.get()) { + sendBlocks(ctx, buf, sendWindow, id) + written += readableBytes + buf.release() + buffered.removeFirst() + } else { + // partial write to fit within window + val toRead = sendWindow.get() - written + sendBlocks(ctx, buf.readSlice(toRead), sendWindow, id) + written += toRead + break + } + } + return written + } + } + + override fun channelRead(ctx: ChannelHandlerContext, msg: Any) { + msg as YamuxFrame + when (msg.type) { + YamuxType.DATA -> handleDataRead(msg) + YamuxType.WINDOW_UPDATE -> handleWindowUpdate(msg) + YamuxType.PING -> handlePing(msg) + YamuxType.GO_AWAY -> handleGoAway(msg) + } + } + + private fun handlePing(msg: YamuxFrame) { + val ctx = getChannelHandlerContext() + when (msg.flags) { + YamuxFlags.SYN -> ctx.writeAndFlush( + YamuxFrame( + MuxId(msg.id.parentId, 0, msg.id.initiator), + YamuxType.PING, + YamuxFlags.ACK, + msg.length + ) + ) + + YamuxFlags.ACK -> {} + } + } + + private fun handleFlags(msg: YamuxFrame) { + val ctx = getChannelHandlerContext() + when (msg.flags) { + YamuxFlags.SYN -> { + // ACK the new stream + onRemoteYamuxOpen(msg.id) + ctx.writeAndFlush(YamuxFrame(msg.id, YamuxType.WINDOW_UPDATE, YamuxFlags.ACK, 0)) + } + + YamuxFlags.FIN -> onRemoteDisconnect(msg.id) + YamuxFlags.RST -> onRemoteClose(msg.id) + } + } + + private fun handleDataRead(msg: YamuxFrame) { + val ctx = getChannelHandlerContext() + val size = msg.length + handleFlags(msg) + if (size.toInt() == 0) { + return + } + val recWindow = receiveWindows[msg.id] + if (recWindow == null) { + releaseMessage(msg.data!!) + throw Libp2pException("No receive window for " + msg.id) + } + val newWindow = recWindow.addAndGet(-size.toInt()) + if (newWindow < INITIAL_WINDOW_SIZE / 2) { + val delta = INITIAL_WINDOW_SIZE - newWindow + recWindow.addAndGet(delta) + ctx.write(YamuxFrame(msg.id, YamuxType.WINDOW_UPDATE, 0, delta.toLong())) + ctx.flush() + } + childRead(msg.id, msg.data!!) + } + + private fun handleWindowUpdate(msg: YamuxFrame) { + handleFlags(msg) + val size = msg.length.toInt() + if (size == 0) { + return + } + val sendWindow = sendWindows[msg.id] ?: return + sendWindow.addAndGet(size) + val buffer = sendBuffers[msg.id] + if (buffer != null) { + val writtenBytes = buffer.flush(sendWindow, msg.id) + totalBufferedWrites.addAndGet(-writtenBytes) + } + } + + private fun handleGoAway(msg: YamuxFrame) { + log.debug("Session will be terminated. Go Away message with with error code ${msg.length} has been received.") + onRemoteClose(msg.id) + } + + override fun onChildWrite(child: MuxChannel, data: ByteBuf) { + val ctx = getChannelHandlerContext() + + val sendWindow = sendWindows[child.id] ?: throw Libp2pException("No send window for " + child.id) + + if (sendWindow.get() <= 0) { + // wait until the window is increased to send more data + val buffer = sendBuffers.getOrPut(child.id) { SendBuffer(ctx) } + buffer.add(data) + if (totalBufferedWrites.addAndGet(data.readableBytes()) > MAX_BUFFERED_CONNECTION_WRITES) { + throw Libp2pException("Overflowed send buffer for connection") + } + return + } + sendBlocks(ctx, data, sendWindow, child.id) + } + + fun sendBlocks(ctx: ChannelHandlerContext, data: ByteBuf, sendWindow: AtomicInteger, id: MuxId) { + data.sliceMaxSize(minOf(maxFrameDataLength, sendWindow.get())) + .map { frameSliceBuf -> + sendWindow.addAndGet(-frameSliceBuf.readableBytes()) + YamuxFrame(id, YamuxType.DATA, 0, frameSliceBuf.readableBytes().toLong(), frameSliceBuf) + }.forEach { muxFrame -> + ctx.write(muxFrame) + } + ctx.flush() + } + + override fun onLocalOpen(child: MuxChannel) { + onStreamCreate(child.id) + getChannelHandlerContext().writeAndFlush(YamuxFrame(child.id, YamuxType.DATA, YamuxFlags.SYN, 0)) + } + + private fun onRemoteYamuxOpen(id: MuxId) { + onStreamCreate(id) + onRemoteOpen(id) + } + + private fun onStreamCreate(childId: MuxId) { + receiveWindows.putIfAbsent(childId, AtomicInteger(INITIAL_WINDOW_SIZE)) + sendWindows.putIfAbsent(childId, AtomicInteger(INITIAL_WINDOW_SIZE)) + } + + override fun onLocalDisconnect(child: MuxChannel) { + val sendWindow = sendWindows.remove(child.id) + val buffered = sendBuffers.remove(child.id) + if (buffered != null && sendWindow != null) { + buffered.flush(sendWindow, child.id) + } + getChannelHandlerContext().writeAndFlush(YamuxFrame(child.id, YamuxType.DATA, YamuxFlags.FIN, 0)) + } + + override fun onLocalClose(child: MuxChannel) { + getChannelHandlerContext().writeAndFlush(YamuxFrame(child.id, YamuxType.DATA, YamuxFlags.RST, 0)) + } + + override fun onChildClosed(child: MuxChannel) { + sendWindows.remove(child.id) + receiveWindows.remove(child.id) + sendBuffers.remove(child.id) + } + + override fun generateNextId() = + MuxId(getChannelHandlerContext().channel().id(), idGenerator.addAndGet(2).toLong(), true) +} diff --git a/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxStreamMuxer.kt b/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxStreamMuxer.kt new file mode 100644 index 000000000..4b43a0597 --- /dev/null +++ b/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxStreamMuxer.kt @@ -0,0 +1,39 @@ +package io.libp2p.mux.yamux + +import io.libp2p.core.ChannelVisitor +import io.libp2p.core.Connection +import io.libp2p.core.P2PChannel +import io.libp2p.core.StreamHandler +import io.libp2p.core.multistream.MultistreamProtocol +import io.libp2p.core.multistream.ProtocolDescriptor +import io.libp2p.core.mux.StreamMuxer +import io.libp2p.core.mux.StreamMuxerDebug +import java.util.concurrent.CompletableFuture + +class YamuxStreamMuxer( + val inboundStreamHandler: StreamHandler<*>, + private val multistreamProtocol: MultistreamProtocol +) : StreamMuxer, StreamMuxerDebug { + + override val protocolDescriptor = ProtocolDescriptor("/yamux/1.0.0") + override var muxFramesDebugHandler: ChannelVisitor? = null + + override fun initChannel(ch: P2PChannel, selectedProtocol: String): CompletableFuture { + val muxSessionReady = CompletableFuture() + + val yamuxFrameCodec = YamuxFrameCodec(ch.isInitiator) + ch.pushHandler(yamuxFrameCodec) + muxFramesDebugHandler?.also { it.visit(ch as Connection) } + ch.pushHandler( + YamuxHandler( + multistreamProtocol, + yamuxFrameCodec.maxFrameDataLength, + muxSessionReady, + inboundStreamHandler, + ch.isInitiator + ) + ) + + return muxSessionReady + } +} diff --git a/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxType.kt b/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxType.kt new file mode 100644 index 000000000..0746c8cf8 --- /dev/null +++ b/libp2p/src/main/kotlin/io/libp2p/mux/yamux/YamuxType.kt @@ -0,0 +1,11 @@ +package io.libp2p.mux.yamux + +/** + * Contains all the permissible values for types in the yamux protocol. + */ +object YamuxType { + const val DATA = 0 + const val WINDOW_UPDATE = 1 + const val PING = 2 + const val GO_AWAY = 3 +} diff --git a/libp2p/src/main/kotlin/io/libp2p/protocol/Ping.kt b/libp2p/src/main/kotlin/io/libp2p/protocol/Ping.kt index 616cd6450..7a9c20a0f 100644 --- a/libp2p/src/main/kotlin/io/libp2p/protocol/Ping.kt +++ b/libp2p/src/main/kotlin/io/libp2p/protocol/Ping.kt @@ -22,20 +22,23 @@ interface PingController { fun ping(): CompletableFuture } -class Ping : PingBinding(PingProtocol()) +class Ping(pingSize: Int) : PingBinding(PingProtocol(pingSize)) { + constructor() : this(32) +} open class PingBinding(ping: PingProtocol) : StrictProtocolBinding("/ipfs/ping/1.0.0", ping) class PingTimeoutException : Libp2pException() -open class PingProtocol : ProtocolHandler(Long.MAX_VALUE, Long.MAX_VALUE) { +open class PingProtocol(var pingSize: Int) : ProtocolHandler(Long.MAX_VALUE, Long.MAX_VALUE) { var timeoutScheduler by lazyVar { Executors.newSingleThreadScheduledExecutor() } var curTime: () -> Long = { System.currentTimeMillis() } var random = Random() - var pingSize = 32 var pingTimeout = Duration.ofSeconds(10) + constructor() : this(32) + override fun onStartInitiator(stream: Stream): CompletableFuture { val handler = PingInitiator() stream.pushHandler(handler) diff --git a/libp2p/src/main/kotlin/io/libp2p/pubsub/AbstractRouter.kt b/libp2p/src/main/kotlin/io/libp2p/pubsub/AbstractRouter.kt index 1af329720..eb8033cea 100644 --- a/libp2p/src/main/kotlin/io/libp2p/pubsub/AbstractRouter.kt +++ b/libp2p/src/main/kotlin/io/libp2p/pubsub/AbstractRouter.kt @@ -11,7 +11,7 @@ import io.netty.channel.ChannelHandler import io.netty.handler.codec.protobuf.ProtobufDecoder import io.netty.handler.codec.protobuf.ProtobufEncoder import io.netty.handler.codec.protobuf.ProtobufVarint32LengthFieldPrepender -import org.apache.logging.log4j.LogManager +import org.slf4j.LoggerFactory import pubsub.pb.Rpc import java.util.Collections.singletonList import java.util.Optional @@ -29,7 +29,7 @@ open class DefaultPubsubMessage(override val protobufMessage: Rpc.Message) : Abs override val messageId: MessageId = protobufMessage.from.toWBytes() + protobufMessage.seqno.toWBytes() } -private val logger = LogManager.getLogger(AbstractRouter::class.java) +private val logger = LoggerFactory.getLogger(AbstractRouter::class.java) /** * Implements common logic for pubsub routers diff --git a/libp2p/src/main/kotlin/io/libp2p/pubsub/gossip/GossipRouter.kt b/libp2p/src/main/kotlin/io/libp2p/pubsub/gossip/GossipRouter.kt index 0afc3eb24..2d3b21625 100644 --- a/libp2p/src/main/kotlin/io/libp2p/pubsub/gossip/GossipRouter.kt +++ b/libp2p/src/main/kotlin/io/libp2p/pubsub/gossip/GossipRouter.kt @@ -7,7 +7,7 @@ import io.libp2p.core.pubsub.ValidationResult import io.libp2p.etc.types.* import io.libp2p.etc.util.P2PService import io.libp2p.pubsub.* -import org.apache.logging.log4j.LogManager +import org.slf4j.LoggerFactory import pubsub.pb.Rpc import java.time.Duration import java.util.* @@ -71,7 +71,7 @@ fun P2PService.PeerHandler.getPeerProtocol(): PubsubProtocol { return PubsubProtocol.fromProtocol(proto) } -private val logger = LogManager.getLogger(GossipRouter::class.java) +private val logger = LoggerFactory.getLogger(GossipRouter::class.java) /** * Router implementing this protocol: https://github.com/libp2p/specs/tree/master/pubsub/gossipsub diff --git a/libp2p/src/main/kotlin/io/libp2p/security/noise/NoiseSecureChannelSession.kt b/libp2p/src/main/kotlin/io/libp2p/security/noise/NoiseSecureChannelSession.kt index b6e5485ef..bdf032a57 100644 --- a/libp2p/src/main/kotlin/io/libp2p/security/noise/NoiseSecureChannelSession.kt +++ b/libp2p/src/main/kotlin/io/libp2p/security/noise/NoiseSecureChannelSession.kt @@ -11,4 +11,4 @@ class NoiseSecureChannelSession( remotePubKey: PubKey, val aliceCipher: CipherState, val bobCipher: CipherState -) : SecureChannel.Session(localId, remoteId, remotePubKey) +) : SecureChannel.Session(localId, remoteId, remotePubKey, null) diff --git a/libp2p/src/main/kotlin/io/libp2p/security/noise/NoiseXXCodec.kt b/libp2p/src/main/kotlin/io/libp2p/security/noise/NoiseXXCodec.kt index 0ccd56305..548d1bce1 100644 --- a/libp2p/src/main/kotlin/io/libp2p/security/noise/NoiseXXCodec.kt +++ b/libp2p/src/main/kotlin/io/libp2p/security/noise/NoiseXXCodec.kt @@ -9,11 +9,11 @@ import io.netty.buffer.ByteBuf import io.netty.buffer.Unpooled import io.netty.channel.ChannelHandlerContext import io.netty.handler.codec.MessageToMessageCodec -import org.apache.logging.log4j.LogManager +import org.slf4j.LoggerFactory import java.io.IOException import java.security.GeneralSecurityException -private val logger = LogManager.getLogger(NoiseXXSecureChannel::class.java.name) +private val logger = LoggerFactory.getLogger(NoiseXXSecureChannel::class.java.name) class NoiseXXCodec(val aliceCipher: CipherState, val bobCipher: CipherState) : MessageToMessageCodec() { diff --git a/libp2p/src/main/kotlin/io/libp2p/security/noise/NoiseXXSecureChannel.kt b/libp2p/src/main/kotlin/io/libp2p/security/noise/NoiseXXSecureChannel.kt index 40bff231b..9bb407ef6 100644 --- a/libp2p/src/main/kotlin/io/libp2p/security/noise/NoiseXXSecureChannel.kt +++ b/libp2p/src/main/kotlin/io/libp2p/security/noise/NoiseXXSecureChannel.kt @@ -11,6 +11,7 @@ import io.libp2p.core.crypto.PubKey import io.libp2p.core.crypto.marshalPublicKey import io.libp2p.core.crypto.unmarshalPublicKey import io.libp2p.core.multistream.ProtocolDescriptor +import io.libp2p.core.mux.StreamMuxer import io.libp2p.core.security.SecureChannel import io.libp2p.etc.REMOTE_PEER_ID import io.libp2p.etc.types.toByteArray @@ -27,13 +28,13 @@ import io.netty.channel.SimpleChannelInboundHandler import io.netty.handler.codec.LengthFieldBasedFrameDecoder import io.netty.handler.codec.LengthFieldPrepender import io.netty.handler.timeout.ReadTimeoutHandler -import org.apache.logging.log4j.LogManager +import org.slf4j.LoggerFactory import spipe.pb.Spipe import java.util.concurrent.CompletableFuture enum class Role(val intVal: Int) { INIT(HandshakeState.INITIATOR), RESP(HandshakeState.RESPONDER) } -private val log = LogManager.getLogger(NoiseXXSecureChannel::class.java) +private val log = LoggerFactory.getLogger(NoiseXXSecureChannel::class.java) const val HandshakeNettyHandlerName = "HandshakeNettyHandler" const val HandshakeReadTimeoutNettyHandlerName = "HandshakeReadTimeoutNettyHandler" const val NoiseCodeNettyHandlerName = "NoiseXXCodec" @@ -48,6 +49,9 @@ class UShortLengthCodec : CombinedChannelDuplexHandler) : this(localKey) + companion object { const val protocolName = "Noise_XX_25519_ChaChaPoly_SHA256" const val announce = "/noise" @@ -63,10 +67,6 @@ class NoiseXXSecureChannel(private val localKey: PrivKey) : override val protocolDescriptor = ProtocolDescriptor(announce) - fun initChannel(ch: P2PChannel): CompletableFuture { - return initChannel(ch, "") - } // initChannel - override fun initChannel( ch: P2PChannel, selectedProtocol: String diff --git a/libp2p/src/main/kotlin/io/libp2p/security/plaintext/PlaintextInsecureChannel.kt b/libp2p/src/main/kotlin/io/libp2p/security/plaintext/PlaintextInsecureChannel.kt index 66518ef18..a75c71ce3 100644 --- a/libp2p/src/main/kotlin/io/libp2p/security/plaintext/PlaintextInsecureChannel.kt +++ b/libp2p/src/main/kotlin/io/libp2p/security/plaintext/PlaintextInsecureChannel.kt @@ -9,6 +9,7 @@ import io.libp2p.core.crypto.PrivKey import io.libp2p.core.crypto.PubKey import io.libp2p.core.crypto.unmarshalPublicKey import io.libp2p.core.multistream.ProtocolDescriptor +import io.libp2p.core.mux.StreamMuxer import io.libp2p.core.security.SecureChannel import io.libp2p.etc.types.toProtobuf import io.libp2p.security.InvalidInitialPacket @@ -23,6 +24,10 @@ import plaintext.pb.Plaintext import java.util.concurrent.CompletableFuture class PlaintextInsecureChannel(private val localKey: PrivKey) : SecureChannel { + + @Suppress("UNUSED_PARAMETER") + constructor(localKey: PrivKey, muxerProtocols: List) : this(localKey) + override val protocolDescriptor = ProtocolDescriptor("/plaintext/2.0.0") override fun initChannel(ch: P2PChannel, selectedProtocol: String): CompletableFuture { @@ -107,7 +112,8 @@ class PlaintextHandshakeHandler( val session = SecureChannel.Session( localPeerId, remotePeerId, - remotePubKey + remotePubKey, + null ) handshakeCompleted.complete(session) diff --git a/libp2p/src/main/kotlin/io/libp2p/security/secio/SecIoCodec.kt b/libp2p/src/main/kotlin/io/libp2p/security/secio/SecIoCodec.kt index fb97ba04e..dfdbbb3d5 100644 --- a/libp2p/src/main/kotlin/io/libp2p/security/secio/SecIoCodec.kt +++ b/libp2p/src/main/kotlin/io/libp2p/security/secio/SecIoCodec.kt @@ -10,16 +10,16 @@ import io.netty.buffer.ByteBuf import io.netty.buffer.Unpooled import io.netty.channel.ChannelHandlerContext import io.netty.handler.codec.MessageToMessageCodec -import org.apache.logging.log4j.LogManager import org.bouncycastle.crypto.StreamCipher import org.bouncycastle.crypto.engines.AESEngine import org.bouncycastle.crypto.modes.SICBlockCipher import org.bouncycastle.crypto.params.KeyParameter import org.bouncycastle.crypto.params.ParametersWithIV +import org.slf4j.LoggerFactory import java.io.IOException class SecIoCodec(val local: SecioParams, val remote: SecioParams) : MessageToMessageCodec() { - private val log = LogManager.getLogger(SecIoCodec::class.java) + private val log = LoggerFactory.getLogger(SecIoCodec::class.java) private val localCipher = createCipher(local) private val remoteCipher = createCipher(remote) diff --git a/libp2p/src/main/kotlin/io/libp2p/security/secio/SecIoSecureChannel.kt b/libp2p/src/main/kotlin/io/libp2p/security/secio/SecIoSecureChannel.kt index 27157648b..ae049e9bf 100644 --- a/libp2p/src/main/kotlin/io/libp2p/security/secio/SecIoSecureChannel.kt +++ b/libp2p/src/main/kotlin/io/libp2p/security/secio/SecIoSecureChannel.kt @@ -5,6 +5,7 @@ import io.libp2p.core.P2PChannel import io.libp2p.core.PeerId import io.libp2p.core.crypto.PrivKey import io.libp2p.core.multistream.ProtocolDescriptor +import io.libp2p.core.mux.StreamMuxer import io.libp2p.core.security.SecureChannel import io.libp2p.etc.REMOTE_PEER_ID import io.netty.buffer.ByteBuf @@ -12,13 +13,17 @@ import io.netty.channel.ChannelHandlerContext import io.netty.channel.SimpleChannelInboundHandler import io.netty.handler.codec.LengthFieldBasedFrameDecoder import io.netty.handler.codec.LengthFieldPrepender -import org.apache.logging.log4j.LogManager +import org.slf4j.LoggerFactory import java.util.concurrent.CompletableFuture -private val log = LogManager.getLogger(SecIoSecureChannel::class.java) +private val log = LoggerFactory.getLogger(SecIoSecureChannel::class.java) private val HandshakeHandlerName = "SecIoHandshake" class SecIoSecureChannel(private val localKey: PrivKey) : SecureChannel { + + @Suppress("UNUSED_PARAMETER") + constructor(localKey: PrivKey, muxerProtocols: List) : this(localKey) + override val protocolDescriptor = ProtocolDescriptor("/secio/1.0.0") override fun initChannel(ch: P2PChannel, selectedProtocol: String): CompletableFuture { @@ -68,7 +73,8 @@ private class SecIoHandshake( val session = SecureChannel.Session( PeerId.fromPubKey(secIoCodec.local.permanentPubKey), PeerId.fromPubKey(secIoCodec.remote.permanentPubKey), - secIoCodec.remote.permanentPubKey + secIoCodec.remote.permanentPubKey, + null ) handshakeComplete.complete(session) ctx.channel().pipeline().remove(HandshakeHandlerName) diff --git a/libp2p/src/main/kotlin/io/libp2p/security/tls/TLSSecureChannel.kt b/libp2p/src/main/kotlin/io/libp2p/security/tls/TLSSecureChannel.kt new file mode 100644 index 000000000..ad4d229f5 --- /dev/null +++ b/libp2p/src/main/kotlin/io/libp2p/security/tls/TLSSecureChannel.kt @@ -0,0 +1,348 @@ +package io.libp2p.security.tls + +import crypto.pb.Crypto +import io.libp2p.core.* +import io.libp2p.core.crypto.PrivKey +import io.libp2p.core.crypto.PubKey +import io.libp2p.core.crypto.unmarshalPublicKey +import io.libp2p.core.multistream.ProtocolBinding +import io.libp2p.core.multistream.ProtocolDescriptor +import io.libp2p.core.multistream.ProtocolId +import io.libp2p.core.mux.NegotiatedStreamMuxer +import io.libp2p.core.mux.StreamMuxer +import io.libp2p.core.security.SecureChannel +import io.libp2p.crypto.Libp2pCrypto +import io.libp2p.crypto.keys.EcdsaPublicKey +import io.libp2p.crypto.keys.Ed25519PublicKey +import io.libp2p.crypto.keys.generateEcdsaKeyPair +import io.libp2p.crypto.keys.generateEd25519KeyPair +import io.libp2p.etc.REMOTE_PEER_ID +import io.libp2p.security.InvalidRemotePubKey +import io.netty.buffer.ByteBuf +import io.netty.channel.ChannelHandlerContext +import io.netty.channel.SimpleChannelInboundHandler +import io.netty.handler.ssl.ApplicationProtocolConfig +import io.netty.handler.ssl.ClientAuth +import io.netty.handler.ssl.SslContextBuilder +import io.netty.handler.ssl.SslHandler +import org.bouncycastle.asn1.* +import org.bouncycastle.asn1.edec.EdECObjectIdentifiers +import org.bouncycastle.asn1.pkcs.PrivateKeyInfo +import org.bouncycastle.asn1.x500.X500Name +import org.bouncycastle.asn1.x509.AlgorithmIdentifier +import org.bouncycastle.asn1.x509.SubjectPublicKeyInfo +import org.bouncycastle.cert.X509CertificateHolder +import org.bouncycastle.cert.X509v3CertificateBuilder +import org.bouncycastle.cert.jcajce.JcaX509CertificateConverter +import org.bouncycastle.crypto.params.Ed25519PublicKeyParameters +import org.bouncycastle.jcajce.interfaces.EdDSAPublicKey +import org.bouncycastle.jsse.provider.BouncyCastleJsseProvider +import org.bouncycastle.operator.jcajce.JcaContentSignerBuilder +import java.math.BigInteger +import java.security.KeyFactory +import java.security.PrivateKey +import java.security.PublicKey +import java.security.Security +import java.security.cert.Certificate +import java.security.cert.CertificateException +import java.security.cert.X509Certificate +import java.security.interfaces.ECPublicKey +import java.security.spec.* +import java.time.Instant +import java.util.* +import java.util.concurrent.CompletableFuture +import java.util.logging.Level +import java.util.logging.Logger +import javax.net.ssl.X509TrustManager + +private val log = Logger.getLogger(TlsSecureChannel::class.java.name) + +const val NoEarlyMuxerNegotiationEntry = "libp2p" +const val SetupHandlerName = "TlsSetup" +val certificatePrefix = "libp2p-tls-handshake:".encodeToByteArray() + +class TlsSecureChannel(private val localKey: PrivKey, private val muxers: List, private val certAlgorithm: String) : + SecureChannel { + + constructor(localKey: PrivKey, muxerIds: List) : this(localKey, muxerIds, "Ed25519") {} + + companion object { + const val announce = "/tls/1.0.0" + init { + Security.insertProviderAt(Libp2pCrypto.provider, 1) + Security.insertProviderAt(BouncyCastleJsseProvider(), 2) + Security.setProperty("ssl.KeyManagerFactory.algorithm", "PKIX") + Security.setProperty("ssl.TrustManagerFactory.algorithm", "PKIX") + } + + @JvmStatic + fun ECDSA(localKey: PrivKey, muxerIds: List): TlsSecureChannel { + return TlsSecureChannel(localKey, muxerIds, "ECDSA") + } + } + + override val protocolDescriptor = ProtocolDescriptor(announce) + + fun initChannel(ch: P2PChannel): CompletableFuture { + return initChannel(ch, "") + } + + override fun initChannel( + ch: P2PChannel, + selectedProtocol: String + ): CompletableFuture { + val handshakeComplete = CompletableFuture() + ch.pushHandler(SetupHandlerName, ChannelSetup(localKey, muxers, certAlgorithm, ch, handshakeComplete)) + return handshakeComplete + } +} + +fun buildTlsHandler( + localKey: PrivKey, + expectedRemotePeer: Optional, + muxers: List, + certAlgorithm: String, + ch: P2PChannel, + handshakeComplete: CompletableFuture, + ctx: ChannelHandlerContext +): SslHandler { + val connectionKeys = if (certAlgorithm.equals("ECDSA")) generateEcdsaKeyPair() else generateEd25519KeyPair() + val javaPrivateKey = getJavaKey(connectionKeys.first) + val sslContext = ( + if (ch.isInitiator) + SslContextBuilder.forClient().keyManager(javaPrivateKey, listOf(buildCert(localKey, connectionKeys.first))) + else + SslContextBuilder.forServer(javaPrivateKey, listOf(buildCert(localKey, connectionKeys.first))) + ) + .protocols(listOf("TLSv1.3")) + .ciphers(listOf("TLS_AES_128_GCM_SHA256", "TLS_AES_256_GCM_SHA384", "TLS_CHACHA20_POLY1305_SHA256")) + .clientAuth(ClientAuth.REQUIRE) + .trustManager(Libp2pTrustManager(expectedRemotePeer)) + .sslContextProvider(BouncyCastleJsseProvider()) + .applicationProtocolConfig( + ApplicationProtocolConfig( + ApplicationProtocolConfig.Protocol.ALPN, + ApplicationProtocolConfig.SelectorFailureBehavior.FATAL_ALERT, + ApplicationProtocolConfig.SelectedListenerFailureBehavior.FATAL_ALERT, + muxers.allProtocols + NoEarlyMuxerNegotiationEntry // early muxer negotiation + ) + ) + .build() + val handler = sslContext.newHandler(ctx.alloc()) + handler.sslCloseFuture().addListener { _ -> ctx.close() } + val handshake = handler.handshakeFuture() + val engine = handler.engine() + handshake.addListener { fut -> + if (! fut.isSuccess) { + var cause = fut.cause() + if (cause != null && cause.cause != null) + cause = cause.cause + handshakeComplete.completeExceptionally(cause) + } else { + val nextProtocol = handler.applicationProtocol() + val selectedMuxer = muxers + .filter { mux -> + mux.protocolDescriptor.protocolMatcher.matches(nextProtocol) + } + .map { mux -> + NegotiatedStreamMuxer(mux, nextProtocol) + } + .firstOrNull() + handshakeComplete.complete( + SecureChannel.Session( + PeerId.fromPubKey(localKey.publicKey()), + verifyAndExtractPeerId(engine.session.peerCertificates), + getPublicKeyFromCert(engine.session.peerCertificates), + selectedMuxer + ) + ) + ctx.fireChannelActive() + } + } + return handler +} + +private val > List.allProtocols: List get() = + this.flatMap { it.protocolDescriptor.announceProtocols } + +private class ChannelSetup( + private val localKey: PrivKey, + private val muxers: List, + private val certAlgorithm: String, + private val ch: P2PChannel, + private val handshakeComplete: CompletableFuture +) : SimpleChannelInboundHandler() { + private var activated = false + + override fun channelActive(ctx: ChannelHandlerContext) { + if (! activated) { + activated = true + val expectedRemotePeerId = ctx.channel().attr(REMOTE_PEER_ID).get() + ctx.channel().pipeline().addLast( + buildTlsHandler( + localKey, + Optional.ofNullable(expectedRemotePeerId), + muxers, + certAlgorithm, + ch, + handshakeComplete, + ctx + ) + ) + ctx.channel().pipeline().remove(SetupHandlerName) + } + } + + override fun channelRead0(ctx: ChannelHandlerContext, msg: ByteBuf) { + // it seems there is no guarantee from Netty that channelActive() must be called before channelRead() + channelActive(ctx) + ctx.fireChannelRead(msg) + ctx.fireChannelActive() + } + + override fun exceptionCaught(ctx: ChannelHandlerContext, cause: Throwable) { + handshakeComplete.completeExceptionally(cause) + log.log(Level.FINE, "TLS setup failed", cause) + ctx.channel().close() + } + + override fun channelUnregistered(ctx: ChannelHandlerContext) { + handshakeComplete.completeExceptionally(ConnectionClosedException("Connection was closed ${ctx.channel()}")) + super.channelUnregistered(ctx) + } +} + +class Libp2pTrustManager(private val expectedRemotePeer: Optional) : X509TrustManager { + override fun checkClientTrusted(certs: Array?, authType: String?) { + if (certs?.size != 1) + throw CertificateException() + val claimedPeerId = verifyAndExtractPeerId(arrayOf(certs.get(0))) + if (expectedRemotePeer.map { ex -> ! ex.equals(claimedPeerId) }.orElse(false)) + throw InvalidRemotePubKey() + } + + override fun checkServerTrusted(certs: Array?, authType: String?) { + return checkClientTrusted(certs, authType) + } + + override fun getAcceptedIssuers(): Array { + return arrayOf() + } +} + +fun getJavaKey(priv: PrivKey): PrivateKey { + if (priv.keyType == Crypto.KeyType.Ed25519) { + val kf = KeyFactory.getInstance("Ed25519", Libp2pCrypto.provider) + val privKeyInfo = + PrivateKeyInfo(AlgorithmIdentifier(EdECObjectIdentifiers.id_Ed25519), DEROctetString(priv.raw())) + val pkcs8KeySpec = PKCS8EncodedKeySpec(privKeyInfo.encoded) + return kf.generatePrivate(pkcs8KeySpec) + } + if (priv.keyType == Crypto.KeyType.ECDSA) { + val kf = KeyFactory.getInstance("ECDSA", Libp2pCrypto.provider) + val pkcs8KeySpec = PKCS8EncodedKeySpec(priv.raw()) + return kf.generatePrivate(pkcs8KeySpec) + } + + if (priv.keyType == Crypto.KeyType.RSA) { + throw IllegalStateException("Unimplemented RSA key support for TLS") + } + throw IllegalArgumentException("Unsupported TLS key type:" + priv.keyType) +} + +fun getAsn1EncodedPublicKey(pub: PubKey): ByteArray { + if (pub.keyType == Crypto.KeyType.Ed25519) { + return SubjectPublicKeyInfo(AlgorithmIdentifier(EdECObjectIdentifiers.id_Ed25519), pub.raw()).encoded + } + if (pub.keyType == Crypto.KeyType.ECDSA) { + return (pub as EcdsaPublicKey).javaKey().encoded + } + throw IllegalArgumentException("Unsupported TLS key type:" + pub.keyType) +} + +fun getPubKey(pub: PublicKey): PubKey { + if (pub.algorithm.equals("EdDSA") || pub.algorithm.equals("Ed25519")) { + val raw = (pub as EdDSAPublicKey).pointEncoding + return Ed25519PublicKey(Ed25519PublicKeyParameters(raw)) + } + if (pub.algorithm.equals("EC")) { + return EcdsaPublicKey(pub as ECPublicKey) + } + if (pub.algorithm.equals("RSA")) + throw IllegalStateException("Unimplemented RSA public key support for TLS") + throw IllegalStateException("Unsupported key type: " + pub.algorithm) +} + +fun verifyAndExtractPeerId(chain: Array): PeerId { + if (chain.size != 1) + throw java.lang.IllegalStateException("Cert chain must have exactly 1 element!") + val cert = chain.get(0) + // peerid is in the certificate extension + val bcCert = org.bouncycastle.asn1.x509.Certificate + .getInstance(ASN1Primitive.fromByteArray(cert.getEncoded())) + val bcX509Cert = X509CertificateHolder(bcCert) + val libp2pOid = ASN1ObjectIdentifier("1.3.6.1.4.1.53594.1.1") + val extension = bcX509Cert.extensions.getExtension(libp2pOid) + if (extension == null) + throw IllegalStateException("Certificate extension not present!") + val input = ASN1InputStream(extension.extnValue.encoded) + val wrapper = input.readObject() as DEROctetString + val seq = ASN1InputStream(wrapper.octets).readObject() as DLSequence + val pubKeyProto = (seq.getObjectAt(0) as DEROctetString).octets + val signature = (seq.getObjectAt(1) as DEROctetString).octets + val pubKey = unmarshalPublicKey(pubKeyProto) + if (! pubKey.verify(certificatePrefix.plus(cert.publicKey.encoded), signature)) + throw IllegalStateException("Invalid signature on TLS certificate extension!") + + cert.verify(cert.publicKey) + val now = Date() + if (bcCert.endDate.date.before(now)) + throw IllegalStateException("TLS certificate has expired!") + if (bcCert.startDate.date.after(now)) + throw IllegalStateException("TLS certificate is not valid yet!") + return PeerId.fromPubKey(pubKey) +} + +fun getPublicKeyFromCert(chain: Array): PubKey { + if (chain.size != 1) + throw java.lang.IllegalStateException("Cert chain must have exactly 1 element!") + val cert = chain.get(0) + return getPubKey(cert.publicKey) +} + +/** Build a self signed cert, with an extension containing the host key + sig(cert public key) + * + */ +fun buildCert(hostKey: PrivKey, subjectKey: PrivKey): X509Certificate { + val publicKeyAsn1 = getAsn1EncodedPublicKey(subjectKey.publicKey()) + val subPubKeyInfo = SubjectPublicKeyInfo.getInstance(publicKeyAsn1) + + val now = Instant.now() + val validFrom = Date.from(now.minusSeconds(3600)) + val oneYear = 60L * 60 * 24 * 365 + val validTo = Date.from(now.plusSeconds(oneYear)) + val issuer = X500Name("O=Peergos,L=Oxford,C=UK") + val subject = issuer + + val signature = hostKey.sign(certificatePrefix.plus(publicKeyAsn1)) + val hostPublicProto = hostKey.publicKey().bytes() + val extension = DERSequence(arrayOf(DEROctetString(hostPublicProto), DEROctetString(signature))) + + var certBuilder = X509v3CertificateBuilder( + issuer, + BigInteger.valueOf(now.toEpochMilli()), + validFrom, + validTo, + subject, + subPubKeyInfo + ).addExtension(ASN1ObjectIdentifier("1.3.6.1.4.1.53594.1.1"), true, extension) + val sigAlg = when (subjectKey.keyType) { + Crypto.KeyType.Ed25519 -> "Ed25519" + Crypto.KeyType.ECDSA -> "SHA256withECDSA" + else -> throw IllegalStateException("Unsupported certificate key type: " + subjectKey.keyType) + } + val signer = JcaContentSignerBuilder(sigAlg) + .setProvider(Libp2pCrypto.provider) + .build(getJavaKey(subjectKey)) + return JcaX509CertificateConverter().getCertificate(certBuilder.build(signer)) +} diff --git a/libp2p/src/main/kotlin/io/libp2p/transport/ConnectionUpgrader.kt b/libp2p/src/main/kotlin/io/libp2p/transport/ConnectionUpgrader.kt index 9aa6776d0..4df94b197 100644 --- a/libp2p/src/main/kotlin/io/libp2p/transport/ConnectionUpgrader.kt +++ b/libp2p/src/main/kotlin/io/libp2p/transport/ConnectionUpgrader.kt @@ -3,8 +3,12 @@ package io.libp2p.transport import io.libp2p.core.Connection import io.libp2p.core.multistream.MultistreamProtocol import io.libp2p.core.multistream.ProtocolBinding +import io.libp2p.core.mux.NegotiatedStreamMuxer import io.libp2p.core.mux.StreamMuxer import io.libp2p.core.security.SecureChannel +import io.libp2p.etc.getP2PChannel +import io.libp2p.etc.types.forward +import io.libp2p.etc.util.netty.nettyInitializer import java.util.concurrent.CompletableFuture /** @@ -31,14 +35,26 @@ open class ConnectionUpgrader( connection, muxers ) - } // establishMuxer + } private fun , R> establish( multistreamProtocol: MultistreamProtocol, connection: Connection, - channels: List + bindings: List ): CompletableFuture { - val multistream = multistreamProtocol.createMultistream(channels) + val multistream = multistreamProtocol.createMultistream(bindings) return multistream.initChannel(connection) } // establish + + companion object { + fun establishMuxer(muxer: NegotiatedStreamMuxer, connection: Connection): CompletableFuture { + val res = CompletableFuture() + connection.pushHandler( + nettyInitializer { + muxer.initChannel(it.channel.getP2PChannel()).forward(res) + } + ) + return res + } + } } // ConnectionUpgrader diff --git a/libp2p/src/main/kotlin/io/libp2p/transport/implementation/ConnectionBuilder.kt b/libp2p/src/main/kotlin/io/libp2p/transport/implementation/ConnectionBuilder.kt index b4efd1073..960c94f33 100644 --- a/libp2p/src/main/kotlin/io/libp2p/transport/implementation/ConnectionBuilder.kt +++ b/libp2p/src/main/kotlin/io/libp2p/transport/implementation/ConnectionBuilder.kt @@ -32,7 +32,11 @@ class ConnectionBuilder( upgrader.establishSecureChannel(connection) .thenCompose { connection.setSecureSession(it) - upgrader.establishMuxer(connection) + if (it.earlyMuxer != null) { + ConnectionUpgrader.establishMuxer(it.earlyMuxer, connection) + } else { + upgrader.establishMuxer(connection) + } }.thenApply { connection.setMuxerSession(it) connHandler.handleConnection(connection) diff --git a/libp2p/src/test/java/io/libp2p/core/HostTestJava.java b/libp2p/src/test/java/io/libp2p/core/HostTestJava.java index b3e729112..f5e63059e 100644 --- a/libp2p/src/test/java/io/libp2p/core/HostTestJava.java +++ b/libp2p/src/test/java/io/libp2p/core/HostTestJava.java @@ -9,7 +9,7 @@ import io.libp2p.core.mux.StreamMuxerProtocol; import io.libp2p.protocol.Ping; import io.libp2p.protocol.PingController; -import io.libp2p.security.secio.SecIoSecureChannel; +import io.libp2p.security.tls.*; import io.libp2p.transport.tcp.TcpTransport; import kotlin.Pair; import org.junit.jupiter.api.Assertions; @@ -37,14 +37,14 @@ void ping() throws Exception { Host clientHost = new HostBuilder() .transport(TcpTransport::new) - .secureChannel(SecIoSecureChannel::new) - .muxer(StreamMuxerProtocol::getMplex) + .secureChannel((k, m) -> new TlsSecureChannel(k, m, "ECDSA")) + .muxer(StreamMuxerProtocol::getYamux) .build(); Host serverHost = new HostBuilder() .transport(TcpTransport::new) - .secureChannel(SecIoSecureChannel::new) - .muxer(StreamMuxerProtocol::getMplex) + .secureChannel(TlsSecureChannel::new) + .muxer(StreamMuxerProtocol::getYamux) .protocol(new Ping()) .listen(localListenAddress) .build(); @@ -93,6 +93,132 @@ void ping() throws Exception { System.out.println("Server stopped"); } + @Test + void largePing() throws Exception { + int pingSize = 200 * 1024; + String localListenAddress = "/ip4/127.0.0.1/tcp/40002"; + + Host clientHost = new HostBuilder() + .transport(TcpTransport::new) + .secureChannel((k, m) -> new TlsSecureChannel(k, m, "ECDSA")) + .muxer(StreamMuxerProtocol::getYamux) + .build(); + + Host serverHost = new HostBuilder() + .transport(TcpTransport::new) + .secureChannel(TlsSecureChannel::new) + .muxer(StreamMuxerProtocol::getYamux) + .protocol(new Ping(pingSize)) + .listen(localListenAddress) + .build(); + + CompletableFuture clientStarted = clientHost.start(); + CompletableFuture serverStarted = serverHost.start(); + clientStarted.get(5, TimeUnit.SECONDS); + System.out.println("Client started"); + serverStarted.get(5, TimeUnit.SECONDS); + System.out.println("Server started"); + + Assertions.assertEquals(0, clientHost.listenAddresses().size()); + Assertions.assertEquals(1, serverHost.listenAddresses().size()); + Assertions.assertEquals( + localListenAddress + "/p2p/" + serverHost.getPeerId(), + serverHost.listenAddresses().get(0).toString() + ); + + StreamPromise ping = + clientHost.getNetwork().connect( + serverHost.getPeerId(), + new Multiaddr(localListenAddress) + ).thenApply( + it -> it.muxerSession().createStream(new Ping(pingSize)) + ) + .join(); + + Stream pingStream = ping.getStream().get(5, TimeUnit.SECONDS); + System.out.println("Ping stream created"); + PingController pingCtr = ping.getController().get(5, TimeUnit.SECONDS); + System.out.println("Ping controller created"); + + for (int i = 0; i < 10; i++) { + long latency = pingCtr.ping().join();//get(5, TimeUnit.SECONDS); + System.out.println("Ping is " + latency); + } + pingStream.close().get(5, TimeUnit.SECONDS); + System.out.println("Ping stream closed"); + + Assertions.assertThrows(ExecutionException.class, () -> + pingCtr.ping().get(5, TimeUnit.SECONDS)); + + clientHost.stop().get(5, TimeUnit.SECONDS); + System.out.println("Client stopped"); + serverHost.stop().get(5, TimeUnit.SECONDS); + System.out.println("Server stopped"); + } + + @Test + void addPingAfterHostStart() throws Exception { + String localListenAddress = "/ip4/127.0.0.1/tcp/40002"; + + Host clientHost = new HostBuilder() + .transport(TcpTransport::new) + .secureChannel((k, m) -> new TlsSecureChannel(k, m, "ECDSA")) + .muxer(StreamMuxerProtocol::getYamux) + .build(); + + Host serverHost = new HostBuilder() + .transport(TcpTransport::new) + .secureChannel(TlsSecureChannel::new) + .muxer(StreamMuxerProtocol::getYamux) + .listen(localListenAddress) + .build(); + + CompletableFuture clientStarted = clientHost.start(); + CompletableFuture serverStarted = serverHost.start(); + clientStarted.get(5, TimeUnit.SECONDS); + System.out.println("Client started"); + serverStarted.get(5, TimeUnit.SECONDS); + System.out.println("Server started"); + + Assertions.assertEquals(0, clientHost.listenAddresses().size()); + Assertions.assertEquals(1, serverHost.listenAddresses().size()); + Assertions.assertEquals( + localListenAddress + "/p2p/" + serverHost.getPeerId(), + serverHost.listenAddresses().get(0).toString() + ); + + serverHost.addProtocolHandler(new Ping()); + + StreamPromise ping = + clientHost.getNetwork().connect( + serverHost.getPeerId(), + new Multiaddr(localListenAddress) + ).thenApply( + it -> it.muxerSession().createStream(new Ping()) + ) + .get(5, TimeUnit.SECONDS); + + Stream pingStream = ping.getStream().get(5, TimeUnit.SECONDS); + System.out.println("Ping stream created"); + PingController pingCtr = ping.getController().get(5, TimeUnit.SECONDS); + System.out.println("Ping controller created"); + + for (int i = 0; i < 10; i++) { + long latency = pingCtr.ping().get(1, TimeUnit.SECONDS); + System.out.println("Ping is " + latency); + } + pingStream.close().get(5, TimeUnit.SECONDS); + System.out.println("Ping stream closed"); + + Assertions.assertThrows(ExecutionException.class, () -> + pingCtr.ping().get(5, TimeUnit.SECONDS)); + + clientHost.stop().get(5, TimeUnit.SECONDS); + System.out.println("Client stopped"); + serverHost.stop().get(5, TimeUnit.SECONDS); + System.out.println("Server stopped"); + } + @Test void keyPairGeneration() { Pair pair = KeyKt.generateKeyPair(KEY_TYPE.SECP256K1); diff --git a/libp2p/src/test/kotlin/io/libp2p/core/HostTest.kt b/libp2p/src/test/kotlin/io/libp2p/core/HostTest.kt index f0ad484d9..eb4513f99 100644 --- a/libp2p/src/test/kotlin/io/libp2p/core/HostTest.kt +++ b/libp2p/src/test/kotlin/io/libp2p/core/HostTest.kt @@ -4,7 +4,7 @@ import io.libp2p.core.multistream.ProtocolMatcher import io.libp2p.etc.PROTOCOL import io.libp2p.etc.types.seconds import io.libp2p.etc.types.toByteArray -import io.libp2p.mux.MuxFrame +import io.libp2p.mux.mplex.MplexFrame import io.libp2p.protocol.Ping import io.libp2p.protocol.PingBinding import io.libp2p.protocol.PingProtocol @@ -131,7 +131,7 @@ class HostTest { val afterSecureTestHandler1 = TestByteBufChannelHandler("1-afterSecure") val preStreamTestHandler1 = TestByteBufChannelHandler("1-preStream") val streamTestHandler1 = TestByteBufChannelHandler("1-stream") - val muxFrameTestHandler1 = TestChannelHandler("1-mux") + val muxFrameTestHandler1 = TestChannelHandler("1-mux") hostFactory.hostBuilderModifier = { debug { @@ -148,7 +148,7 @@ class HostTest { val afterSecureTestHandler2 = TestByteBufChannelHandler("2-afterSecure") val preStreamTestHandler2 = TestByteBufChannelHandler("2-preStream") val streamTestHandler2 = TestByteBufChannelHandler("2-stream") - val muxFrameTestHandler2 = TestChannelHandler("2-mux") + val muxFrameTestHandler2 = TestChannelHandler("2-mux") hostFactory.hostBuilderModifier = { debug { diff --git a/libp2p/src/test/kotlin/io/libp2p/core/dsl/BuilderDefaultsTest.kt b/libp2p/src/test/kotlin/io/libp2p/core/dsl/BuilderDefaultsTest.kt index e2609512b..f7bc79b7a 100644 --- a/libp2p/src/test/kotlin/io/libp2p/core/dsl/BuilderDefaultsTest.kt +++ b/libp2p/src/test/kotlin/io/libp2p/core/dsl/BuilderDefaultsTest.kt @@ -46,7 +46,7 @@ class BuilderDefaultsTest { host(Builder.Defaults.None) { identity { random() } transports { +::TcpTransport } - secureChannels { +::SecIoSecureChannel } + secureChannels { add(::SecIoSecureChannel) } } } } @@ -56,7 +56,7 @@ class BuilderDefaultsTest { val host = host(Builder.Defaults.None) { identity { random() } transports { +::TcpTransport } - secureChannels { +::SecIoSecureChannel } + secureChannels { add(::SecIoSecureChannel) } muxers { + StreamMuxerProtocol.Mplex } } diff --git a/libp2p/src/test/kotlin/io/libp2p/core/multiformats/MultiaddrTest.kt b/libp2p/src/test/kotlin/io/libp2p/core/multiformats/MultiaddrTest.kt index a5ef55cea..e1c274aaf 100644 --- a/libp2p/src/test/kotlin/io/libp2p/core/multiformats/MultiaddrTest.kt +++ b/libp2p/src/test/kotlin/io/libp2p/core/multiformats/MultiaddrTest.kt @@ -133,7 +133,8 @@ class MultiaddrTest { "/ip4/1.2.3.4/tcp/80/unix/a/b/c/d/e/f", "/ip4/127.0.0.1/ipfs/QmcgpsyWgH8Y8ajJz1Cu72KnS5uo2Aa2LpzU7kinSupNKC/tcp/1234/unix/stdio", "/ip4/127.0.0.1/p2p/QmcgpsyWgH8Y8ajJz1Cu72KnS5uo2Aa2LpzU7kinSupNKC/tcp/1234/unix/stdio", - "/ip4/127.0.0.1/tcp/40001/p2p/16Uiu2HAkuqGKz8D6khfrnJnDrN5VxWWCoLU8Aq4eCFJuyXmfakB5" + "/ip4/127.0.0.1/tcp/40001/p2p/16Uiu2HAkuqGKz8D6khfrnJnDrN5VxWWCoLU8Aq4eCFJuyXmfakB5", + "/ip6/2001:6b0:30:1000:d00e:1dff:fe0b:c764/udp/4001/quic-v1/webtransport/certhash/uEiAEz_3prFf34VZff8XqA1iTdq2Ytp467ErTGr5dRFo60Q/certhash/uEiDyL7yksuIGJsYUvf0AHieLkTux5R5KBk-UsFtA1AG18A" ) @JvmStatic diff --git a/libp2p/src/test/kotlin/io/libp2p/crypto/KeyTypesTest.kt b/libp2p/src/test/kotlin/io/libp2p/crypto/KeyTypesTest.kt new file mode 100644 index 000000000..9a5b5805a --- /dev/null +++ b/libp2p/src/test/kotlin/io/libp2p/crypto/KeyTypesTest.kt @@ -0,0 +1,43 @@ +package io.libp2p.crypto + +import io.libp2p.crypto.keys.generateEcdsaKeyPair +import io.libp2p.crypto.keys.generateEd25519KeyPair +import io.libp2p.crypto.keys.generateRsaKeyPair +import io.libp2p.crypto.keys.generateSecp256k1KeyPair +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test + +class KeyTypesTest { + + @Test + fun ed25519() { + val pair = generateEd25519KeyPair() + val toSign = "G'day!".toByteArray() + val signed = pair.first.sign(toSign) + assertTrue(pair.second.verify(toSign, signed)) + } + + @Test + fun rsa() { + val pair = generateRsaKeyPair(2048) + val toSign = "G'day!".toByteArray() + val signed = pair.first.sign(toSign) + assertTrue(pair.second.verify(toSign, signed)) + } + + @Test + fun secp256k1() { + val pair = generateSecp256k1KeyPair() + val toSign = "G'day!".toByteArray() + val signed = pair.first.sign(toSign) + assertTrue(pair.second.verify(toSign, signed)) + } + + @Test + fun ecdsa() { + val pair = generateEcdsaKeyPair() // p-256 + val toSign = "G'day!".toByteArray() + val signed = pair.first.sign(toSign) + assertTrue(pair.second.verify(toSign, signed)) + } +} diff --git a/libp2p/src/test/kotlin/io/libp2p/mux/MultiplexHandlerTest.kt b/libp2p/src/test/kotlin/io/libp2p/mux/MultiplexHandlerTest.kt deleted file mode 100644 index 5a8e18013..000000000 --- a/libp2p/src/test/kotlin/io/libp2p/mux/MultiplexHandlerTest.kt +++ /dev/null @@ -1,306 +0,0 @@ -package io.libp2p.mux - -import io.libp2p.core.ConnectionClosedException -import io.libp2p.core.Libp2pException -import io.libp2p.core.Stream -import io.libp2p.core.StreamHandler -import io.libp2p.core.multistream.MultistreamProtocolV1 -import io.libp2p.etc.types.fromHex -import io.libp2p.etc.types.getX -import io.libp2p.etc.types.toByteArray -import io.libp2p.etc.types.toByteBuf -import io.libp2p.etc.types.toHex -import io.libp2p.etc.util.netty.mux.MuxId -import io.libp2p.etc.util.netty.nettyInitializer -import io.libp2p.mux.MuxFrame.Flag.DATA -import io.libp2p.mux.MuxFrame.Flag.OPEN -import io.libp2p.mux.MuxFrame.Flag.RESET -import io.libp2p.mux.mplex.DEFAULT_MAX_MPLEX_FRAME_DATA_LENGTH -import io.libp2p.mux.mplex.MplexHandler -import io.libp2p.tools.TestChannel -import io.netty.buffer.ByteBuf -import io.netty.channel.ChannelHandler -import io.netty.channel.ChannelHandlerContext -import io.netty.channel.ChannelInboundHandlerAdapter -import io.netty.channel.DefaultChannelId -import io.netty.handler.logging.LogLevel -import io.netty.handler.logging.LoggingHandler -import org.assertj.core.api.Assertions.assertThat -import org.junit.jupiter.api.Assertions.assertEquals -import org.junit.jupiter.api.Assertions.assertFalse -import org.junit.jupiter.api.Assertions.assertThrows -import org.junit.jupiter.api.Assertions.assertTrue -import org.junit.jupiter.api.BeforeEach -import org.junit.jupiter.api.Test -import java.util.concurrent.CompletableFuture - -/** - * Created by Anton Nashatyrev on 09.07.2019. - */ -class MultiplexHandlerTest { - val dummyParentChannelId = DefaultChannelId.newInstance() - val childHandlers = mutableListOf() - lateinit var multistreamHandler: MuxHandler - lateinit var ech: TestChannel - - @BeforeEach - fun startMultiplexor() { - childHandlers.clear() - val streamHandler = createStreamHandler( - nettyInitializer { - println("New child channel created") - val handler = TestHandler() - it.addLastLocal(handler) - childHandlers += handler - } - ) - multistreamHandler = object : MplexHandler( - MultistreamProtocolV1, DEFAULT_MAX_MPLEX_FRAME_DATA_LENGTH, null, streamHandler - ) { - // MuxHandler consumes the exception. Override this behaviour for testing - @Deprecated("Deprecated in Java") - override fun exceptionCaught(ctx: ChannelHandlerContext, cause: Throwable) { - ctx.fireExceptionCaught(cause) - } - } - - ech = TestChannel("test", true, LoggingHandler(LogLevel.ERROR), multistreamHandler) - } - - @Test - fun singleStream() { - openStream(12) - assertHandlerCount(1) - - writeStream(12, "22") - assertHandlerCount(1) - assertEquals(1, childHandlers[0].inboundMessages.size) - assertEquals("22", childHandlers[0].inboundMessages.last()) - - writeStream(12, "44") - assertHandlerCount(1) - assertEquals(2, childHandlers[0].inboundMessages.size) - assertEquals("44", childHandlers[0].inboundMessages.last()) - - writeStream(12, "66") - assertHandlerCount(1) - assertEquals(3, childHandlers[0].inboundMessages.size) - assertEquals("66", childHandlers[0].inboundMessages.last()) - } - - @Test - fun `test that readComplete event is fired to child channel`() { - openStream(12) - - assertThat(childHandlers[0].readCompleteEventCount).isZero() - - writeStream(12, "22") - - assertThat(childHandlers[0].readCompleteEventCount).isEqualTo(1) - - writeStream(12, "23") - - assertThat(childHandlers[0].readCompleteEventCount).isEqualTo(2) - } - - @Test - fun `test that readComplete event is fired to reading channels only`() { - openStream(12) - openStream(13) - - assertThat(childHandlers[0].readCompleteEventCount).isZero() - assertThat(childHandlers[1].readCompleteEventCount).isZero() - - writeStream(12, "22") - - assertThat(childHandlers[0].readCompleteEventCount).isEqualTo(1) - assertThat(childHandlers[1].readCompleteEventCount).isEqualTo(0) - - writeStream(13, "23") - - assertThat(childHandlers[0].readCompleteEventCount).isEqualTo(1) - assertThat(childHandlers[1].readCompleteEventCount).isEqualTo(1) - } - - @Test - fun twoStreamsInterleaved() { - openStream(12) - writeStream(12, "22") - - assertHandlerCount(1) - assertLastMessage(0, 1, "22") - - writeStream(12, "23") - assertHandlerCount(1) - assertLastMessage(0, 2, "23") - - openStream(22) - writeStream(22, "33") - assertHandlerCount(2) - assertLastMessage(1, 1, "33") - - writeStream(12, "24") - assertHandlerCount(2) - assertLastMessage(0, 3, "24") - - writeStream(22, "34") - assertHandlerCount(2) - assertLastMessage(1, 2, "34") - } - - @Test - fun twoStreamsSequential() { - openStream(12) - writeStream(12, "22") - - assertHandlerCount(1) - assertLastMessage(0, 1, "22") - - writeStream(12, "23") - assertHandlerCount(1) - assertLastMessage(0, 2, "23") - - writeStream(12, "24") - assertHandlerCount(1) - assertLastMessage(0, 3, "24") - - writeStream(12, "25") - assertHandlerCount(1) - assertLastMessage(0, 4, "25") - - resetStream(12) - assertHandlerCount(1) - - openStream(22) - writeStream(22, "33") - assertHandlerCount(2) - assertLastMessage(1, 1, "33") - - writeStream(22, "34") - assertHandlerCount(2) - assertLastMessage(1, 2, "34") - - resetStream(12) - assertHandlerCount(2) - } - - @Test - fun streamIsReset() { - openStream(22) - assertFalse(childHandlers[0].ctx!!.channel().closeFuture().isDone) - - resetStream(22) - assertTrue(childHandlers[0].ctx!!.channel().closeFuture().isDone) - } - - @Test - fun streamIsResetWhenChannelIsClosed() { - openStream(22) - assertFalse(childHandlers[0].ctx!!.channel().closeFuture().isDone) - - ech.close().await() - - assertTrue(childHandlers[0].ctx!!.channel().closeFuture().isDone) - } - - @Test - fun cantWriteToResetStream() { - openStream(18) - resetStream(18) - - assertThrows(Libp2pException::class.java) { - writeStream(18, "35") - } - } - - @Test - fun cantWriteToNonExistentStream() { - assertThrows(Libp2pException::class.java) { - writeStream(92, "35") - } - } - - @Test - fun canResetNonExistentStream() { - resetStream(99) - } - - @Test - fun cantOpenStreamOnClosedChannel() { - ech.close().await() - - val staleStream = - multistreamHandler.createStream { - println("This shouldn't be displayed: parent stream is closed") - CompletableFuture.completedFuture(Unit) - } - - assertThrows(ConnectionClosedException::class.java) { staleStream.stream.getX(3.0) } - } - - fun assertHandlerCount(count: Int) = assertEquals(count, childHandlers.size) - fun assertLastMessage(handler: Int, msgCount: Int, msg: String) { - val messages = childHandlers[handler].inboundMessages - assertEquals(msgCount, messages.size) - assertEquals(msg, messages.last()) - } - - fun openStream(id: Long) = writeFrame(id, OPEN) - fun writeStream(id: Long, msg: String) = writeFrame(id, DATA, msg.fromHex().toByteBuf()) - fun resetStream(id: Long) = writeFrame(id, RESET) - fun writeFrame(id: Long, flag: MuxFrame.Flag, data: ByteBuf? = null) = - ech.writeInbound(MuxFrame(MuxId(dummyParentChannelId, id, true), flag, data)) - - fun createStreamHandler(channelInitializer: ChannelHandler) = object : StreamHandler { - override fun handleStream(stream: Stream): CompletableFuture { - stream.pushHandler(channelInitializer) - return CompletableFuture.completedFuture(Unit) - } - } - - class TestHandler : ChannelInboundHandlerAdapter() { - val inboundMessages = mutableListOf() - var ctx: ChannelHandlerContext? = null - var readCompleteEventCount = 0 - - override fun channelInactive(ctx: ChannelHandlerContext?) { - println("MultiplexHandlerTest.channelInactive") - } - - override fun channelRead(ctx: ChannelHandlerContext?, msg: Any?) { - println("MultiplexHandlerTest.channelRead") - msg as ByteBuf - inboundMessages += msg.toByteArray().toHex() - } - - override fun channelUnregistered(ctx: ChannelHandlerContext?) { - println("MultiplexHandlerTest.channelUnregistered") - } - - override fun channelActive(ctx: ChannelHandlerContext?) { - println("MultiplexHandlerTest.channelActive") - } - - override fun channelRegistered(ctx: ChannelHandlerContext?) { - println("MultiplexHandlerTest.channelRegistered") - } - - override fun channelReadComplete(ctx: ChannelHandlerContext?) { - readCompleteEventCount++ - println("MultiplexHandlerTest.channelReadComplete") - } - - override fun handlerAdded(ctx: ChannelHandlerContext?) { - println("MultiplexHandlerTest.handlerAdded") - this.ctx = ctx - } - - override fun exceptionCaught(ctx: ChannelHandlerContext?, cause: Throwable?) { - println("MultiplexHandlerTest.exceptionCaught") - } - - override fun handlerRemoved(ctx: ChannelHandlerContext?) { - println("MultiplexHandlerTest.handlerRemoved") - } - } -} diff --git a/libp2p/src/test/kotlin/io/libp2p/mux/MuxHandlerAbstractTest.kt b/libp2p/src/test/kotlin/io/libp2p/mux/MuxHandlerAbstractTest.kt new file mode 100644 index 000000000..83792b559 --- /dev/null +++ b/libp2p/src/test/kotlin/io/libp2p/mux/MuxHandlerAbstractTest.kt @@ -0,0 +1,547 @@ +package io.libp2p.mux + +import io.libp2p.core.ConnectionClosedException +import io.libp2p.core.Libp2pException +import io.libp2p.core.StreamHandler +import io.libp2p.etc.types.fromHex +import io.libp2p.etc.types.getX +import io.libp2p.etc.types.toHex +import io.libp2p.etc.util.netty.mux.MuxId +import io.libp2p.etc.util.netty.mux.RemoteWriteClosed +import io.libp2p.etc.util.netty.nettyInitializer +import io.libp2p.mux.MuxHandlerAbstractTest.AbstractTestMuxFrame.Flag.* +import io.libp2p.mux.MuxHandlerAbstractTest.TestEventHandler +import io.libp2p.tools.TestChannel +import io.libp2p.tools.readAllBytesAndRelease +import io.netty.buffer.ByteBuf +import io.netty.buffer.Unpooled +import io.netty.channel.ChannelHandlerContext +import io.netty.channel.ChannelInboundHandlerAdapter +import io.netty.handler.logging.LogLevel +import io.netty.handler.logging.LoggingHandler +import org.assertj.core.api.Assertions.assertThat +import org.assertj.core.data.Index +import org.junit.jupiter.api.AfterEach +import org.junit.jupiter.api.Assertions.* +import org.junit.jupiter.api.BeforeEach +import org.junit.jupiter.api.Test +import java.util.concurrent.CompletableFuture + +/** + * Created by Anton Nashatyrev on 09.07.2019. + */ +abstract class MuxHandlerAbstractTest { + val childHandlers = mutableListOf() + lateinit var multistreamHandler: MuxHandler + lateinit var ech: TestChannel + val parentChannelId get() = ech.id() + + val allocatedBufs = mutableListOf() + val activeEventHandlers = mutableListOf() + + abstract val maxFrameDataLength: Int + abstract fun createMuxHandler(streamHandler: StreamHandler<*>): MuxHandler + + fun createTestStreamHandler(): StreamHandler = + StreamHandler { stream -> + val handler = TestHandler() + stream.pushHandler( + nettyInitializer { + it.addLastLocal(handler) + } + ) + CompletableFuture.completedFuture(handler) + } + + fun StreamHandler.onNewStream(block: (T) -> Unit): StreamHandler = + StreamHandler { stream -> + this.handleStream(stream) + .thenApply { + block(it) + it + } + } + + @BeforeEach + fun startMultiplexor() { + val streamHandler = createTestStreamHandler() + .onNewStream { + childHandlers += it + } + multistreamHandler = createMuxHandler(streamHandler) + + ech = TestChannel("test", true, LoggingHandler(LogLevel.ERROR), multistreamHandler) + } + + @AfterEach + open fun cleanUpAndCheck() { + childHandlers.forEach { + assertThat(it.exceptions).isEmpty() + } + childHandlers.clear() + + allocatedBufs.forEach { + assertThat(it.refCnt()).isEqualTo(1) + } + allocatedBufs.clear() + } + + data class AbstractTestMuxFrame( + val streamId: Long, + val flag: Flag, + val data: String = "" + ) { + enum class Flag { Open, Data, Close, Reset } + } + + fun Long.toMuxId() = MuxId(parentChannelId, this, true) + + abstract fun writeFrame(frame: AbstractTestMuxFrame) + abstract fun readFrame(): AbstractTestMuxFrame? + fun readFrameOrThrow() = readFrame() ?: throw AssertionError("No outbound frames") + fun openStream(id: Long) = writeFrame(AbstractTestMuxFrame(id, Open)) + fun writeStream(id: Long, msg: String) = writeFrame(AbstractTestMuxFrame(id, Data, msg)) + fun closeStream(id: Long) = writeFrame(AbstractTestMuxFrame(id, Close)) + fun resetStream(id: Long) = writeFrame(AbstractTestMuxFrame(id, Reset)) + + fun openStreamByLocal(): TestHandler { + val handlerFut = multistreamHandler.createStream(createTestStreamHandler()).controller + ech.runPendingTasks() + return handlerFut.get() + } + + protected fun allocateBuf(): ByteBuf { + val buf = Unpooled.buffer() + buf.retain() // ref counter to 2 to check that exactly 1 ref remains at the end + allocatedBufs += buf + return buf + } + + protected fun allocateMessage(hexBytes: String) = hexBytes.fromHex().toByteBuf(allocateBuf()) + + fun assertHandlerCount(count: Int) = assertEquals(count, childHandlers.size) + fun assertLastMessage(handler: Int, msgCount: Int, msg: String) { + val messages = childHandlers[handler].inboundMessages + assertEquals(msgCount, messages.size) + assertEquals(msg, messages.last()) + } + + @Test + fun singleStream() { + openStream(12) + assertHandlerCount(1) + assertTrue(childHandlers[0].isActivated) + + writeStream(12, "22") + assertHandlerCount(1) + assertEquals(1, childHandlers[0].inboundMessages.size) + assertEquals("22", childHandlers[0].inboundMessages.last()) + + writeStream(12, "44") + assertHandlerCount(1) + assertEquals(2, childHandlers[0].inboundMessages.size) + assertEquals("44", childHandlers[0].inboundMessages.last()) + + writeStream(12, "66") + assertHandlerCount(1) + assertEquals(3, childHandlers[0].inboundMessages.size) + assertEquals("66", childHandlers[0].inboundMessages.last()) + + assertFalse(childHandlers[0].isInactivated) + } + + @Test + fun `test that readComplete event is fired to child channel`() { + openStream(12) + + assertThat(childHandlers[0].readCompleteEventCount).isZero() + + writeStream(12, "22") + + assertThat(childHandlers[0].readCompleteEventCount).isEqualTo(1) + + writeStream(12, "23") + + assertThat(childHandlers[0].readCompleteEventCount).isEqualTo(2) + } + + @Test + fun `test that readComplete event is fired to reading channels only`() { + openStream(12) + openStream(13) + + assertThat(childHandlers[0].readCompleteEventCount).isZero() + assertThat(childHandlers[1].readCompleteEventCount).isZero() + + writeStream(12, "22") + + assertThat(childHandlers[0].readCompleteEventCount).isEqualTo(1) + assertThat(childHandlers[1].readCompleteEventCount).isEqualTo(0) + + writeStream(13, "23") + + assertThat(childHandlers[0].readCompleteEventCount).isEqualTo(1) + assertThat(childHandlers[1].readCompleteEventCount).isEqualTo(1) + } + + @Test + fun twoStreamsInterleaved() { + openStream(12) + writeStream(12, "22") + + assertHandlerCount(1) + assertLastMessage(0, 1, "22") + + writeStream(12, "23") + assertHandlerCount(1) + assertLastMessage(0, 2, "23") + + openStream(22) + writeStream(22, "33") + assertHandlerCount(2) + assertLastMessage(1, 1, "33") + + writeStream(12, "24") + assertHandlerCount(2) + assertLastMessage(0, 3, "24") + + writeStream(22, "34") + assertHandlerCount(2) + assertLastMessage(1, 2, "34") + + assertFalse(childHandlers[0].isInactivated) + assertFalse(childHandlers[1].isInactivated) + } + + @Test + fun twoStreamsSequential() { + openStream(12) + writeStream(12, "22") + + assertHandlerCount(1) + assertLastMessage(0, 1, "22") + + writeStream(12, "23") + assertHandlerCount(1) + assertLastMessage(0, 2, "23") + + writeStream(12, "24") + assertHandlerCount(1) + assertLastMessage(0, 3, "24") + + writeStream(12, "25") + assertHandlerCount(1) + assertLastMessage(0, 4, "25") + + assertFalse(childHandlers[0].isInactivated) + resetStream(12) + assertTrue(childHandlers[0].isHandlerRemoved) + + openStream(22) + writeStream(22, "33") + assertHandlerCount(2) + assertLastMessage(1, 1, "33") + + writeStream(22, "34") + assertHandlerCount(2) + assertLastMessage(1, 2, "34") + + assertFalse(childHandlers[1].isInactivated) + resetStream(22) + assertTrue(childHandlers[1].isHandlerRemoved) + } + + @Test + fun streamIsReset() { + openStream(22) + assertFalse(childHandlers[0].ctx.channel().closeFuture().isDone) + assertFalse(childHandlers[0].isInactivated) + + resetStream(22) + assertTrue(childHandlers[0].ctx.channel().closeFuture().isDone) + assertTrue(childHandlers[0].isHandlerRemoved) + } + + @Test + fun streamIsResetWhenChannelIsClosed() { + openStream(22) + assertFalse(childHandlers[0].ctx.channel().closeFuture().isDone) + + ech.close().await() + + assertTrue(childHandlers[0].ctx.channel().closeFuture().isDone) + assertTrue(childHandlers[0].isHandlerRemoved) + } + + @Test + fun cantReceiveOnResetStream() { + openStream(18) + resetStream(18) + + assertThrows(Libp2pException::class.java) { + writeStream(18, "35") + } + assertTrue(childHandlers[0].isHandlerRemoved) + } + + @Test + fun cantReceiveOnClosedStream() { + openStream(18) + closeStream(18) + + assertThrows(Libp2pException::class.java) { + writeStream(18, "35") + } + assertFalse(childHandlers[0].isInactivated) + } + + @Test + fun cantReceiveOnNonExistentStream() { + assertThrows(Libp2pException::class.java) { + writeStream(92, "35") + } + assertHandlerCount(0) + } + + @Test + fun canResetNonExistentStream() { + resetStream(99) + assertHandlerCount(0) + } + + @Test + fun cantOpenStreamOnClosedChannel() { + ech.close().await() + + val staleStream = + multistreamHandler.createStream { + println("This shouldn't be displayed: parent stream is closed") + CompletableFuture.completedFuture(Unit) + } + + assertThrows(ConnectionClosedException::class.java) { staleStream.stream.getX(3.0) } + assertHandlerCount(0) + } + + @Test + fun `local create and after local disconnect should still read`() { + val handler = openStreamByLocal() + handler.ctx.writeAndFlush(allocateMessage("1984")) + handler.ctx.disconnect().sync() + + val openFrame = readFrameOrThrow() + assertThat(openFrame.flag).isEqualTo(Open) + + val dataFrame = readFrameOrThrow() + assertThat(dataFrame.flag).isEqualTo(Data) + assertThat(dataFrame.streamId).isEqualTo(openFrame.streamId) + + val closeFrame = readFrameOrThrow() + assertThat(closeFrame.flag).isEqualTo(Close) + + assertThat(readFrame()).isNull() + assertThat(handler.isInactivated).isTrue() + assertThat(handler.isUnregistered).isFalse() + assertThat(handler.inboundMessages).isEmpty() + + writeStream(dataFrame.streamId, "1122") + assertThat(handler.inboundMessages).isNotEmpty + } + + @Test + fun `local create and after remote disconnect should still write`() { + val handler = openStreamByLocal() + + val openFrame = readFrameOrThrow() + assertThat(openFrame.flag).isEqualTo(Open) + assertThat(readFrame()).isNull() + + closeStream(openFrame.streamId) + + assertThat(handler.isInactivated).isFalse() + assertThat(handler.isUnregistered).isFalse() + assertThat(handler.userEvents).containsExactly(RemoteWriteClosed) + + handler.ctx.writeAndFlush(allocateMessage("1984")) + + val readFrame = readFrameOrThrow() + assertThat(readFrame.flag).isEqualTo(Data) + assertThat(readFrame.data).isEqualTo("1984") + assertThat(readFrame()).isNull() + } + + @Test + fun `test remote and local disconnect closes stream`() { + val handler = openStreamByLocal() + handler.ctx.disconnect().sync() + + readFrameOrThrow() + val closeFrame = readFrameOrThrow() + assertThat(closeFrame.flag).isEqualTo(Close) + + assertThat(handler.isInactivated).isTrue() + assertThat(handler.isUnregistered).isFalse() + + closeStream(closeFrame.streamId) + + assertThat(handler.isHandlerRemoved).isTrue() + } + + @Test + fun `test large message is split onto slices`() { + val handler = openStreamByLocal() + readFrameOrThrow() + + val largeMessage = "42".repeat(maxFrameDataLength - 1) + "4344" + handler.ctx.writeAndFlush(allocateMessage(largeMessage)) + + val dataFrame1 = readFrameOrThrow() + assertThat(dataFrame1.data.fromHex()) + .hasSize(maxFrameDataLength) + .contains(0x42, Index.atIndex(0)) + .contains(0x42, Index.atIndex(maxFrameDataLength - 2)) + .contains(0x43, Index.atIndex(maxFrameDataLength - 1)) + + val dataFrame2 = readFrameOrThrow() + assertThat(dataFrame2.data.fromHex()) + .hasSize(1) + .contains(0x44, Index.atIndex(0)) + + assertThat(readFrame()).isNull() + } + + @Test + fun `should throw when writing to locally closed stream`() { + val handler = openStreamByLocal() + handler.ctx.disconnect() + + assertThrows(Exception::class.java) { + handler.ctx.writeAndFlush(allocateMessage("42")).sync() + } + } + + @Test + fun `should throw when writing to reset stream`() { + val handler = openStreamByLocal() + handler.ctx.close() + + assertThrows(Exception::class.java) { + handler.ctx.writeAndFlush(allocateMessage("42")).sync() + } + } + + @Test + fun `should throw when writing to closed connection`() { + val handler = openStreamByLocal() + ech.close().sync() + + assertThrows(Exception::class.java) { + handler.ctx.writeAndFlush(allocateMessage("42")).sync() + } + } + + @Test + fun `test writing to remotely open stream upon activation`() { + activeEventHandlers += TestEventHandler { + val writePromise = it.ctx.writeAndFlush(allocateMessage("42")) + writePromise.sync() + } + openStream(33) + + val dataFrame = readFrameOrThrow() + assertThat(dataFrame.streamId).isEqualTo(33) + assertThat(dataFrame.data).isEqualTo("42") + } + + fun interface TestEventHandler { + fun handle(testHandler: TestHandler) + } + + inner class TestHandler : ChannelInboundHandlerAdapter() { + val inboundMessages = mutableListOf() + lateinit var ctx: ChannelHandlerContext + var readCompleteEventCount = 0 + + val exceptions = mutableListOf() + val userEvents = mutableListOf() + var isHandlerAdded = false + var isRegistered = false + var isActivated = false + var isInactivated = false + var isUnregistered = false + var isHandlerRemoved = false + + init { + println("New child channel created") + } + + override fun handlerAdded(ctx: ChannelHandlerContext) { + assertFalse(isHandlerAdded) + isHandlerAdded = true + println("MuxHandlerAbstractTest.handlerAdded") + this.ctx = ctx + } + + override fun channelRegistered(ctx: ChannelHandlerContext?) { + assertTrue(isHandlerAdded) + assertFalse(isRegistered) + isRegistered = true + println("MuxHandlerAbstractTest.channelRegistered") + } + + override fun channelActive(ctx: ChannelHandlerContext) { + assertTrue(isRegistered) + assertFalse(isActivated) + isActivated = true + println("MuxHandlerAbstractTest.channelActive") + activeEventHandlers.forEach { it.handle(this) } + } + + override fun channelRead(ctx: ChannelHandlerContext, msg: Any) { + assertTrue(isActivated) + println("MuxHandlerAbstractTest.channelRead") + msg as ByteBuf + inboundMessages += msg.readAllBytesAndRelease().toHex() + } + + override fun channelReadComplete(ctx: ChannelHandlerContext?) { + readCompleteEventCount++ + println("MuxHandlerAbstractTest.channelReadComplete") + } + + override fun userEventTriggered(ctx: ChannelHandlerContext, evt: Any) { + userEvents += evt + println("MuxHandlerAbstractTest.userEventTriggered: $evt") + } + + override fun exceptionCaught(ctx: ChannelHandlerContext, cause: Throwable) { + exceptions += cause + println("MuxHandlerAbstractTest.exceptionCaught") + } + + override fun channelInactive(ctx: ChannelHandlerContext) { + assertTrue(isActivated) + assertFalse(isInactivated) + isInactivated = true + println("MuxHandlerAbstractTest.channelInactive") + } + + override fun channelUnregistered(ctx: ChannelHandlerContext?) { + assertTrue(isInactivated) + assertFalse(isUnregistered) + isUnregistered = true + println("MuxHandlerAbstractTest.channelUnregistered") + } + + override fun handlerRemoved(ctx: ChannelHandlerContext?) { + assertTrue(isUnregistered) + assertFalse(isHandlerRemoved) + isHandlerRemoved = true + println("MuxHandlerAbstractTest.handlerRemoved") + } + } + + companion object { + fun ByteArray.toByteBuf(buf: ByteBuf): ByteBuf = buf.writeBytes(this) + } +} diff --git a/libp2p/src/test/kotlin/io/libp2p/mux/mplex/MplexFrameCodecTest.kt b/libp2p/src/test/kotlin/io/libp2p/mux/mplex/MplexFrameCodecTest.kt index 031350c55..6ba816250 100644 --- a/libp2p/src/test/kotlin/io/libp2p/mux/mplex/MplexFrameCodecTest.kt +++ b/libp2p/src/test/kotlin/io/libp2p/mux/mplex/MplexFrameCodecTest.kt @@ -30,20 +30,20 @@ class MplexFrameCodecTest { ) } val dummyId = DefaultChannelId.newInstance() + val maxFrameDataLength = 1024 + val channel = EmbeddedChannel(MplexFrameCodec(maxFrameDataLength = maxFrameDataLength)) @Test fun `check max frame size limit`() { - val channelLarge = EmbeddedChannel(MplexFrameCodec(maxFrameDataLength = 1024)) - val mplexFrame = MplexFrame( - MuxId(dummyId, 777, true), MplexFlags.MessageInitiator, - ByteArray(1024).toByteBuf() + MuxId(dummyId, 777, true), MplexFlag.MessageInitiator, + ByteArray(maxFrameDataLength).toByteBuf() ) assertTrue( - channelLarge.writeOutbound(mplexFrame) + channel.writeOutbound(mplexFrame) ) - val largeFrameBytes = channelLarge.readOutbound() + val largeFrameBytes = channel.readOutbound() val largeFrameBytesTrunc = largeFrameBytes.slice(0, largeFrameBytes.readableBytes() - 1) val channelSmall = EmbeddedChannel(MplexFrameCodec(maxFrameDataLength = 128)) @@ -58,12 +58,10 @@ class MplexFrameCodecTest { @ParameterizedTest @MethodSource("splitIndexes") fun testDecoder(sliceIdx: List) { - val channel = EmbeddedChannel(MplexFrameCodec()) - val mplexFrames = arrayOf( - MplexFrame(MuxId(dummyId, 777, true), MplexFlags.MessageInitiator, "Hello-1".toByteArray().toByteBuf()), - MplexFrame(MuxId(dummyId, 888, true), MplexFlags.MessageInitiator, "Hello-2".toByteArray().toByteBuf()), - MplexFrame(MuxId(dummyId, 999, true), MplexFlags.MessageInitiator, "Hello-3".toByteArray().toByteBuf()) + MplexFrame(MuxId(dummyId, 777, true), MplexFlag.MessageInitiator, "Hello-1".toByteArray().toByteBuf()), + MplexFrame(MuxId(dummyId, 888, true), MplexFlag.MessageInitiator, "Hello-2".toByteArray().toByteBuf()), + MplexFrame(MuxId(dummyId, 999, true), MplexFlag.MessageInitiator, "Hello-3".toByteArray().toByteBuf()) ) assertTrue( channel.writeOutbound(*mplexFrames) @@ -86,8 +84,51 @@ class MplexFrameCodecTest { assertEquals(777, resultFrames[0].id.id) assertEquals(888, resultFrames[1].id.id) assertEquals(999, resultFrames[2].id.id) - assertEquals("Hello-1", resultFrames[0].data!!.toByteArray().toString(UTF_8)) - assertEquals("Hello-2", resultFrames[1].data!!.toByteArray().toString(UTF_8)) - assertEquals("Hello-3", resultFrames[2].data!!.toByteArray().toString(UTF_8)) + assertEquals("Hello-1", resultFrames[0].data.toByteArray().toString(UTF_8)) + assertEquals("Hello-2", resultFrames[1].data.toByteArray().toString(UTF_8)) + assertEquals("Hello-3", resultFrames[2].data.toByteArray().toString(UTF_8)) + } + + @Test + fun `test id initiator is inverted on decoding`() { + val mplexFrames = arrayOf( + MplexFrame.createOpenFrame(MuxId(dummyId, 1, true)), + MplexFrame.createDataFrame(MuxId(dummyId, 2, true), "Hello-2".toByteArray().toByteBuf()), + MplexFrame.createDataFrame(MuxId(dummyId, 3, false), "Hello-3".toByteArray().toByteBuf()), + MplexFrame.createCloseFrame(MuxId(dummyId, 4, true)), + MplexFrame.createCloseFrame(MuxId(dummyId, 5, false)), + MplexFrame.createResetFrame(MuxId(dummyId, 6, true)), + MplexFrame.createResetFrame(MuxId(dummyId, 7, false)), + ) + assertTrue( + channel.writeOutbound(*mplexFrames) + ) + + repeat(mplexFrames.size) { idx -> + val wireBytes = channel.readOutbound() + channel.writeInbound(wireBytes) + val resFrame = channel.readInbound() + + assertEquals(mplexFrames[idx].id.id, resFrame.id.id) + assertEquals(!mplexFrames[idx].id.initiator, resFrame.id.initiator) + assertEquals(mplexFrames[idx].flag, resFrame.flag) + } + } + + @Test + fun `check the frame underlying buffer is released after send`() { + val frameDataBuf = "Hello-1".toByteArray().toByteBuf() + + assertTrue(frameDataBuf.refCnt() == 1) + + channel.writeOutbound( + MplexFrame(MuxId(dummyId, 777, true), MplexFlag.MessageInitiator, frameDataBuf) + ) + + val encodedFrame = channel.readOutbound() + // bytes are released after sending to the wire + encodedFrame.release() + + assertTrue(frameDataBuf.refCnt() == 0) } } diff --git a/libp2p/src/test/kotlin/io/libp2p/mux/mplex/MplexHandlerTest.kt b/libp2p/src/test/kotlin/io/libp2p/mux/mplex/MplexHandlerTest.kt new file mode 100644 index 000000000..bd9fd88fd --- /dev/null +++ b/libp2p/src/test/kotlin/io/libp2p/mux/mplex/MplexHandlerTest.kt @@ -0,0 +1,59 @@ +package io.libp2p.mux.mplex + +import io.libp2p.core.StreamHandler +import io.libp2p.core.multistream.MultistreamProtocolV1 +import io.libp2p.etc.types.fromHex +import io.libp2p.etc.types.toHex +import io.libp2p.mux.MuxHandler +import io.libp2p.mux.MuxHandlerAbstractTest +import io.libp2p.mux.MuxHandlerAbstractTest.AbstractTestMuxFrame.Flag.* +import io.libp2p.tools.readAllBytesAndRelease +import io.netty.buffer.Unpooled +import io.netty.channel.ChannelHandlerContext + +class MplexHandlerTest : MuxHandlerAbstractTest() { + + override val maxFrameDataLength = 256 + + override fun createMuxHandler(streamHandler: StreamHandler<*>): MuxHandler = + object : MplexHandler( + MultistreamProtocolV1, maxFrameDataLength, null, streamHandler + ) { + // MuxHandler consumes the exception. Override this behaviour for testing + @Deprecated("Deprecated in Java") + override fun exceptionCaught(ctx: ChannelHandlerContext, cause: Throwable) { + ctx.fireExceptionCaught(cause) + } + } + + override fun writeFrame(frame: AbstractTestMuxFrame) { + val muxId = frame.streamId.toMuxId() + val mplexFlag = when (frame.flag) { + Open -> MplexFlag.Type.OPEN + Data -> MplexFlag.Type.DATA + Close -> MplexFlag.Type.CLOSE + Reset -> MplexFlag.Type.RESET + } + val data = when { + frame.data.isEmpty() -> Unpooled.EMPTY_BUFFER + else -> frame.data.fromHex().toByteBuf(allocateBuf()) + } + val mplexFrame = + MplexFrame(muxId, MplexFlag.getByType(mplexFlag, true), data) + ech.writeInbound(mplexFrame) + } + + override fun readFrame(): AbstractTestMuxFrame? { + val maybeMplexFrame = ech.readOutbound() + return maybeMplexFrame?.let { mplexFrame -> + val flag = when (mplexFrame.flag.type) { + MplexFlag.Type.OPEN -> Open + MplexFlag.Type.DATA -> Data + MplexFlag.Type.CLOSE -> Close + MplexFlag.Type.RESET -> Reset + } + val data = maybeMplexFrame.data.readAllBytesAndRelease().toHex() + AbstractTestMuxFrame(mplexFrame.id.id, flag, data) + } + } +} diff --git a/libp2p/src/test/kotlin/io/libp2p/mux/yamux/YamuxHandlerTest.kt b/libp2p/src/test/kotlin/io/libp2p/mux/yamux/YamuxHandlerTest.kt new file mode 100644 index 000000000..640dc8d40 --- /dev/null +++ b/libp2p/src/test/kotlin/io/libp2p/mux/yamux/YamuxHandlerTest.kt @@ -0,0 +1,188 @@ +package io.libp2p.mux.yamux + +import io.libp2p.core.StreamHandler +import io.libp2p.core.multistream.MultistreamProtocolV1 +import io.libp2p.etc.types.fromHex +import io.libp2p.etc.types.toHex +import io.libp2p.mux.MuxHandler +import io.libp2p.mux.MuxHandlerAbstractTest +import io.libp2p.mux.MuxHandlerAbstractTest.AbstractTestMuxFrame.Flag.* +import io.libp2p.tools.readAllBytesAndRelease +import io.netty.channel.ChannelHandlerContext +import org.assertj.core.api.Assertions.assertThat +import org.junit.jupiter.api.Test + +class YamuxHandlerTest : MuxHandlerAbstractTest() { + + override val maxFrameDataLength = 256 + private val readFrameQueue = ArrayDeque() + + override fun createMuxHandler(streamHandler: StreamHandler<*>): MuxHandler = + object : YamuxHandler( + MultistreamProtocolV1, + maxFrameDataLength, + null, + streamHandler, + true + ) { + // MuxHandler consumes the exception. Override this behaviour for testing + @Deprecated("Deprecated in Java") + override fun exceptionCaught(ctx: ChannelHandlerContext, cause: Throwable) { + ctx.fireExceptionCaught(cause) + } + } + + override fun writeFrame(frame: AbstractTestMuxFrame) { + val muxId = frame.streamId.toMuxId() + val yamuxFrame = when (frame.flag) { + Open -> YamuxFrame(muxId, YamuxType.DATA, YamuxFlags.SYN, 0) + Data -> { + val data = frame.data.fromHex() + YamuxFrame( + muxId, + YamuxType.DATA, + 0, + data.size.toLong(), + data.toByteBuf(allocateBuf()) + ) + } + + Close -> YamuxFrame(muxId, YamuxType.DATA, YamuxFlags.FIN, 0) + Reset -> YamuxFrame(muxId, YamuxType.DATA, YamuxFlags.RST, 0) + } + ech.writeInbound(yamuxFrame) + } + + override fun readFrame(): AbstractTestMuxFrame? { + val yamuxFrame = readYamuxFrame() + if (yamuxFrame != null) { + when (yamuxFrame.flags) { + YamuxFlags.SYN -> readFrameQueue += AbstractTestMuxFrame(yamuxFrame.id.id, Open) + } + + val data = yamuxFrame.data?.readAllBytesAndRelease()?.toHex() ?: "" + when { + yamuxFrame.type == YamuxType.DATA && data.isNotEmpty() -> + readFrameQueue += AbstractTestMuxFrame(yamuxFrame.id.id, Data, data) + } + + when (yamuxFrame.flags) { + YamuxFlags.FIN -> readFrameQueue += AbstractTestMuxFrame(yamuxFrame.id.id, Close) + YamuxFlags.RST -> readFrameQueue += AbstractTestMuxFrame(yamuxFrame.id.id, Reset) + } + } + + return readFrameQueue.removeFirstOrNull() + } + + private fun readYamuxFrame(): YamuxFrame? { + return ech.readOutbound() + } + + private fun readYamuxFrameOrThrow() = readYamuxFrame() ?: throw AssertionError("No outbound frames") + + @Test + fun `test ack new stream`() { + // signal opening of new stream + openStream(12) + + writeStream(12, "23") + + val ackFrame = readYamuxFrameOrThrow() + + // receives ack stream + assertThat(ackFrame.flags).isEqualTo(YamuxFlags.ACK) + assertThat(ackFrame.type).isEqualTo(YamuxType.WINDOW_UPDATE) + + closeStream(12) + } + + @Test + fun `test window update`() { + openStream(12) + + val largeMessage = "42".repeat(INITIAL_WINDOW_SIZE + 1) + writeStream(12, largeMessage) + + // ignore ack stream frame + readYamuxFrameOrThrow() + + val windowUpdateFrame = readYamuxFrameOrThrow() + + assertThat(windowUpdateFrame.flags).isZero() + assertThat(windowUpdateFrame.type).isEqualTo(YamuxType.WINDOW_UPDATE) + assertThat(windowUpdateFrame.length).isEqualTo((INITIAL_WINDOW_SIZE + 1).toLong()) + + assertLastMessage(0, 1, largeMessage) + + closeStream(12) + } + + @Test + fun `data should be buffered and sent after window increased from zero`() { + val handler = openStreamByLocal() + val streamId = readFrameOrThrow().streamId + + ech.writeInbound( + YamuxFrame( + streamId.toMuxId(), + YamuxType.WINDOW_UPDATE, + YamuxFlags.ACK, + -INITIAL_WINDOW_SIZE.toLong() + ) + ) + + handler.ctx.writeAndFlush("1984".fromHex().toByteBuf(allocateBuf())) + + assertThat(readFrame()).isNull() + + ech.writeInbound(YamuxFrame(streamId.toMuxId(), YamuxType.WINDOW_UPDATE, YamuxFlags.ACK, 5000)) + val frame = readFrameOrThrow() + assertThat(frame.data).isEqualTo("1984") + } + + @Test + fun `test ping`() { + val id: Long = 0 + openStream(id) + ech.writeInbound( + YamuxFrame( + id.toMuxId(), + YamuxType.PING, + YamuxFlags.SYN, + // opaque value, echoed back + 3 + ) + ) + + // ignore ack stream frame + readYamuxFrameOrThrow() + + val pingFrame = readYamuxFrameOrThrow() + + assertThat(pingFrame.flags).isEqualTo(YamuxFlags.ACK) + assertThat(pingFrame.type).isEqualTo(YamuxType.PING) + assertThat(pingFrame.length).isEqualTo(3) + + closeStream(id) + } + + @Test + fun `test go away`() { + val id: Long = 0 + openStream(id) + ech.writeInbound( + YamuxFrame( + id.toMuxId(), + YamuxType.GO_AWAY, + 0, + // normal termination + 0x0 + ) + ) + + // verify session termination + assertThat(childHandlers[0].isHandlerRemoved).isTrue() + assertThat(childHandlers[0].isUnregistered).isTrue() + } +} diff --git a/libp2p/src/test/kotlin/io/libp2p/security/CipherSecureChannelTest.kt b/libp2p/src/test/kotlin/io/libp2p/security/CipherSecureChannelTest.kt index 7d0cd9bcc..b466f04e6 100644 --- a/libp2p/src/test/kotlin/io/libp2p/security/CipherSecureChannelTest.kt +++ b/libp2p/src/test/kotlin/io/libp2p/security/CipherSecureChannelTest.kt @@ -3,6 +3,7 @@ package io.libp2p.security import io.libp2p.core.PeerId import io.libp2p.core.crypto.KEY_TYPE import io.libp2p.core.crypto.generateKeyPair +import io.libp2p.core.mux.StreamMuxer import io.libp2p.tools.TestChannel import io.libp2p.tools.TestLogAppender import io.netty.buffer.Unpooled @@ -12,8 +13,8 @@ import org.assertj.core.api.Assertions.assertThatThrownBy import org.junit.jupiter.api.Test import java.util.concurrent.TimeUnit.SECONDS -abstract class CipherSecureChannelTest(secureChannelCtor: SecureChannelCtor, announce: String) : - SecureChannelTestBase(secureChannelCtor, announce) { +abstract class CipherSecureChannelTest(secureChannelCtor: SecureChannelCtor, muxers: List, announce: String) : + SecureChannelTestBase(secureChannelCtor, muxers, announce) { @Test fun `incorrect initiator remote PeerId should throw`() { @@ -21,8 +22,8 @@ abstract class CipherSecureChannelTest(secureChannelCtor: SecureChannelCtor, ann val (privKey2, _) = generateKeyPair(KEY_TYPE.ECDSA) val (_, wrongPubKey) = generateKeyPair(KEY_TYPE.ECDSA) - val protocolSelect1 = makeSelector(privKey1) - val protocolSelect2 = makeSelector(privKey2) + val protocolSelect1 = makeSelector(privKey1, muxerIds) + val protocolSelect2 = makeSelector(privKey2, muxerIds) val eCh1 = makeDialChannel("#1", protocolSelect1, PeerId.fromPubKey(wrongPubKey)) val eCh2 = makeListenChannel("#2", protocolSelect2) @@ -39,8 +40,8 @@ abstract class CipherSecureChannelTest(secureChannelCtor: SecureChannelCtor, ann val (privKey1, _) = generateKeyPair(KEY_TYPE.ECDSA) val (privKey2, pubKey2) = generateKeyPair(KEY_TYPE.ECDSA) - val protocolSelect1 = makeSelector(privKey1) - val protocolSelect2 = makeSelector(privKey2) + val protocolSelect1 = makeSelector(privKey1, muxerIds) + val protocolSelect2 = makeSelector(privKey2, muxerIds) val eCh1 = makeDialChannel("#1", protocolSelect1, PeerId.fromPubKey(pubKey2)) val eCh2 = makeListenChannel("#2", protocolSelect2) diff --git a/libp2p/src/test/kotlin/io/libp2p/security/SecureChannelTestBase.kt b/libp2p/src/test/kotlin/io/libp2p/security/SecureChannelTestBase.kt index 7c6f9fa06..a91651a75 100644 --- a/libp2p/src/test/kotlin/io/libp2p/security/SecureChannelTestBase.kt +++ b/libp2p/src/test/kotlin/io/libp2p/security/SecureChannelTestBase.kt @@ -5,6 +5,7 @@ import io.libp2p.core.crypto.KEY_TYPE import io.libp2p.core.crypto.PrivKey import io.libp2p.core.crypto.generateKeyPair import io.libp2p.core.multistream.ProtocolMatcher +import io.libp2p.core.mux.StreamMuxer import io.libp2p.core.security.SecureChannel import io.libp2p.etc.types.seconds import io.libp2p.etc.types.toByteArray @@ -18,20 +19,21 @@ import io.netty.buffer.ByteBuf import io.netty.channel.ChannelHandlerContext import io.netty.channel.ChannelInboundHandlerAdapter import io.netty.util.ResourceLeakDetector -import org.apache.logging.log4j.LogManager import org.assertj.core.api.Assertions.fail import org.junit.jupiter.params.ParameterizedTest import org.junit.jupiter.params.provider.MethodSource +import org.slf4j.LoggerFactory import java.nio.charset.StandardCharsets import java.util.concurrent.LinkedBlockingQueue import java.util.concurrent.TimeUnit -typealias SecureChannelCtor = (PrivKey) -> SecureChannel +typealias SecureChannelCtor = (PrivKey, List) -> SecureChannel -val logger = LogManager.getLogger(SecureChannelTestBase::class.java) +val logger = LoggerFactory.getLogger(SecureChannelTestBase::class.java) abstract class SecureChannelTestBase( val secureChannelCtor: SecureChannelCtor, + val muxerIds: List, val announce: String ) { init { @@ -56,8 +58,8 @@ abstract class SecureChannelTestBase( val (privKey1, _) = generateKeyPair(KEY_TYPE.ECDSA) val (privKey2, pubKey2) = generateKeyPair(KEY_TYPE.ECDSA) - val protocolSelect1 = makeSelector(privKey1) - val protocolSelect2 = makeSelector(privKey2) + val protocolSelect1 = makeSelector(privKey1, muxerIds) + val protocolSelect2 = makeSelector(privKey2, muxerIds) val eCh1 = makeDialChannel("#1", protocolSelect1, PeerId.fromPubKey(pubKey2)) val eCh2 = makeListenChannel("#2", protocolSelect2) @@ -117,7 +119,7 @@ abstract class SecureChannelTestBase( } } // secureInterconnect - protected fun makeSelector(key: PrivKey) = ProtocolSelect(listOf(secureChannelCtor(key))) + protected fun makeSelector(key: PrivKey, muxers: List) = ProtocolSelect(listOf(secureChannelCtor(key, muxers))) protected fun makeDialChannel( name: String, diff --git a/libp2p/src/test/kotlin/io/libp2p/security/noise/NoiseHandshakeTest.kt b/libp2p/src/test/kotlin/io/libp2p/security/noise/NoiseHandshakeTest.kt index a2dfb0282..a6f6454b2 100644 --- a/libp2p/src/test/kotlin/io/libp2p/security/noise/NoiseHandshakeTest.kt +++ b/libp2p/src/test/kotlin/io/libp2p/security/noise/NoiseHandshakeTest.kt @@ -146,7 +146,7 @@ class NoiseHandshakeTest { fun testAnnounceAndMatch() { val (privKey1, _) = generateKeyPair(KEY_TYPE.ECDSA) - val ch1 = NoiseXXSecureChannel(privKey1) + val ch1 = NoiseXXSecureChannel(privKey1, listOf()) Assertions.assertTrue( ch1.protocolDescriptor.matchesAny(ch1.protocolDescriptor.announceProtocols) @@ -156,11 +156,11 @@ class NoiseHandshakeTest { @Test fun testStaticNoiseKeyPerProcess() { val (privKey1, _) = generateKeyPair(KEY_TYPE.ECDSA) - NoiseXXSecureChannel(privKey1) + NoiseXXSecureChannel(privKey1, listOf()) val b1 = NoiseXXSecureChannel.localStaticPrivateKey25519.copyOf() val (privKey2, _) = generateKeyPair(KEY_TYPE.ECDSA) - NoiseXXSecureChannel(privKey2) + NoiseXXSecureChannel(privKey2, listOf()) val b2 = NoiseXXSecureChannel.localStaticPrivateKey25519.copyOf() Assertions.assertTrue(b1.contentEquals(b2), "NoiseXX static keys are not maintained between sessions.") diff --git a/libp2p/src/test/kotlin/io/libp2p/security/noise/NoiseSecureChannelTest.kt b/libp2p/src/test/kotlin/io/libp2p/security/noise/NoiseSecureChannelTest.kt index 031144ba5..b08ad60a2 100644 --- a/libp2p/src/test/kotlin/io/libp2p/security/noise/NoiseSecureChannelTest.kt +++ b/libp2p/src/test/kotlin/io/libp2p/security/noise/NoiseSecureChannelTest.kt @@ -8,5 +8,6 @@ import org.junit.jupiter.api.condition.DisabledIfEnvironmentVariable @Tag("secure-channel") class NoiseSecureChannelTest : CipherSecureChannelTest( ::NoiseXXSecureChannel, + listOf(), NoiseXXSecureChannel.announce ) diff --git a/libp2p/src/test/kotlin/io/libp2p/security/plaintext/PlaintextInsecureChannelTest.kt b/libp2p/src/test/kotlin/io/libp2p/security/plaintext/PlaintextInsecureChannelTest.kt index d16e41c82..a3b751ab8 100644 --- a/libp2p/src/test/kotlin/io/libp2p/security/plaintext/PlaintextInsecureChannelTest.kt +++ b/libp2p/src/test/kotlin/io/libp2p/security/plaintext/PlaintextInsecureChannelTest.kt @@ -6,5 +6,6 @@ import org.junit.jupiter.api.Tag @Tag("secure-channel") class PlaintextInsecureChannelTest : SecureChannelTestBase( ::PlaintextInsecureChannel, + listOf(), "/plaintext/2.0.0" ) diff --git a/libp2p/src/test/kotlin/io/libp2p/security/secio/EchoSampleTest.kt b/libp2p/src/test/kotlin/io/libp2p/security/secio/EchoSampleTest.kt index 1d885e21c..f1c8b097c 100644 --- a/libp2p/src/test/kotlin/io/libp2p/security/secio/EchoSampleTest.kt +++ b/libp2p/src/test/kotlin/io/libp2p/security/secio/EchoSampleTest.kt @@ -21,10 +21,10 @@ import io.netty.channel.ChannelHandler import io.netty.channel.ChannelHandlerContext import io.netty.handler.logging.LogLevel import io.netty.handler.logging.LoggingHandler -import org.apache.logging.log4j.LogManager import org.junit.jupiter.api.Assertions import org.junit.jupiter.api.Disabled import org.junit.jupiter.api.Test +import org.slf4j.LoggerFactory import java.util.concurrent.CompletableFuture import java.util.concurrent.TimeUnit @@ -51,7 +51,7 @@ class EchoSampleTest { @Test @Disabled fun connect1() { - val logger = LogManager.getLogger("test") + val logger = LoggerFactory.getLogger("test") val (privKey1, _) = generateKeyPair(KEY_TYPE.ECDSA) val applicationProtocols = listOf(createSimpleBinding("/echo/1.0.0") { EchoProtocol() }) @@ -63,7 +63,7 @@ class EchoSampleTest { } val upgrader = ConnectionUpgrader( MultistreamProtocolV1.copyWithHandlers(nettyToChannelHandler(LoggingHandler("#1", LogLevel.INFO))), - listOf(SecIoSecureChannel(privKey1)), + listOf(SecIoSecureChannel(privKey1, listOf())), MultistreamProtocolV1.copyWithHandlers(nettyToChannelHandler(LoggingHandler("#2", LogLevel.INFO))), listOf(muxer) ) diff --git a/libp2p/src/test/kotlin/io/libp2p/security/secio/SecIoSecureChannelTest.kt b/libp2p/src/test/kotlin/io/libp2p/security/secio/SecIoSecureChannelTest.kt index 96eefa6bf..4df5314f3 100644 --- a/libp2p/src/test/kotlin/io/libp2p/security/secio/SecIoSecureChannelTest.kt +++ b/libp2p/src/test/kotlin/io/libp2p/security/secio/SecIoSecureChannelTest.kt @@ -6,5 +6,6 @@ import org.junit.jupiter.api.Tag @Tag("secure-channel") class SecIoSecureChannelTest : CipherSecureChannelTest( ::SecIoSecureChannel, + listOf(), "/secio/1.0.0" ) diff --git a/libp2p/src/test/kotlin/io/libp2p/security/tls/CertificatesTest.kt b/libp2p/src/test/kotlin/io/libp2p/security/tls/CertificatesTest.kt new file mode 100644 index 000000000..feb2e6a98 --- /dev/null +++ b/libp2p/src/test/kotlin/io/libp2p/security/tls/CertificatesTest.kt @@ -0,0 +1,66 @@ +package io.libp2p.security.tls + +import io.libp2p.core.PeerId +import io.libp2p.crypto.keys.generateEd25519KeyPair +import org.bouncycastle.cert.X509CertificateHolder +import org.bouncycastle.cert.jcajce.JcaX509CertificateConverter +import org.bouncycastle.jce.provider.BouncyCastleProvider +import org.bouncycastle.util.encoders.Hex +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Test +import org.junit.jupiter.api.assertThrows + +class CertificatesTest { + + @Test + fun ed25519Peer() { + val hex = "308201773082011ea003020102020900f5bd0debaa597f52300a06082a8648ce3d04030230003020170d3735303130313030303030305a180f34303936303130313030303030305a30003059301306072a8648ce3d020106082a8648ce3d030107034200046bf9871220d71dcb3483ecdfcbfcc7c103f8509d0974b3c18ab1f1be1302d643103a08f7a7722c1b247ba3876fe2c59e26526f479d7718a85202ddbe47562358a37f307d307b060a2b0601040183a25a01010101ff046a30680424080112207fda21856709c5ae12fd6e8450623f15f11955d384212b89f56e7e136d2e17280440aaa6bffabe91b6f30c35e3aa4f94b1188fed96b0ffdd393f4c58c1c047854120e674ce64c788406d1c2c4b116581fd7411b309881c3c7f20b46e54c7e6fe7f0f300a06082a8648ce3d040302034700304402207d1a1dbd2bda235ff2ec87daf006f9b04ba076a5a5530180cd9c2e8f6399e09d0220458527178c7e77024601dbb1b256593e9b96d961b96349d1f560114f61a87595" + val certBytes = Hex.decode(hex) + val certHolder = X509CertificateHolder(certBytes) + val cert = JcaX509CertificateConverter().setProvider(BouncyCastleProvider()).getCertificate(certHolder) + val peerIdFromCert = verifyAndExtractPeerId(arrayOf(cert)) + val expectedPeerId = PeerId.fromBase58("12D3KooWJRSrypvnpHgc6ZAgyCni4KcSmbV7uGRaMw5LgMKT18fq") + assertEquals(peerIdFromCert, expectedPeerId) + } + + @Test + fun ecdsaPeer() { + val hex = "308201c030820166a003020102020900eaf419a6e3edb4a6300a06082a8648ce3d04030230003020170d3735303130313030303030305a180f34303936303130313030303030305a30003059301306072a8648ce3d020106082a8648ce3d030107034200048dbf1116c7c608d6d5292bd826c3feb53483a89fce434bf64538a359c8e07538ff71f6766239be6a146dcc1a5f3bb934bcd4ae2ae1d4da28ac68b4a20593f06ba381c63081c33081c0060a2b0601040183a25a01010101ff0481ae3081ab045f0803125b3059301306072a8648ce3d020106082a8648ce3d0301070342000484b93fa456a74bd0153919f036db7bc63c802f055bc7023395d0203de718ee0fc7b570b767cdd858aca6c7c4113ff002e78bd2138ac1a3b26dde3519e06979ad04483046022100bc84014cea5a41feabdf4c161096564b9ccf4b62fbef4fe1cd382c84e11101780221009204f086a84cb8ed8a9ddd7868dc90c792ee434adf62c66f99a08a5eba11615b300a06082a8648ce3d0403020348003045022054b437be9a2edf591312d68ff24bf91367ad4143f76cf80b5658f232ade820da022100e23b48de9df9c25d4c83ddddf75d2676f0b9318ee2a6c88a736d85eab94a912f" + val certBytes = Hex.decode(hex) + val certHolder = X509CertificateHolder(certBytes) + val cert = JcaX509CertificateConverter().setProvider(BouncyCastleProvider()).getCertificate(certHolder) + val peerIdFromCert = verifyAndExtractPeerId(arrayOf(cert)) + val expectedPeerId = PeerId.fromBase58("QmZcrvr3r4S3QvwFdae3c2EWTfo792Y14UpzCZurhmiWeX") + assertEquals(peerIdFromCert, expectedPeerId) + } + + @Test + fun secp256k1Peer() { + val hex = "3082018230820128a003020102020900f3b305f55622cfdf300a06082a8648ce3d04030230003020170d3735303130313030303030305a180f34303936303130313030303030305a30003059301306072a8648ce3d020106082a8648ce3d0301070342000458f7e9581748ff9bdd933b655cc0e5552a1248f840658cc221dec2186b5a2fe4641b86ab7590a3422cdbb1000cf97662f27e5910d7569f22feed8829c8b52e0fa38188308185308182060a2b0601040183a25a01010101ff0471306f042508021221026b053094d1112bce799dc8026040ae6d4eb574157929f1598172061f753d9b1b04463044022040712707e97794c478d93989aaa28ae1f71c03af524a8a4bd2d98424948a782302207b61b7f074b696a25fb9e0059141a811cccc4cc28042d9301b9b2a4015e87470300a06082a8648ce3d04030203480030450220143ae4d86fdc8675d2480bb6912eca5e39165df7f572d836aa2f2d6acfab13f8022100831d1979a98f0c4a6fb5069ca374de92f1a1205c962a6d90ad3d7554cb7d9df4" + val certBytes = Hex.decode(hex) + val certHolder = X509CertificateHolder(certBytes) + val cert = JcaX509CertificateConverter().setProvider(BouncyCastleProvider()).getCertificate(certHolder) + val peerIdFromCert = verifyAndExtractPeerId(arrayOf(cert)) + val expectedPeerId = PeerId.fromBase58("16Uiu2HAm2dSCBFxuge46aEt7U1oejtYuBUZXxASHqmcfVmk4gsbx") + assertEquals(peerIdFromCert, expectedPeerId) + } + + @Test + fun invalidCert() { + val hex = "308201773082011da003020102020830a73c5d896a1109300a06082a8648ce3d04030230003020170d3735303130313030303030305a180f34303936303130313030303030305a30003059301306072a8648ce3d020106082a8648ce3d03010703420004bbe62df9a7c1c46b7f1f21d556deec5382a36df146fb29c7f1240e60d7d5328570e3b71d99602b77a65c9b3655f62837f8d66b59f1763b8c9beba3be07778043a37f307d307b060a2b0601040183a25a01010101ff046a3068042408011220ec8094573afb9728088860864f7bcea2d4fd412fef09a8e2d24d482377c20db60440ecabae8354afa2f0af4b8d2ad871e865cb5a7c0c8d3dbdbf42de577f92461a0ebb0a28703e33581af7d2a4f2270fc37aec6261fcc95f8af08f3f4806581c730a300a06082a8648ce3d040302034800304502202dfb17a6fa0f94ee0e2e6a3b9fb6e986f311dee27392058016464bd130930a61022100ba4b937a11c8d3172b81e7cd04aedb79b978c4379c2b5b24d565dd5d67d3cb3c" + val certBytes = Hex.decode(hex) + val certHolder = X509CertificateHolder(certBytes) + val cert = JcaX509CertificateConverter().setProvider(BouncyCastleProvider()).getCertificate(certHolder) + assertThrows({ verifyAndExtractPeerId(arrayOf(cert)) }) + } + + @Test + fun buildEd25519Cert() { + val host = generateEd25519KeyPair() + val conn = generateEd25519KeyPair() + val cert = buildCert(host.first, conn.first) + val peerIdFromCert = verifyAndExtractPeerId(arrayOf(cert)) + val expectedPeerId = PeerId.fromPubKey(host.second) + assertEquals(peerIdFromCert, expectedPeerId) + } +} diff --git a/libp2p/src/test/kotlin/io/libp2p/security/tls/TlsSecureChannelTest.kt b/libp2p/src/test/kotlin/io/libp2p/security/tls/TlsSecureChannelTest.kt new file mode 100644 index 000000000..1d5fe5ed5 --- /dev/null +++ b/libp2p/src/test/kotlin/io/libp2p/security/tls/TlsSecureChannelTest.kt @@ -0,0 +1,42 @@ +package io.libp2p.security.tls + +import io.libp2p.core.PeerId +import io.libp2p.core.crypto.KEY_TYPE +import io.libp2p.core.crypto.generateKeyPair +import io.libp2p.core.multistream.MultistreamProtocolDebug +import io.libp2p.core.mux.StreamMuxerProtocol +import io.libp2p.multistream.MultistreamProtocolDebugV1 +import io.libp2p.security.InvalidRemotePubKey +import io.libp2p.security.SecureChannelTestBase +import io.libp2p.tools.TestChannel +import org.assertj.core.api.Assertions +import org.junit.jupiter.api.Tag +import org.junit.jupiter.api.Test +import java.util.concurrent.TimeUnit + +val MultistreamProtocolV1: MultistreamProtocolDebug = MultistreamProtocolDebugV1() + +@Tag("secure-channel") +class TlsSecureChannelTest : SecureChannelTestBase( + ::TlsSecureChannel, + listOf(StreamMuxerProtocol.Yamux.createMuxer(MultistreamProtocolV1, listOf())), + TlsSecureChannel.announce +) { + @Test + fun `incorrect initiator remote PeerId should throw`() { + val (privKey1, _) = generateKeyPair(KEY_TYPE.ECDSA) + val (privKey2, _) = generateKeyPair(KEY_TYPE.ECDSA) + val (_, wrongPubKey) = generateKeyPair(KEY_TYPE.ECDSA) + + val protocolSelect1 = makeSelector(privKey1, muxerIds) + val protocolSelect2 = makeSelector(privKey2, muxerIds) + + val eCh1 = makeDialChannel("#1", protocolSelect1, PeerId.fromPubKey(wrongPubKey)) + val eCh2 = makeListenChannel("#2", protocolSelect2) + + TestChannel.interConnect(eCh1, eCh2) + + Assertions.assertThatThrownBy { protocolSelect1.selectedFuture.get(10, TimeUnit.SECONDS) } + .hasCauseInstanceOf(InvalidRemotePubKey::class.java) + } +} diff --git a/libp2p/src/test/kotlin/io/libp2p/transport/TransportTests.kt b/libp2p/src/test/kotlin/io/libp2p/transport/TransportTests.kt index 2fff556d5..7e0975434 100644 --- a/libp2p/src/test/kotlin/io/libp2p/transport/TransportTests.kt +++ b/libp2p/src/test/kotlin/io/libp2p/transport/TransportTests.kt @@ -5,7 +5,6 @@ import io.libp2p.core.ConnectionHandler import io.libp2p.core.Libp2pException import io.libp2p.core.multiformats.Multiaddr import io.libp2p.core.transport.Transport -import org.apache.logging.log4j.LogManager import org.junit.jupiter.api.AfterEach import org.junit.jupiter.api.Assertions.assertEquals import org.junit.jupiter.api.Assertions.assertThrows @@ -13,6 +12,7 @@ import org.junit.jupiter.api.BeforeEach import org.junit.jupiter.api.Tag import org.junit.jupiter.api.Test import org.junit.jupiter.api.condition.DisabledIfEnvironmentVariable +import org.slf4j.LoggerFactory import java.util.concurrent.CompletableFuture import java.util.concurrent.TimeUnit.SECONDS import java.util.concurrent.atomic.AtomicBoolean @@ -25,7 +25,7 @@ abstract class TransportTests { protected lateinit var transportUnderTest: Transport protected val nullConnHandler = ConnectionHandler { } - protected val logger = LogManager.getLogger("test") + protected val logger = LoggerFactory.getLogger("test") protected fun startListeners(server: Transport, startPortNumber: Int, howMany: Int) { val listening = (0 until howMany).map { diff --git a/libp2p/src/test/kotlin/io/libp2p/pubsub/DeterministicFuzz.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/pubsub/DeterministicFuzz.kt similarity index 100% rename from libp2p/src/test/kotlin/io/libp2p/pubsub/DeterministicFuzz.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/pubsub/DeterministicFuzz.kt diff --git a/libp2p/src/test/kotlin/io/libp2p/pubsub/MockRouter.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/pubsub/MockRouter.kt similarity index 100% rename from libp2p/src/test/kotlin/io/libp2p/pubsub/MockRouter.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/pubsub/MockRouter.kt diff --git a/libp2p/src/test/kotlin/io/libp2p/pubsub/TestRouter.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/pubsub/TestRouter.kt similarity index 99% rename from libp2p/src/test/kotlin/io/libp2p/pubsub/TestRouter.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/pubsub/TestRouter.kt index cf8fd7e7e..a90a9f45e 100644 --- a/libp2p/src/test/kotlin/io/libp2p/pubsub/TestRouter.kt +++ b/libp2p/src/testFixtures/kotlin/io/libp2p/pubsub/TestRouter.kt @@ -68,7 +68,7 @@ class TestRouter( ConnectionOverNetty(parentChannel, NullTransport(), initiator) connection.setSecureSession( SecureChannel.Session( - peerId, remoteRouter.peerId, remoteRouter.keyPair.second + peerId, remoteRouter.peerId, remoteRouter.keyPair.second, null ) ) diff --git a/libp2p/src/test/kotlin/io/libp2p/pubsub/gossip/Eth2GossipParams.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/pubsub/gossip/Eth2GossipParams.kt similarity index 100% rename from libp2p/src/test/kotlin/io/libp2p/pubsub/gossip/Eth2GossipParams.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/pubsub/gossip/Eth2GossipParams.kt diff --git a/libp2p/src/testFixtures/kotlin/io/libp2p/tools/ByteBufExt.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/ByteBufExt.kt new file mode 100644 index 000000000..bb91e5840 --- /dev/null +++ b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/ByteBufExt.kt @@ -0,0 +1,10 @@ +package io.libp2p.tools + +import io.netty.buffer.ByteBuf + +fun ByteBuf.readAllBytesAndRelease(): ByteArray { + val arr = ByteArray(readableBytes()) + this.readBytes(arr) + this.release() + return arr +} diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/CountingPingProtocol.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/CountingPingProtocol.kt similarity index 100% rename from libp2p/src/test/kotlin/io/libp2p/tools/CountingPingProtocol.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/CountingPingProtocol.kt diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/DnsAvailability.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/DnsAvailability.kt similarity index 100% rename from libp2p/src/test/kotlin/io/libp2p/tools/DnsAvailability.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/DnsAvailability.kt diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/DoNothingProtocol.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/DoNothingProtocol.kt similarity index 100% rename from libp2p/src/test/kotlin/io/libp2p/tools/DoNothingProtocol.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/DoNothingProtocol.kt diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/EchoProtocol.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/EchoProtocol.kt similarity index 97% rename from libp2p/src/test/kotlin/io/libp2p/tools/EchoProtocol.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/EchoProtocol.kt index 4da4b6581..17b14dda4 100644 --- a/libp2p/src/test/kotlin/io/libp2p/tools/EchoProtocol.kt +++ b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/EchoProtocol.kt @@ -44,7 +44,8 @@ open class EchoProtocol : ProtocolHandler(Long.MAX_VALUE, Long.M } } - open inner class EchoInitiator(val ready: CompletableFuture) : ProtocolMessageHandler, EchoController { + open inner class EchoInitiator(val ready: CompletableFuture) : + ProtocolMessageHandler, EchoController { lateinit var stream: Stream var ret = CompletableFuture() diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/HostFactory.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/HostFactory.kt similarity index 96% rename from libp2p/src/test/kotlin/io/libp2p/tools/HostFactory.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/HostFactory.kt index 2af4b035f..08d3be9e3 100644 --- a/libp2p/src/test/kotlin/io/libp2p/tools/HostFactory.kt +++ b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/HostFactory.kt @@ -7,6 +7,7 @@ import io.libp2p.core.crypto.PrivKey import io.libp2p.core.crypto.PubKey import io.libp2p.core.crypto.generateKeyPair import io.libp2p.core.dsl.Builder +import io.libp2p.core.dsl.SecureChannelCtor import io.libp2p.core.dsl.host import io.libp2p.core.multiformats.Multiaddr import io.libp2p.core.multiformats.Protocol @@ -26,7 +27,7 @@ class HostFactory { var keyType = KEY_TYPE.ECDSA var tcpPort = 5000 var transportCtor = ::TcpTransport - var secureCtor = ::NoiseXXSecureChannel + var secureCtor: SecureChannelCtor = ::NoiseXXSecureChannel var mplexCtor = ::MplexStreamMuxer var muxLogLevel: LogLevel? = LogLevel.DEBUG diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/NullHost.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/NullHost.kt similarity index 95% rename from libp2p/src/test/kotlin/io/libp2p/tools/NullHost.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/NullHost.kt index 2e3f84c44..476bd0e2e 100644 --- a/libp2p/src/test/kotlin/io/libp2p/tools/NullHost.kt +++ b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/NullHost.kt @@ -55,6 +55,10 @@ open class NullHost : Host { TODO("not implemented") } + override fun getProtocols(): List> { + TODO("not implemented") + } + override fun addConnectionHandler(handler: ConnectionHandler) { TODO("not implemented") } diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/NullTransport.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/NullTransport.kt similarity index 100% rename from libp2p/src/test/kotlin/io/libp2p/tools/NullTransport.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/NullTransport.kt diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/P2pdRunner.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/P2pdRunner.kt similarity index 100% rename from libp2p/src/test/kotlin/io/libp2p/tools/P2pdRunner.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/P2pdRunner.kt diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/Stubs.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/Stubs.kt similarity index 100% rename from libp2p/src/test/kotlin/io/libp2p/tools/Stubs.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/Stubs.kt diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/TCPProxy.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/TCPProxy.kt similarity index 97% rename from libp2p/src/test/kotlin/io/libp2p/tools/TCPProxy.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/TCPProxy.kt index 6c3de7390..3f4ad2346 100644 --- a/libp2p/src/test/kotlin/io/libp2p/tools/TCPProxy.kt +++ b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/TCPProxy.kt @@ -12,8 +12,6 @@ import io.netty.channel.socket.nio.NioServerSocketChannel import io.netty.channel.socket.nio.NioSocketChannel import io.netty.handler.logging.LogLevel import io.netty.handler.logging.LoggingHandler -import org.junit.jupiter.api.Disabled -import org.junit.jupiter.api.Test import java.util.concurrent.CompletableFuture // Utility class (aka sniffer) that just forwards TCP traffic back'n'forth to another TCP address and log it @@ -67,8 +65,6 @@ class TCPProxy { return future } - @Test - @Disabled fun run() { start(11111, "localhost", 10000) .channel().closeFuture().await() diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/TestChannel.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/TestChannel.kt similarity index 96% rename from libp2p/src/test/kotlin/io/libp2p/tools/TestChannel.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/TestChannel.kt index ab00151fb..74ba9f44f 100644 --- a/libp2p/src/test/kotlin/io/libp2p/tools/TestChannel.kt +++ b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/TestChannel.kt @@ -10,7 +10,7 @@ import io.libp2p.transport.implementation.ConnectionOverNetty import io.netty.channel.ChannelHandler import io.netty.channel.ChannelId import io.netty.channel.embedded.EmbeddedChannel -import org.apache.logging.log4j.LogManager +import org.slf4j.LoggerFactory import java.net.InetSocketAddress import java.net.SocketAddress import java.util.concurrent.CompletableFuture @@ -101,7 +101,7 @@ class TestChannel( return TestConnection(ch1, ch2) } - private val logger = LogManager.getLogger(TestChannel::class.java) + private val logger = LoggerFactory.getLogger(TestChannel::class.java) } class TestConnection(val ch1: TestChannel, val ch2: TestChannel) { diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/TestHandler.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/TestHandler.kt similarity index 93% rename from libp2p/src/test/kotlin/io/libp2p/tools/TestHandler.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/TestHandler.kt index 28edfd85d..7fcf6b333 100644 --- a/libp2p/src/test/kotlin/io/libp2p/tools/TestHandler.kt +++ b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/TestHandler.kt @@ -5,7 +5,7 @@ import io.libp2p.etc.types.toHex import io.netty.buffer.ByteBuf import io.netty.channel.ChannelHandlerContext import io.netty.channel.ChannelInboundHandlerAdapter -import org.apache.logging.log4j.LogManager +import org.slf4j.LoggerFactory open class TestHandler(val name: String = "") : ChannelInboundHandlerAdapter() { override fun channelActive(ctx: ChannelHandlerContext) { @@ -49,6 +49,6 @@ open class TestHandler(val name: String = "") : ChannelInboundHandlerAdapter() { } companion object { - private val logger = LogManager.getLogger(TestHandler::class.java) + private val logger = LoggerFactory.getLogger(TestHandler::class.java) } } diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/TestLogAppender.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/TestLogAppender.kt similarity index 100% rename from libp2p/src/test/kotlin/io/libp2p/tools/TestLogAppender.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/TestLogAppender.kt diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/TestStreamChannel.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/TestStreamChannel.kt similarity index 100% rename from libp2p/src/test/kotlin/io/libp2p/tools/TestStreamChannel.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/TestStreamChannel.kt diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/protobuf/ProtobufUtils.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/protobuf/ProtobufUtils.kt similarity index 100% rename from libp2p/src/test/kotlin/io/libp2p/tools/protobuf/ProtobufUtils.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/protobuf/ProtobufUtils.kt diff --git a/libp2p/src/test/kotlin/io/libp2p/tools/protobuf/RpcBuilder.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/tools/protobuf/RpcBuilder.kt similarity index 100% rename from libp2p/src/test/kotlin/io/libp2p/tools/protobuf/RpcBuilder.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/tools/protobuf/RpcBuilder.kt diff --git a/libp2p/src/test/kotlin/io/libp2p/transport/NullConnectionUpgrader.kt b/libp2p/src/testFixtures/kotlin/io/libp2p/transport/NullConnectionUpgrader.kt similarity index 95% rename from libp2p/src/test/kotlin/io/libp2p/transport/NullConnectionUpgrader.kt rename to libp2p/src/testFixtures/kotlin/io/libp2p/transport/NullConnectionUpgrader.kt index 64e6c173f..62f1b6c2c 100644 --- a/libp2p/src/test/kotlin/io/libp2p/transport/NullConnectionUpgrader.kt +++ b/libp2p/src/testFixtures/kotlin/io/libp2p/transport/NullConnectionUpgrader.kt @@ -25,7 +25,8 @@ class NullConnectionUpgrader : val nonsenseSession = SecureChannel.Session( PeerId.random(), PeerId.random(), - generateKeyPair(KEY_TYPE.RSA).second + generateKeyPair(KEY_TYPE.RSA).second, + null ) return CompletableFuture.completedFuture(nonsenseSession) } // establishSecureChannel diff --git a/settings.gradle b/settings.gradle index a42edcd64..3b2d71fae 100644 --- a/settings.gradle +++ b/settings.gradle @@ -1,4 +1,21 @@ -rootProject.name = 'jvm-libp2p-minimal' +pluginManagement { + repositories { + gradlePluginPortal() + mavenCentral() + google() + } +} + +dependencyResolutionManagement { + repositories { + mavenCentral() + maven { url "https://artifacts.consensys.net/public/maven/maven/" } + maven { url "https://jitpack.io" } + google() + } +} + +rootProject.name = 'jvm-libp2p' include ':libp2p' include ':tools:schedulers' @@ -6,3 +23,27 @@ include ':tools:simulator' include ':examples:chatter' include ':examples:cli-chatter' include ':examples:pinger' + +def getAndroidSdkDir() { + def localPropertiesSdkDir = null + if (file('local.properties').canRead()) { + def properties = new Properties() + properties.load(file('local.properties').newDataInputStream()) + localPropertiesSdkDir = properties.getProperty('sdk.dir') + } + def androidHomeEnv = System.getenv("ANDROID_HOME") + if (localPropertiesSdkDir != null) { + return localPropertiesSdkDir + } else { + return androidHomeEnv + } +} + +if (getAndroidSdkDir() != null) { + println "Build configured with Android submodules using Android SDK: ${getAndroidSdkDir()}" + include ':examples:android-chatter' +} else { + println "Build configured without Android submodules." + println " To include Android submodules define a valid SDK location with an ANDROID_HOME environment variable " + println " or by setting the sdk.dir path in your project's local properties file local.properties." +} diff --git a/tools/schedulers/src/main/java/io/libp2p/tools/schedulers/DefaultSchedulers.java b/tools/schedulers/src/main/java/io/libp2p/tools/schedulers/DefaultSchedulers.java index f3cad73bf..837012929 100644 --- a/tools/schedulers/src/main/java/io/libp2p/tools/schedulers/DefaultSchedulers.java +++ b/tools/schedulers/src/main/java/io/libp2p/tools/schedulers/DefaultSchedulers.java @@ -1,8 +1,8 @@ package io.libp2p.tools.schedulers; import com.google.common.util.concurrent.ThreadFactoryBuilder; -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.util.concurrent.Executors; import java.util.concurrent.ScheduledExecutorService; @@ -11,7 +11,7 @@ public class DefaultSchedulers extends AbstractSchedulers { - private static final Logger logger = LogManager.getLogger(DefaultSchedulers.class); + private static final Logger logger = LoggerFactory.getLogger(DefaultSchedulers.class); private Consumer errorHandler = t -> logger.error("Unhandled exception:", t); private volatile boolean started; diff --git a/tools/schedulers/src/main/java/io/libp2p/tools/schedulers/LoggerMDCExecutor.java b/tools/schedulers/src/main/java/io/libp2p/tools/schedulers/LoggerMDCExecutor.java index f35747a32..2bd55b4c8 100644 --- a/tools/schedulers/src/main/java/io/libp2p/tools/schedulers/LoggerMDCExecutor.java +++ b/tools/schedulers/src/main/java/io/libp2p/tools/schedulers/LoggerMDCExecutor.java @@ -1,6 +1,6 @@ package io.libp2p.tools.schedulers; -import org.apache.logging.log4j.ThreadContext; +import org.slf4j.MDC; import java.util.ArrayList; import java.util.List; @@ -31,15 +31,15 @@ public LoggerMDCExecutor add(String mdcKey, Supplier mdcValueSupplier) { public void execute(Runnable command) { List oldValues = new ArrayList<>(mdcKeys.size()); for (int i = 0; i < mdcKeys.size(); i++) { - oldValues.add(ThreadContext.get(mdcKeys.get(i))); - ThreadContext.put(mdcKeys.get(i), mdcValueSuppliers.get(i).get()); + oldValues.add(MDC.get(mdcKeys.get(i))); + MDC.put(mdcKeys.get(i), mdcValueSuppliers.get(i).get()); } delegateExecutor.execute(command); for (int i = 0; i < mdcKeys.size(); i++) { if (oldValues.get(i) == null) { - ThreadContext.remove(mdcKeys.get(i)); + MDC.remove(mdcKeys.get(i)); } else { - ThreadContext.put(mdcKeys.get(i), oldValues.get(i)); + MDC.put(mdcKeys.get(i), oldValues.get(i)); } } } diff --git a/tools/simulator/build.gradle b/tools/simulator/build.gradle index 2529f783a..bab7ad655 100644 --- a/tools/simulator/build.gradle +++ b/tools/simulator/build.gradle @@ -5,12 +5,13 @@ dependencies { api project(':libp2p') api project(':tools:schedulers') + implementation("io.netty:netty-handler") + implementation("org.jgrapht:jgrapht-core:1.3.1") api("org.apache.commons:commons-math3:3.6.1") implementation("org.jetbrains.kotlin:kotlin-reflect:1.3.0") - runtimeOnly("org.apache.logging.log4j:log4j-core") - runtimeOnly("org.apache.logging.log4j:log4j-slf4j-impl") + runtimeOnly("org.apache.logging.log4j:log4j-slf4j2-impl") } diff --git a/tools/simulator/src/main/kotlin/io/libp2p/simulate/stream/Libp2pConnectionImpl.kt b/tools/simulator/src/main/kotlin/io/libp2p/simulate/stream/Libp2pConnectionImpl.kt index 55c6f4f1e..9c17a0050 100644 --- a/tools/simulator/src/main/kotlin/io/libp2p/simulate/stream/Libp2pConnectionImpl.kt +++ b/tools/simulator/src/main/kotlin/io/libp2p/simulate/stream/Libp2pConnectionImpl.kt @@ -24,7 +24,8 @@ class Libp2pConnectionImpl( SecureChannel.Session( PeerId.fromPubKey(localPubkey), PeerId.fromPubKey(remotePubkey), - remotePubkey + remotePubkey, + null ) ) } diff --git a/versions.gradle b/versions.gradle index d7c299f79..44bf4a254 100644 --- a/versions.gradle +++ b/versions.gradle @@ -3,10 +3,10 @@ dependencyManagement { dependency "org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.4" dependency "com.google.guava:guava:31.1-jre" - dependencySet(group: 'org.apache.logging.log4j', version: '2.19.0') { - entry 'log4j-api' + + dependency "org.slf4j:slf4j-api:2.0.7" + dependencySet(group: 'org.apache.logging.log4j', version: '2.20.0') { entry 'log4j-core' - entry 'log4j-slf4j-impl' entry 'log4j-slf4j2-impl' } @@ -27,12 +27,20 @@ dependencyManagement { entry 'protobuf-java' entry 'protoc' } - dependency "io.netty:netty-all:4.1.87.Final" + dependencySet(group: "io.netty", version: "4.1.90.Final") { + entry 'netty-common' + entry 'netty-handler' + entry 'netty-transport' + entry 'netty-buffer' + entry 'netty-codec-http' + entry 'netty-transport-classes-epoll' + } dependency "commons-codec:commons-codec:1.15" dependency "tech.pegasys:noise-java:22.1.0" dependencySet(group: "org.bouncycastle", version: "1.70") { entry 'bcprov-jdk15on' entry 'bcpkix-jdk15on' + entry 'bctls-jdk15on' } } } \ No newline at end of file