Add Multipath TCP (MPTCP) support (Proxy)

Multipath TCP (MPTCP), standardized in RFC8684 [1],
is a TCP extension that enables a TCP connection to
use different paths.

Multipath TCP has been used for several use cases.
On smartphones, MPTCP enables seamless handovers between
cellular and Wi-Fi networks while preserving established
connections. This use-case is what pushed Apple to use
MPTCP since 2013 in multiple applications [2]. On dual-stack
hosts, Multipath TCP enables the TCP connection to
automatically use the best performing path, either IPv4
or IPv6. If one path fails, MPTCP automatically uses
the other path.

To benefit from MPTCP, both the client and the server
have to support it. Multipath TCP is a backward-compatible
TCP extension that is enabled by default on recent
Linux distributions (Debian, Ubuntu, Redhat, ...). Multipath
TCP is included in the Linux kernel since version 5.6 [3].
To use it on Linux, an application must explicitly enable
it when creating the socket. No need to change anything
else in the application.

Adding the possibility to create MPTCP sockets would thus
be a really fine addition to httpd, by allowing clients
to make use of their different interfaces.

This patch introduces the possibilty to connect to backend
servers using MPTCP. Note however that these changes are
only available on Linux, as IPPROTO_MPTCP is Linux specific
for the time being.

For proxies, we can connect using MPTCP by passing the
\"multipathtcp\" parameter:

ProxyPass \"/example\" \"http://backend.example.com\" multipathtcp=On

We then store this information in the worker and create sockets
appropriately according to this value.

Link: https://www.rfc-editor.org/rfc/rfc8684.html [1]
Link: https://www.tessares.net/apples-mptcp-story-so-far/ [2]
Link: https://www.mptcp.dev [3]
Add Multipath TCP (MPTCP) support (Core)

Multipath TCP (MPTCP), standardized in RFC8684 [1],
is a TCP extension that enables a TCP connection to
use different paths.

Multipath TCP has been used for several use cases.
On smartphones, MPTCP enables seamless handovers between
cellular and Wi-Fi networks while preserving established
connections. This use-case is what pushed Apple to use
MPTCP since 2013 in multiple applications [2]. On dual-stack
hosts, Multipath TCP enables the TCP connection to
automatically use the best performing path, either IPv4
or IPv6. If one path fails, MPTCP automatically uses
the other path.

To benefit from MPTCP, both the client and the server
have to support it. Multipath TCP is a backward-compatible
TCP extension that is enabled by default on recent
Linux distributions (Debian, Ubuntu, Redhat, ...). Multipath
TCP is included in the Linux kernel since version 5.6 [3].
To use it on Linux, an application must explicitly enable
it when creating the socket. No need to change anything
else in the application.

Adding the possibility to create MPTCP sockets would thus
be a really fine addition to httpd, by allowing clients
to make use of their different interfaces.

This patch introduces the possibility to listen with MPTCP
sockets. Note however that these changes are only available
on Linux, as IPPROTO_MPTCP is Linux specific for the time being.

To do so, we extended the Listen directive to include
a \"multipathtcp\" option, allowing to create MPTCP sockets
instead of regular TCP ones:

Listen 80 options=multipathtcp

We then store this information in flags for the listen directive
and create sockets appropriately according to this value.

Link: https://www.rfc-editor.org/rfc/rfc8684.html [1]
Link: https://www.tessares.net/apples-mptcp-story-so-far/ [2]
Link: https://www.mptcp.dev [3]

Submitted by: Aperence <anthony.doeraene hotmail.com>
Github: closes #476


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1920586 13f79535-47bb-0310-9956-ffa450edef68

CHANGES

@@ -1,6 +1,12 @@
                                                          -*- coding: utf-8 -*-
 Changes with Apache 2.5.1
 
+  *) core: Add "multipathtcp" Listen option. PR 69292.
+     [Anthony Doeraene <anthony.doeraene hotmail.com>]
+
+  *) mod_proxy: Add "multipathtcp" worker parameter.  PR 69292.
+     [Anthony Doeraene <anthony.doeraene hotmail.com>]
+
   *) mod_authnz_ldap.c: Make sure the authentication variables are set in
      all cases where another module is the source of the authentication,
      and that authenticated user is looked up in LDAP. [Graham Leggett]

docs/manual/mod/mod_proxy.xml

@@ -1326,6 +1326,10 @@ ProxyPass "/example" "http://backend.example.com" max=20 ttl=120 retry=300
         <td><p>TTL in seconds for how long DNS resolutions of the backend address are cached.
         -1 means until restart of Apache httpd.</p>
     </td></tr>
+    <tr><td><a id="multipathtcp" name="multipathtcp">multipathtcp</a></td>
+        <td>Off</td>
+        <td><p>Enable/disable the use of <a href="https://mptcp.dev">Multipath TCP (MPTCP)</a></p>
+    </td></tr>
 
     </table>
 

docs/manual/mod/mpm_common.xml

@@ -278,6 +278,12 @@ Listen 192.170.2.5:8000
       to the same port.  (If the server is built with IPv4-mapped
       addresses <em>disabled</em>, this is the default behaviour and
       this option has no effect.)</li>
+
+      <li><code>multipathtcp</code>: Enable the use of
+      <a href="https://mptcp.dev">Multipath TCP (MPTCP)</a> for the
+      sockets. Beware that this option is currently limited to Linux
+      only.
+      </li>
     </ul>
 
     <note><title>Error condition</title>

include/ap_listen.h

@@ -44,6 +44,7 @@ typedef apr_status_t (*accept_function)(void **csd, ap_listen_rec *lr, apr_pool_
 #define AP_LISTEN_FREEBIND        (0x0002)
 #define AP_LISTEN_REUSEPORT       (0x0004)
 #define AP_LISTEN_V6ONLY          (0x0008)
+#define AP_LISTEN_MPTCP           (0x0010)
 
 /**
  * @brief Apache's listeners record.

include/ap_mmn.h

@@ -732,14 +732,15 @@
  * 20211221.24 (2.5.1-dev) Add ap_proxy_fixup_uds_filename()
  * 20211221.25 (2.5.1-dev) AP_SLASHES and AP_IS_SLASH
  * 20211221.26 (2.5.1-dev) Add is_host_matchable to proxy_worker_shared
+ * 20211221.27 (2.5.1-dev) Add sock_proto to proxy_worker_shared, and AP_LISTEN_MPTCP
  */
 
 #define MODULE_MAGIC_COOKIE 0x41503235UL /* "AP25" */
 
 #ifndef MODULE_MAGIC_NUMBER_MAJOR
 #define MODULE_MAGIC_NUMBER_MAJOR 20211221
 #endif
-#define MODULE_MAGIC_NUMBER_MINOR 26             /* 0...n */
+#define MODULE_MAGIC_NUMBER_MINOR 27             /* 0...n */
 
 /**
  * Determine if the server's current MODULE_MAGIC_NUMBER is at least a

modules/proxy/mod_proxy.c

@@ -356,6 +356,18 @@ static const char *set_worker_param(apr_pool_t *p,
         worker->s->response_field_size = (s ? s : HUGE_STRING_LEN);
         worker->s->response_field_size_set = 1;
     }
+    else if (!strcasecmp(key, "multipathtcp")) {
+#ifdef IPPROTO_MPTCP
+        if (!strcasecmp(val, "On"))
+            worker->s->sock_proto = IPPROTO_MPTCP;
+        else if (!strcasecmp(val, "Off"))
+            worker->s->sock_proto = APR_PROTO_TCP;
+        else
+            return "multipathtcp must be On|Off";
+#else
+        return "multipathtcp is not supported on your platform";
+#endif
+    }
     else {
         if (set_worker_hc_param_f) {
             return set_worker_hc_param_f(p, s, worker, key, val, NULL);

modules/proxy/mod_proxy.h

@@ -497,6 +497,7 @@ typedef struct {
     unsigned int     address_ttl_set:1;
     apr_int32_t      address_ttl;    /* backend address' TTL (seconds) */
     apr_uint32_t     address_expiry; /* backend address' next expiry time */
+    int              sock_proto;     /* The protocol to use to create the socket */
 } proxy_worker_shared;
 
 #define ALIGNED_PROXY_WORKER_SHARED_SIZE (APR_ALIGN_DEFAULT(sizeof(proxy_worker_shared)))

modules/proxy/proxy_util.c

@@ -3858,7 +3858,7 @@ PROXY_DECLARE(int) ap_proxy_connect_backend(const char *proxy_function,
 #endif
         {
             if ((rv = apr_socket_create(&newsock, backend_addr->family,
-                                        SOCK_STREAM, APR_PROTO_TCP,
+                                        SOCK_STREAM, worker->s->sock_proto,
                                         conn->scpool)) != APR_SUCCESS) {
                 loglevel = backend_addr->next ? APLOG_DEBUG : APLOG_ERR;
                 ap_log_error(APLOG_MARK, loglevel, rv, s, APLOGNO(00952)

server/listen.c

@@ -496,6 +496,12 @@ static const char *alloc_listener(process_rec *process, const char *addr,
 
     while (sa) {
         ap_listen_rec *new;
+        int sock_proto = 0;
+
+#ifdef IPPROTO_MPTCP
+        if (flags & AP_LISTEN_MPTCP)
+            sock_proto = IPPROTO_MPTCP;
+#endif
 
         /* this has to survive restarts */
         new = apr_palloc(process->pool, sizeof(ap_listen_rec));
@@ -509,7 +515,7 @@ static const char *alloc_listener(process_rec *process, const char *addr,
         sa = sa->next;
 
         status = apr_socket_create(&new->sd, new->bind_addr->family,
-                                    SOCK_STREAM, 0, process->pool);
+                                    SOCK_STREAM, sock_proto, process->pool);
 
 #if APR_HAVE_IPV6
         /* What could happen is that we got an IPv6 address, but this system
@@ -864,6 +870,7 @@ AP_DECLARE(apr_status_t) ap_duplicate_listeners(apr_pool_t *p, server_rec *s,
             char *hostname;
             apr_port_t port;
             apr_sockaddr_t *sa;
+            int sock_proto = 0;
 #ifdef HAVE_SYSTEMD
             if (use_systemd) {
                 int thesock;
@@ -891,8 +898,12 @@ AP_DECLARE(apr_status_t) ap_duplicate_listeners(apr_pool_t *p, server_rec *s,
                 duplr->bind_addr = sa;
                 duplr->next = NULL;
                 duplr->flags = lr->flags;
+#ifdef IPPROTO_MPTCP
+                if (duplr->flags & AP_LISTEN_MPTCP)
+                    sock_proto = IPPROTO_MPTCP;
+#endif
                 stat = apr_socket_create(&duplr->sd, duplr->bind_addr->family,
-                                         SOCK_STREAM, 0, p);
+                                         SOCK_STREAM, sock_proto, p);
                 if (stat != APR_SUCCESS) {
                     ap_log_perror(APLOG_MARK, APLOG_CRIT, 0, p, APLOGNO(02640)
                                 "ap_duplicate_listeners: for address %pI, "
@@ -1038,6 +1049,13 @@ static const char *parse_listen_flags(apr_pool_t *temp_pool, const char *arg,
             flags |= AP_LISTEN_REUSEPORT;
         else if (ap_cstr_casecmp(token, "v6only") == 0)
             flags |= AP_LISTEN_V6ONLY;
+        else if (ap_cstr_casecmp(token, "multipathtcp") == 0)
+#ifdef IPPROTO_MPTCP
+            flags |= AP_LISTEN_MPTCP;
+#else
+            return apr_psprintf(temp_pool, "Listen option '%s' in '%s' is not supported on this system",
+                                token, arg);
+#endif
         else
             return apr_psprintf(temp_pool, "Unknown Listen option '%s' in '%s'",
                                 token, arg);