Onion messages (EXPERIMENTAL) #3600

rustyrussell · 2020-03-20T05:13:52Z

Based on #3599 this implements simple messaging across the lightning network. There are no contents defined yet, except for those required for routing and replies, but the intent is to use this mechanism for offers.

The specification for messaging is in lightning/bolts#759 but the blinded path implementation is stolen straight from https://github.com/lightningnetwork/lightning-rfc/blob/route-blinding/proposals/route-blinding.md.

We use route blinding for two things: one, you can place them inside an onion to send to an otherwise unknown destination, and also you can include a path for the recipient to use to reply. Both are implemented here.

TODO:

We need to add a feature bit, so you know how to route these.
We still need to implement decoy decoding: if we use a decoy route, we need to check that it's actually included as specified.

Changelog-None

cdecker

Looks quite good, just some minor things

cdecker · 2020-03-24T16:26:12Z

lightningd/onion_message.c

+	/* FIXME: We should allow this for closingd too, and we should
+	 * allow incoming via openingd!. */


Why restrict this to receiving when in openingd? As far as I understood this messaging layer is orthogonal to the existence of a channel, and should be available independently of whether we have a channel with our peer open or not.

Yes, agreed.

cdecker · 2020-03-24T16:27:17Z

lightningd/onion_message.c

+		return;
+	}
+
+	log_info(channel->log, "Got onionmsg%s%s",


Suggested change

log_info(channel->log, "Got onionmsg%s%s",

log_debug(channel->log, "Got onionmsg%s%s",

or even better log_io

Yeah, it was a debug thing. Will remove.

Oh, actually it's used in tests. I lowered it to debug in the commit which added the hook at this point.

cdecker · 2020-03-24T16:39:49Z

devtools/blindedpath.c

+static bool simpleout = false;
+


If we are hoping to simplify interaction with callers, why not go all the way and make this output a JSON stream? That'd then enable us to pipe a large variety of tools together.

Gathering all the pieces of information in a single result struct and switching between output formats only at the end when printing all the deferred prints would be good for this use-case as well, and would reduce the clutter of having two different print modes.

Because this tool should not exist. There should be a python library to do this properly.

cdecker · 2020-03-24T17:16:24Z

tests/test_misc.py

+    output = subprocess.check_output(
+        [blindedpathtool, '--simple-output', 'create', l2.info['id'], l1.info['id']]
+    ).decode('ASCII').strip()
+    print(output)


Suggested change

print(output)

rustyrussell · 2020-03-29T05:33:28Z

Thanks!

OK, rebased, and those minor fixups.

Also did two more changes:

Updated to latest version of the draft (changes numbers of codes).
Fix blindedpath tool not to tweak op.ephemeralkey direclty, as this invalidates the following onion.

This makes "check" more accurate, and also simplifies the code a little. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

This tracks lightning/bolts#759 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We do most of the decoding here, and just hand the results to lightningd. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

It's a fairly complex API, but maps simply onto the spec. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

e.g. $ PUBKEY1=0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 $ PRIVKEY1=41bfd2660762506c9933ade59f1debf7e6495b10c14a92dbcd2d623da2507d3d $ PUBKEY2=022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59 $ PRIVKEY1=c4a813f81ffdca1da6864db81795ad2d320add274452cafa1fb2ac2d07d062bd # First line is blinding, second is contents and nodeids for onion. $ ./devtools/blindedpath create $PUBKEY1 $PUBKEY2 03f006a18d5653c4edf5391ff23a61f03ff83d237e880ee61187fa9f379a028e0a 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518/350633c340f28bc69cbc86f568b7b9e99fa41eb581452d066fcd70dd53c43ace14d034eebfbe472a2b9901b11c268d2cc2034a77928a 0326f31ff78e584461420e5026fe72374af2ef853e65c47a3f2406348b7c6c0911/00 # Generate the onion $ /devtools/onion generate 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518/350633c340f28bc69cbc86f568b7b9e99fa41eb581452d066fcd70dd53c43ace14d034eebfbe472a2b9901b11c268d2cc2034a77928a 0326f31ff78e584461420e5026fe72374af2ef853e65c47a3f2406348b7c6c0911/00 > /tmp/onion.dat # First node unwraps it, gives next blinding and onion $ ./devtools/blindedpath --first-node unwrap $PRIVKEY1 `cat /tmp/onion.dat` 03f006a18d5653c4edf5391ff23a61f03ff83d237e880ee61187fa9f379a028e0a Contents: 04210326f31ff78e584461420e5026fe72374af2ef853e65c47a3f2406348b7c6c0911 Next blinding: 021295ce94fcadc42c3e5187a12dd80122214c8f9da61635163cddb63282f1ee9b Next onion: 0003c8fa9d4997ebd049480db14f0a90db211ec2b9f062e569419ee2c2b528a2d7adc63aa9e7b74997c2c122c4c1923e2f4587fc62532965666dbe55a76aa1ea903cfe6c498c6b7e80c14609d3c0f98f86a30f94b0b8a405067ee3801aab03420922cdc847d6f2fc359655408047a8d7d7892a595f630cdf114d1cc0d1164dc8099378042cfba7a13711dff64000356aac29726a6039bf938b81cc0dcc7f65dc126ae58838c0446d6492b6381f0402a33318a9ae71486bcb4b58f66c9a056fca306668655c11f7d7d0cd447e4162100565369629ca8b705b2b999a40ad5493953cb70b35f382e6acdc04e5a933783f9c5859fb0beeaa9c54e5220f5de3b107813d33148501aebcf67e190d3dcf10553714d4de8a1643b519cd124da9a345e2da0d669954a10fce9c1e7795572fb2ac8fe6de4db856bdbd327f0c4ae3cb11f6e1422f663423ad57891d069bbc5bdc7613c742a2227d3789d9039dcdcbddc2703835002dd176004c56cc497d88deae2328ad1376877f4582c71a7fa1eba4ae4e6696782bd97d7b362f41e81335b47273a74c983c3bc80499069a08c6b7ffa32cc77f54a98d8bc2f80f38c370c98edd8f6d6f95c6bbf5c8040296f68560de3b50c3450de4fcceae41469bea6a24c83141c92956fa4c4087f11e9c26b282e9c3974fadc8f9bb9fb9c3fbd2ead0cf4ceaba452eb8791828a159ace7a2e1e8ff5c69704a821c6c898a3c38439149862f14b7ed34afb93640c3ae61b089011ef698c9e26dc16b8a5a8ec66fce702b7bdf04a21cf9a2ebff6b89b29904e7e3e6a98088c2848951c0d6915249a3e1199c2affe4635ac6ef3a16ddfefc5790435a14067b24d5ecd16a26d2f7dbd8065b8e5b86f73f878cf55ca0c9f12104d861d03ac5c4b3dcbd0e30ad7ba888ddcc2e89acb3c04be2ef4bce3ef8c8878fef5be65664f1ec288f91dbd1748e2e53bcbd7dc9cecd75a246992e76a844ab122c5e179f97531190c7d91586289410c9ddad33eb156ab7312c82e55f3d643f3c12468ce79f7221051da608dd17ceb235b7df89f3c4b9aa9448bf36206b2db7bb97f544d062d6aa1b1706376fb6e3e8ef1ac293b9adca478458e9e51845dc7b554c70a91e32c331962968f98db26faa5b10a39bc778b0aab5a9fd11fdbab7b456db286049e584b7e4d1c76d6c3c6249b567aa357982ceead2ad8d5113a866818997b24018059e93eb5cafd293942efec3cf4a43bf322cd444e8370dc2cf1f1164c5147b30a791e262acadc15c30f1e169af4bff5e6c098acf95534b0b59517e3168413134984d50c8242590c8729fe34190d20d7f88505747b919e0bf8b41ed4ec9146743339c6885dcb770dcac627a1570dc145f6f61a976d87d16473195d5f5ce39347ab040c34fe0888b498f0ba25686a9bae51f6e5973d15f10d1c7dbe5fbfa2c7fe05cdd52d8eca8150914b4cb2e75e52010bb4b9241160d7337f47bbebd3bea58ddcf320a0464c34dbfd3d3ea7ffaee966c36064b2cc77babbf4613a7d5d65b3931dc42f91ed902207c57509a46738d31671c439d052c16db5ce3e613c5b37f77e574e9a847ebab20159130f33186557d16ddd1d765e7a9adedb253b755128e1af58da9e8b0fe6ed5834ddf1537c0ee78cd9803ee06031c3c1331d405a94a54f06d1147b26d0788179320d882f57ee9c63ab7a76fb9572eb813fe55369b5061b0a242b07a72095754a9d5699534a46829398204e76f1eda1d65e98fab1e8f3cf7c85257c8e2da0546fed215c3db38d231a637fd7a4e6f10b786d15534137489c662a0f289da824ca8dfef335bdcc623d636d231e002cc32febbe6683046ee54702dbeb55a70911505844c79c0be4630cf49456fba55aec9218d3ba449540370d407fc653007fdf59b4711d1da3c2e569eec4de8cee7b05d248ad0 # Feed that onion and blinding to second node $ ./devtools/blindedpath unwrap $PRIVKEY2 0003c8fa9d4997ebd049480db14f0a90db211ec2b9f062e569419ee2c2b528a2d7adc63aa9e7b74997c2c122c4c1923e2f4587fc62532965666dbe55a76aa1ea903cfe6c498c6b7e80c14609d3c0f98f86a30f94b0b8a405067ee3801aab03420922cdc847d6f2fc359655408047a8d7d7892a595f630cdf114d1cc0d1164dc8099378042cfba7a13711dff64000356aac29726a6039bf938b81cc0dcc7f65dc126ae58838c0446d6492b6381f0402a33318a9ae71486bcb4b58f66c9a056fca306668655c11f7d7d0cd447e4162100565369629ca8b705b2b999a40ad5493953cb70b35f382e6acdc04e5a933783f9c5859fb0beeaa9c54e5220f5de3b107813d33148501aebcf67e190d3dcf10553714d4de8a1643b519cd124da9a345e2da0d669954a10fce9c1e7795572fb2ac8fe6de4db856bdbd327f0c4ae3cb11f6e1422f663423ad57891d069bbc5bdc7613c742a2227d3789d9039dcdcbddc2703835002dd176004c56cc497d88deae2328ad1376877f4582c71a7fa1eba4ae4e6696782bd97d7b362f41e81335b47273a74c983c3bc80499069a08c6b7ffa32cc77f54a98d8bc2f80f38c370c98edd8f6d6f95c6bbf5c8040296f68560de3b50c3450de4fcceae41469bea6a24c83141c92956fa4c4087f11e9c26b282e9c3974fadc8f9bb9fb9c3fbd2ead0cf4ceaba452eb8791828a159ace7a2e1e8ff5c69704a821c6c898a3c38439149862f14b7ed34afb93640c3ae61b089011ef698c9e26dc16b8a5a8ec66fce702b7bdf04a21cf9a2ebff6b89b29904e7e3e6a98088c2848951c0d6915249a3e1199c2affe4635ac6ef3a16ddfefc5790435a14067b24d5ecd16a26d2f7dbd8065b8e5b86f73f878cf55ca0c9f12104d861d03ac5c4b3dcbd0e30ad7ba888ddcc2e89acb3c04be2ef4bce3ef8c8878fef5be65664f1ec288f91dbd1748e2e53bcbd7dc9cecd75a246992e76a844ab122c5e179f97531190c7d91586289410c9ddad33eb156ab7312c82e55f3d643f3c12468ce79f7221051da608dd17ceb235b7df89f3c4b9aa9448bf36206b2db7bb97f544d062d6aa1b1706376fb6e3e8ef1ac293b9adca478458e9e51845dc7b554c70a91e32c331962968f98db26faa5b10a39bc778b0aab5a9fd11fdbab7b456db286049e584b7e4d1c76d6c3c6249b567aa357982ceead2ad8d5113a866818997b24018059e93eb5cafd293942efec3cf4a43bf322cd444e8370dc2cf1f1164c5147b30a791e262acadc15c30f1e169af4bff5e6c098acf95534b0b59517e3168413134984d50c8242590c8729fe34190d20d7f88505747b919e0bf8b41ed4ec9146743339c6885dcb770dcac627a1570dc145f6f61a976d87d16473195d5f5ce39347ab040c34fe0888b498f0ba25686a9bae51f6e5973d15f10d1c7dbe5fbfa2c7fe05cdd52d8eca8150914b4cb2e75e52010bb4b9241160d7337f47bbebd3bea58ddcf320a0464c34dbfd3d3ea7ffaee966c36064b2cc77babbf4613a7d5d65b3931dc42f91ed902207c57509a46738d31671c439d052c16db5ce3e613c5b37f77e574e9a847ebab20159130f33186557d16ddd1d765e7a9adedb253b755128e1af58da9e8b0fe6ed5834ddf1537c0ee78cd9803ee06031c3c1331d405a94a54f06d1147b26d0788179320d882f57ee9c63ab7a76fb9572eb813fe55369b5061b0a242b07a72095754a9d5699534a46829398204e76f1eda1d65e98fab1e8f3cf7c85257c8e2da0546fed215c3db38d231a637fd7a4e6f10b786d15534137489c662a0f289da824ca8dfef335bdcc623d636d231e002cc32febbe6683046ee54702dbeb55a70911505844c79c0be4630cf49456fba55aec9218d3ba449540370d407fc653007fdf59b4711d1da3c2e569eec4de8cee7b05d248ad0 021295ce94fcadc42c3e5187a12dd80122214c8f9da61635163cddb63282f1ee9b Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Header from folded patch 'fixup': fixup! devtool/blindedpath: primitive tool to make blinded onions. On decode, don't mess with op.ephemeralkey, since it will be used to derive the next hop. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Normal output is suitable for feeding to devtools/onion, but for python tests we want something simpler. Ideally, we'd simply generate blinded paths in pyln. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

This is based on https://github.com/lightningnetwork/lightning-rfc/blob/route-blinding/proposals/route-blinding.md Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

rustyrussell · 2020-04-01T03:54:36Z

Rebased again on top of master (trivial rebase).

rustyrussell added the Optech Make Me Famous! Look! Look! Look! COOL NEW FEATURE! label Mar 20, 2020

rustyrussell added this to the 0.8.2 milestone Mar 20, 2020

rustyrussell requested a review from cdecker as a code owner March 20, 2020 05:13

cdecker reviewed Mar 24, 2020

View reviewed changes

rustyrussell force-pushed the guilt/onion-messages branch from 390535c to 9d1848e Compare March 29, 2020 05:32

rustyrussell added 11 commits April 1, 2020 10:21

sendpay: do route parsing inside parameter parsing.

d0605ac

This makes "check" more accurate, and also simplifies the code a little. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

EXPERMENTAL_FEATURES: Import onion message types.

bd2f2ec

This tracks lightning/bolts#759 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

channeld: handle onion messages.

fcdcb5e

We do most of the decoding here, and just hand the results to lightningd. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

lightningd: forward onion messages.

b5c0a9e

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

lightningd: EXPERIMENTAL_FEATURES JSON command to sendonionmessage.

e2faf9c

It's a fairly complex API, but maps simply onto the spec. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

pytest: test sendonionmessage.

7bdab85

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

devtools/blindedpath: add --simple-output for use from python.

556df4f

Normal output is suitable for feeding to devtools/onion, but for python tests we want something simpler. Ideally, we'd simply generate blinded paths in pyln. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

channeld: handle encblob and blinding in messages.

7d9abc2

This is based on https://github.com/lightningnetwork/lightning-rfc/blob/route-blinding/proposals/route-blinding.md Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

plugins: expose onion_message hook (EXPERIMENTAL_FEATURES)

587a181

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

pytest: test the reply functionality (via blinded path) using a plugin.

edd0230

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

rustyrussell force-pushed the guilt/onion-messages branch from 9d1848e to edd0230 Compare April 1, 2020 03:54

rustyrussell merged commit 77e7bee into ElementsProject:master Apr 2, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Onion messages (EXPERIMENTAL) #3600

Onion messages (EXPERIMENTAL) #3600

rustyrussell commented Mar 20, 2020 •

edited

Loading

cdecker left a comment

cdecker Mar 24, 2020

rustyrussell Mar 29, 2020

cdecker Mar 24, 2020

rustyrussell Mar 29, 2020

rustyrussell Mar 29, 2020

cdecker Mar 24, 2020

cdecker Mar 24, 2020

rustyrussell Mar 29, 2020

cdecker Mar 24, 2020

rustyrussell commented Mar 29, 2020

rustyrussell commented Apr 1, 2020

		/* FIXME: We should allow this for closingd too, and we should
		* allow incoming via openingd!. */

	log_info(channel->log, "Got onionmsg%s%s",
	log_debug(channel->log, "Got onionmsg%s%s",

Onion messages (EXPERIMENTAL) #3600

Onion messages (EXPERIMENTAL) #3600

Conversation

rustyrussell commented Mar 20, 2020 • edited Loading

cdecker left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rustyrussell commented Mar 29, 2020

rustyrussell commented Apr 1, 2020

rustyrussell commented Mar 20, 2020 •

edited

Loading