Allow rsa pkcs#1 decryption of ciphertexts less-than or equal to key-size

[arlenyan]

This PR allows RSA PKCS#1 decryption of ciphertexts less-than or equal to key-size for compatibility with other implementations, such as Tom Wu's JSBN/RSA library which seems to occasionally produce ciphertext octet-strings shorter than key-size depending on if it contains leading zero octets.

Example

Private key (2048-bits):

308204a302010002820101008f824748748a7b80e864e87f72fbdd2999c9106efc52c414cb2f5bb58246bdeca805ad689beac56da816a264f4b0aa42b8e62187cfc11e527b634cf7151f9a3a4630cdc72685dc451061bf2b8c030d93693e33e79df2458c9e52136a82294c90ec92517aee8aa571e91a533f28cf7b4536d6d491375fa5b641fad153eba42d4f0c775a348b09e6092c72b25a9fc01d43a323882becf9446fa7e07da9dff289f05785caecf9b9cd37c08a083df9520579dcae11a7332e2fb3d523dda120c34d6f9e47093bcbf7ab7159da24493b36c9886c4f6d512ad315e0baac7cfc7b53a4b62d3d255bf8cdfe18366748a32f252eda93c31ed3b901b6ebafacf329fcd3534902030100010282010002dc16faf4adb40c022255b57876e0de77966ce4f9d319dded6923f3682bd28387f06c393e32f51a8b1194eea97cacb0f7167bf9f8282cec7a51b5e3c16292330b484b9eccd516a5c502d5a32c6a0df5e6d05ae03bcd6b4bfb5bd0365ea21b2f41de87a155ff804be7b5e2f3ca7c08c4b89f8984ed4ab97acf766180173f6252cadc42a0b309fc457291f50e4f0e92955a8097b7810ca69774c0d574adbc07ff2451cca5fc95712b72ec3276af7bcd36acea63a6ff6271bc5f0fe455bb3288fa41cdc1369186d79bb81960b9a0456b9188bcdd867d3aa367154743cffacc1c765f1dbc728d7205cf3b7e7f4165b08dd38689bd2f0de4e559d65d8722339e65b302818100c6fb23a2188520e519af060cdb500e908792750859209b2adf1c433265dcfeb0e7680efd7399f7a727fb498892b05762cffaaa53fca8a31de658009917bbecfcff72740a94a4c9a9cceb8a99ad7a5e4b6efae31d2119aad855a166598fc50723818ac5d1cab72e761a155f4a73ca59eed1728ca55855718c319620c467ad7e6702818100b8a1d012a3dc56971fd69236934400c88c84f69aeaa92ece155279684977ea59bdba0e55626a949e1576117c741177585f3871c692712045edf5ec90a18fab90c0bd798fbcbf8cea5ce1799f4220943e01f688c651cf4d1887f8c819c7c07f90aceaca11503c157de0d58b68231138c9294ff7317e0052d1fae5c392581632cf02818100c2fc449dca4362c9be1259ee6d5714fea22b6d8dc1b106fccae76a1dcfddf0a2b010b367c049677aa0de48df9147464aa91dac9d62a06a3f9982310bd44b2e5f797ed1a1b0a98e885b2b9185288f1a29f755d90aa96dfa91c5cfc4790d3e78d15d56fca4a96e3a1437592c28cc30c552166296da4c26667f87f3e2a63426ba0f028180198118ba0a729c6f81d1651f2ed69cf9171822eca16fbf6c6e5b9733c5ba4fe017aa44e29c96b672e3ea0c5e51b63bf3342c1ae360454a3cfeb312fba3a5b2006bef80844f817258c97dc80c2cd94d41078a63b86982a656b629b292851a5d44c1ee28fd9d44bf7f44f89aaa46f5d10f50aaa02df106e069eb2ba4096a2b9ed70281803ca108d26689e8f0ef8111a28182f84117727fd56d85fb6719dc44b40a5ce2639a8e64850d74c189ea9445a466036808db283d2b8b0a6567b42500f1e193da340f66fcb0899bf146266374e1fdbb13bdf7dc064483866342185a75faf50dd45d82e9a88fa53c8cff469e7df24244c3ec8a127d57b0d633b74fb0b63c69a632a4

Public key:

3082010a02820101008f824748748a7b80e864e87f72fbdd2999c9106efc52c414cb2f5bb58246bdeca805ad689beac56da816a264f4b0aa42b8e62187cfc11e527b634cf7151f9a3a4630cdc72685dc451061bf2b8c030d93693e33e79df2458c9e52136a82294c90ec92517aee8aa571e91a533f28cf7b4536d6d491375fa5b641fad153eba42d4f0c775a348b09e6092c72b25a9fc01d43a323882becf9446fa7e07da9dff289f05785caecf9b9cd37c08a083df9520579dcae11a7332e2fb3d523dda120c34d6f9e47093bcbf7ab7159da24493b36c9886c4f6d512ad315e0baac7cfc7b53a4b62d3d255bf8cdfe18366748a32f252eda93c31ed3b901b6ebafacf329fcd353490203010001

Plaintext value:

0ef20deb2637818f665c124678eedc3ce97f3ef5b0d50e03687f787c3d9575ac

Ciphertext produced by Tom Wu's JSBN/RSA library (255-bytes, one less than the key-size):

42b9632f9d0132884c84b1f3205bd54f33cefc5cf9024460532461e73a6949ac62bcb62be898c2d35bf54b9b9f1b4013e9dce2b98b8cf78b99996c39a04af18a3aceb82cea884229682cad5faca2814b05275ff80db51dde524fcad6f35e78ed2ff9e90222a0ca9c8ca078a7b022ded06dde912da30746308563187d0b8e1558320843045e7cf63cf0bac419f22a54a891973b5460371d9cd73c73a3976b0974ff5da5865ce6d501527e7fd45545723ac00a292298f533f9f05e0a9aba41f46b50c7ae94c45517a5d60b025293aa91b53c0eae6177c716ddd17a3af73b60d949c96e364227921efa3a3497947c6e8697bab9608eddd1bfa2e5de8908951a01

Prior to this PR decrypting the ciphertext would produce an error Ciphertext length must be equal to key size.. After this PR decrypting the ciphertext produces the plaintext value.

Are JSBN/RSA ciphertext-lengths shorter than key-size valid?

RSA PKCS#1 described in rfc3447 section 7.2.1 ultimately produces a ciphertext by an I2OSP (Integer-to-Octet-String primitive) step described in rfc3447 section 4.1.

This I2OSP step suggests the resulting octet string may be of a specified length:

I2OSP converts a nonnegative integer to an octet string of a
specified length.

And it acknowledges that there may be leading zeros if the integer x is too small:

where 0 <= x_i < 256 (note that one or more leading digits will be
zero if x is less than 256^(xLen-1)).

Given this I am still unsure whether or not the ciphertexts produced by Tom Wu's JSBN/RSA library are technically a violation of the specification. However, at one time this library was a de facto standard for client-side RSA and was fairly pervasive. Therefore it might be prudent to handle its ciphertexts if there are no consequences.

Other implementations

Apple's SecKeyDecrypt API decrypts the above example successfully.

The Bouncy Castle package decrypts the above example successfully.

The python-rsa package decrypts the above example successfully.

The pycryptodome package does NOT decrypt the above example successfully.

The micro-rsa-dsa-dh package does NOT decrypt the above example successfully.

[alex]

I haven't gone back to read the spec beyond the portions you quoted, but it seems to me those are quite clear that the output octet sequence should be length long, and that you'll have leading 0s to make that happen.

Therefore my initial impression is that Mr. Wu's library has a bug.

But I'm happy to hear counter arguments if I've misinterpreted anything.

[arlenyan]

Reviewing it again, I agree with your initial impression. It does seem to be a bug in Tom Wu's library.

Specifically in rfc3447 section 7.2.1 it states:

(n, e) recipient's RSA public key (k denotes the length in octets
of the modulus n)
[...]
C ciphertext, an octet string of length k
[...]
c. Convert the ciphertext representative c to a ciphertext C of
length k octets (see Section 4.1):

          C = I2OSP (c, k).

Or that the ciphertext should match the length in octets of the public-key modulus.

I originally was looking primarily at the I2OSP step in rfc3447 section 4.1 and interpreted it to mean the implementation had discretion over the "specified length". However, section 7.2.1 is explicit about the length being the length in octets of the public-key modulus.

I'll close this PR now. Thanks for your consideration!