0.8 beta

After enormous waits for 0.6 and 0.7, the project is finally getting to make more frequent releases, and 0.8 beta is here less than half a year after 0.7.0. The amount of changes is somewhat smaller too, so you don't need to spend extended time just to figure out what's changed (and I don't need to spend even longer to write the release notes!).

This is a good moment for packagers to double-check that their builds are still good with the new library but please hold off from packaging 0.8, just yet. More goodness is coming soon, with the final release planned to happen before the end of June.

As always, thanks to all the contributors, new and seasoned - you rock!

Main news

The highlight of this release is that you can switch E2EE at each Connection object individiually (#609 by @KitsuneRal). By default, E2EE is now switched off; you have to call Connection::setE2eeDefault() to enable it for all Connections created further. On top of this class-scope setting, you can call Connection::enableEncryption() on any Connection object before it is logged in, to alter the mode for that one object.

As of this beta, there's no graceful handling of situations when encryption is enabled but is unusable (e.g. if there is a database but the pickling key doesn't match it or is unavailable). To preserve user data, Debug builds will fail on assertion; Release builds will just disable encryption on the object and keep running is if it was not enabled at all. By the final release, Debug builds will do the same as Release builds.

Other news

• The backend Client-Server API code is generated from Matrix v1.7 files (#669), enabling access, among else, to asynchronous uploads (MSC2246), content redirects (MSC3860), and using an existing session to log in another (MSC3882); these are not wired further into Connection/Room yet but the first step is made
• Thanks to @TobiasFella (#634), Doxygen documentation for the library is now generated and published automatically at https://quotient-im.github.io/libQuotient/ - reports on any inaccuracies/typos/bitrot etc. are welcome!
• @vkrause made another round of build tightening and improvement PRs:
  • Qt5 and Qt6 versions of libQuotient can now be installed side-by-side (#646, #650) - this required to change soname of the Qt6 variant but no known distro shipped that variant anyway
  • Global constants for key names etc. now have one prefered variant (the one without L at the end), with L variants getting deprecated (#648); L-less variant is now QLatin1String, too
  • Fixed dependency finding in CMake (#656, #658)
• @KitsuneRal also gave the redactions code a hard look and fixed things around redacting state events that surfaced (#666)
• Connection::assumeIdentity() emits networkError() (which it never did before) instead of loginError() when the actual problem is with the network (e9c3d37 by @KitsuneRal)
• Connection no more automatically adds and removes itself from the Accounts singleton, and this singleton itself is deprecated now (c2c9bf4 by @KitsuneRal); client authors have to create an AccountRegistry instance for themselves (if they wish, still as a singleton, but be aware that it's considered an anti-pattern)
• BaseJob::aboutToSendRequest() carries a pointer to the modifiable network request object along with it, allowing to adjust the request before sending (389f365 by @KitsuneRal). Use with care.
• Previously deprecated Room::reset*Count() functions are entirely removed (#661) after they were found out to crash NeoChat in obscure ways
• Omittable::ensure() has been removed as it was sitting unused for quite some time (14f31f4 by @KitsuneRal)
• Event::type() is deprecated now and is planned for removal in 0.9 (#667), to prevent the long-standing confusion between type() and matrixType(), with even the library's own code misusing type() even though it doesn't necessarily return the actual type stored in the event JSON
• @TobiasFella fixed aliases not to remap back from the replacement room to its predecessor accidentally (#649)
• @KitsuneRal fixed image requests from QML to be thread-safe (#659)
• @chmeeedalf fixed Matrix URI parsing for colon characters that are percent-encoded, i.e. %3A instead of : (#651) - first contribution to the project!
• @CyberTailor fixed E2EE builds with LibreSSL (#654) - another first-time contributor here!

Full Changelog: 0.7.2...0.8-beta1

0.7.2

Another round of fixes and small improvements has arrived in 0.7 branch. Special thanks go to @vkrause (with @TobiasFella as a supporting act) for a small pile of building and packaging cleanup PRs (#617, #619, #620, #621, #622, #623, #624, #632/#633, and #626/#645).

Notable changes:

• Within #626 mentioned above, the top-level source directory name was changed from lib to Quotient. Although it's not exactly backwards-compatible if you use libQuotient as a submodule, client build scripts were not supposed to rely on the existence of lib directory in the first place. On the other hand, the new top-level directory name allows to use a common Quotient/ prefix in #include statements, regardless of the way libQuotient is consumed (a submodule or an installed package).
• MxcReply has got some love, fixing crashes (#628) and broken WebP images (#627) in NeoChat (by @redstrate - first contribution, congrats and thank you!)
• Send megolm key when there is no new olm session (#629 by @TobiasFella) - fixing the "encrypted messages fail after the first 100" problem
• Fix the library failing to process to-device events beyond the first one in the sync batch (#631 by @TobiasFella)
• Two new utility functions to help client authors figure out E2EE status of a given device (#635 by @TobiasFella)
• A few tweaks and fixes in Connection and AccountRegistry (#636 by @KitsuneRal):
  • AccountRegistry::invokeLogin() is deprecated
  • AccountRegistry::isLoggedIn() actually checks that an account is logged in, not just exists in the registry
  • Connection::assumeIdentity() doesn't use the passed device id any more, taking it from the access token instead
• You can now get the whole list of room account data event types (#638 by @TobiasFella)
• Verification of the user's devices is a bit more reliable thanks to #642 by @TobiasFella
• The library now prefers the correct Base64 algorithm (brought in by MSC3783) for MAC calculation (#640 by @KitsuneRal)

Full Changelog: 0.7.1...0.7.2

0.7.1

Happy New Year! Hot on the heels of 0.7.0, the first bugfix release in 0.7.x series arrives. Most important changes:

• Fix compilation with e2ee on windows (#604 by @TobiasFella)
• To prevent things like the one above from happening CI now covers E2EE on all platforms, not just Linux (#606 by @KitsuneRal)
• Fix SSO flows failing to complete when a client uses a proxy server (#607 by @BLumia)
• Fix exposing QtKeychain dependency to clients (#611 by @TobiasFella and #612 by @KitsuneRal) - particularly nasty in that libQuotient itself builds and installs just fine but an attempt to build a client using certain parts of that installed libQuotient (AccountRegistry, namely) will always fail.

New Contributors

• @BLumia made their first contribution in #607 - many thanks!

Full Changelog: 0.7.0...0.7.1

0.7.0

It took another long wait since 0.6 was released but here we are, the next stable libQuotient is out! As usual, some numbers for statistics - 0.7.0 is getting released 2+ years after 0.6.0 (yikes...) and more than a year since the latest stable release, 0.6.11. Well... there was a reason to that, and it is all the work on E2EE finally getting to beta status! That, together with all the other features, fixes and cleanup, resulted in a monstrous chain of 1100+ commits in Git. See more details below.

This release is the first since a long time with the bus factor for libQuotient becoming more than 1 again. Between 2017 and 2020, I (@KitsuneRal) was the only active developer who knew the library code. Since 2020, the NeoChat leads, @TobiasFella and @CarlSchwan, reimplemented the E2EE foundations in the library that were previously unfinished in 0.6 - learning most of the library code along the way. Not that I plan to retire or anything but it's much better to be in a team. Their contributions are also the biggest that the library has ever seen from a non-project lead, by a large margin. I owe you beverages, guys.

That said, with a really big chunk of work behind further changes will (hopefully!) come in smaller and more rapid batches...

Toolchain updates

Quite a bit of time passed since 0.6, so we've had an opportunity to switch to C++20, now widely adopted. Respectively, the earliest known-to-work compiler versions are GCC 11, Clang 11, MSVC 2019 and Apple Clang 12. Older ones are very unlikely to build the library without considerable patching. CMake (3.16+) is the only build configuration tool supported from now; qmake support has been dropped.

As for Qt, libQuotient 0.7 can be built either with Qt 5 or Qt 6; however, only Qt 5.15 (the only still officially supported) is accepted for the older major version. It is likely that in 0.8 or 0.9 we will stop building with Qt 5 altogether; Qt 6 is already mature and widespread enough, and Qt 5.15 will get less and less attention as The Qt Project collectively moves along.

New dependencies: Qt Keychain (always), Olm and OpenSSL (for E2EE)

Quotient punted the problem of storing secrets (such as access tokens - passwords should never be stored anywhere at all) to clients; both Quaternion and NeoChat used QtKeychain to accomplish that. With introduction of E2EE, there's one more secret to store - a pickling key that is used to decrypt pretty much anything sensitive in the SQLite database that Quotient now has. Therefore, libQuotient is now explicitly and unconditionally dependent on QtKeychain - please make sure it's installed or CMake will complain early on.

When E2EE is switched on, the library will also depend on Olm and OpenSSL. Version 0.6 used QtOlm but this is no more the case; instead, libQuotient incorporated a rewrite of QtOlm pieces and now uses Olm directly. OpenSSL is also necessary to allocate secure heap and use standard crypto algorithms not exposed through Qt, such as AES.

New (optional) submodule: GTAD

Since libQuotient 0.7, you can add and manage GTAD as a submodule within the library's Git tree (#561). Normally you won't need that and even if the submodule is checked out, GTAD building process is intentionally not integrated into the library building in any way. Be mindful that GTAD build requirements may be different from those of the library; as for now, they happen to be the same but there's a high probabilty that GTAD will adopt C++23 (and require newer compilers therefore) before the library does.

Compatibility and maintenance

As before, 0.7 is not backwards compatible with 0.6; fundamentally it's the same API but there are quite a few breaking changes. Micro-versions (0.7.z) will maintain API and ABI compatibility, until 0.8 breaks it again.

With this release, 0.5 becomes officially unmaintained (it didn't receive a single commit for the last couple of years anyway, so it's a pure formality). 0.6 is now on "sustaining support", only receiving critical fixes (e.g. to keep the older clients somehow workable as the Matrix standard changes).

CI changes

After changing their owner, Travis CI abruptly went to a new billing scheme at the end 2020, leaving (non-paying) open-source projects they had long welcomed with no allowance to run builds for more than two months, effectively pushing them to find a new home. Fortunately, GitHub Actions matured just in time and had all the minimally necessary features, so it was adopted for Linux and macOS pipelines (#434). And since GHA also supported Windows as a CI platform, we have migrated from AppVeyor as well (#495) once the migration from Travis CI proved successful. Turns out that Travis CI really lost their integrity under the new owner, suffering from breaches one after another - so, not missing them at all and recommending to move over to anyone who's somehow still with them, paid or not.

To improve on the integrity of our own code, we moved from LGTM - barely maintained after GitHub bought them - to GitHub's own action using the same CodeQL technology LGTM used (cc69883, #495 again). Much bigger static analysis fun came with SonarCloud adoption (originally in 0a46049, with many updates since). The first analysis came up with 1300+ "code smell" warnings; after massaging the Sonar profile the number of warnings halved; and after some massive cleanup effort the number went down to below 300. You can see the whole story at the libQuotient's page in SonarCloud. Another thing brought in with Sonar is coverage tracking (ebabb2f); that triggered me (@KitsuneRal) to actually execute autotests (originally championed by @CarlSchwan) in CI just before running Quotest (#523). The coverage numbers still don't impress: even after adding a few more tests we're still at around 42% so far (and the number will go down if we keep adding things without adding tests - hint, hint!).

Changes

Notable changes since 0.6.11 are listed below; if you want to go through all small things as well, check out the full commit log and the list of closed issues.

E2EE

As mentioned above, E2EE is the biggest part of this release, and @TobiasFella made most of the heavy-lifting. The following parts of E2EE are known to work:

• foundations (#506, #552; additional contributions and refactoring in #550, #556, #571, #581, #582, #589 by @KitsuneRal)
• Olm/Megolm signalling with our parties/devices (#534, #544, #560, #595)
• managing device and one-time keys (#535, #542)
• sending (#540, #572) and receiving (monstrous #477, together with @a-andreyev, @CarlSchwan and @KitsuneRal; #536, #537, #538) new encrypted messages ;
• encrypting/decrypting attachments (#505, #543, #553);
• retrieval and decryption of historical messages keys for which are already in the database (#583, #590)
• device verification (#541, #547, #566, #568, #573, #575, #577; #594 by @nicolasfella)

What is not there yet:

• historical Megolm sessions are not requested from other devices/parties, so you will see quite a few undecryptable messages in rooms with past communication - this might be fixed (if possible without breaking the API) in further 0.7.x releases
• secure server-side storage (SSSS) is not supported at all for now, and will likely require API breakage so will come in 0.8
• soft logout is not supported; this is also a subject of 0.8

Because there's no soft logout the database for a given account is completely reset upon a successful login attempt. This may come as a very unpleasant surprise if you don't keep your login session between client restarts; but this is a necessary trade-off for the current feature set (see #546 for the discussion). Together with the rest of the above not-there-yet list, this means that you MUST NOT use E2EE with libQuotient-backed clients as your only device(s) on the account - there's quite a risk of losing encrypted conversations if anything happens to the database libQuotient keeps key material in, or if you lose your session and have to log in again. Did I mention the whole E2EE functionality is still in beta? Client authors are strongly recommended to show big scary warnings against E2EE for now.

Read markers -> read receipts + fully read markers

First, a bit of history. Originally, Matrix only had a single kind of read marker, m.read; and back in older times clients diverged on when this read marker should be updated. Quaternion and libQuotient (back then libQMatrixClient), for one, took that m.read signified the "fully read" state and the user should scroll back to the marker and read the messages successively in order for the marker to slide over them (or they can force the whole room as "fully read"). Element and some other clients took an approach where merely opening the room updated m.read to the latest currently displayed message. This led to confusion when both kinds of clients were used on the same account and depending on which one displayed the timeline last, m.read was updated differently.

At a certain point the spec received the m.fully_read marker, making it possible for the user to track their progress through the room both ways at the same time. However, libQuotient 0.6 chose to coalesce the two into a single "read marker" to avoid a massive rewrite, late in ...

Version 0.7 RC

As we approach the long-awaited 0.7 release, this pre-release contains (most) of the last bug fixes and improvements, in particular:

• (#581) Tighter internal code for Olm interfacing, by @KitsuneRal
• (#586) Proper treatment of null values in JSON, by @TobiasFella
• (#585) Fix sending files on android, by @TobiasFella
• (#589) Pickling key and random material are now located in a separate protected heap using OpenSSL facilities, by @KitsuneRal
• (#590) No more repeated attempts to recover broken olm sessions, by @TobiasFella
• (#593) More careful tracking of what is considered the beginning of the timeline, by @KitsuneRal

Full Changelog: 0.7-beta2...0.7-rc

Version 0.7 beta 2

This beta is mainly about fixes to issues discovered after beta 1, with a few small improvements on top.

• (#577, @TobiasFella) Fix Key Verification and its test - a known bug as of beta 1, thanks to Tobias for getting to the bottom of it
• (#578, @KitsuneRal) The API for event edits is consolidated in RoomMessageEvent - this is in alignment with the specification, client application authors may have to fix some type casts in their code if they were trying to use this API in an incompliant way before
• (#579, @KitsuneRal ) Tighten up ReactionEvent interface, to make sure only reactions can be put in objects of this class, previously you could stuff any relation in it which certainly wasn't the purpose of the class
• (#580, @subpop) Add UserIdRole to AccountRegistry - first contribution of @subpop, thanks and congrats!
• (#582, @TobiasFella) Fix a broken validation of Olm account unpickling, causing a misleading warning in the logs
• (#583, @TobiasFella) Recover from some decryption errors
• (a904108, @KitsuneRal) Fix buildability of stickerevent.h - the file is not used inside Quotient (library tests still have huge gaps...)
• (587daae, 848797e, @KitsuneRal) More compile-time validations to protect against common errors in using QUO_*EVENT macros
• (72e7665) Refreshed Client-Server API files, enabling clients to pass thread_id to PostReceiptJob

Full Changelog: 0.7-beta1...0.7-beta2

Version 0.7 beta 1

It took a long time since 0.6 was out but here comes the first beta of libQuotient 0.7! Full release notes are left for the final release; in the meantime, here are the most important points:

Main news

The biggest news (and the main reason it took so long to get to beta) is that the work on E2EE has finally reached beta quality and is available behind a CMake switch! @TobiasFella and @CarlSchwan have done most of the hard work - without them that would not have happened. In total recognition of the urge to flip that switch on :) bear in mind that this is BETA and the implementation is not quite complete. Specifically, implemented areas include:

• sending and receiving encrypted messages (with key requests from Quotient, if needed);
• sending and receiving encrypted attachments;
• device keys verification (without the visual part - that has to be done by each client)
  Not implemented:
• keys backup;
• returning E2EE key requests from other clients

There also are/may be issues around this code; overall, E2EE is currently barely usable as a daily driver but certainly good enough for testing. Be very careful, backup your Quotient data and assume that this code is far from being rock-solid. We hope not to keep E2EE in beta for 2 years like Element did (they were the very first to implement E2EE, after all) but don't expect this functionality to stabilise any time this year.

Other news

• Using Matrix v1.4 as the Client-Server API basis - implying that you can use CS API calls related to spaces and threading, even though the library doesn't have native classes for that).
  • The infrastructure to generate API *Job classes has been migrated to work with the matrix-spec repository that is the source of The Spec (instead of matrix-doc) since recently.
• libQuotient can build with Qt 6, next to Qt 5; this is somewhat experimental still but will become the primary configuration before the release.
• Preliminary support of mxc URLs in Quotient::NetworkAccessManager; with certain caveats, this means that mxc URLs can be loaded from inside QML/HTML code (i.e. to show inline images) - thanks to @TobiasFella.
• Initial support for sticker events (basically - StickerEvent type) - thanks to @CarlSchwan.
• Read receipts are handled completely separately from the fully-read marker, bringing libQuotient in compliance with the spec on that front (previously there was a converged "last read marker" that was closer to the fully-read marker with the read receipt pinned to it).
• Pinned messages support - thanks to @arawaaa
• Account tokens (among other key things) are now stored in the keychain - thanks to @TobiasFella

Here's the full changelog.

Compatibility

As before, the API has changed in 0.7. The codebase made for 0.6 will most likely not build with 0.7 beyond basic functionality. There are no guarantees that even the API will remain frozen between the beta and the release but the changes should be relatively minor from now (except the E2EE code; but that is mostly internal to libQuotient).

NB: the library will report its version as 0.7.0 (this is mainly because there were already some 0.7-git kind of packages in the wild).

Toolchain

Version 0.6 took C++17 on; with 0.7, the library is moving further and requires a C++20 toolchain. Full C++20 compliance is not necessary (see README.md for details); basically, you'll need a compiler (and a library) that is no older than a couple of years. CMake 3.16 is now the oldest that libQuotient can be configured with.

The required Qt version is also much newer now. The only acceptable Qt 5 version is Qt 5.15 (it's also the one recommended for the moment - this may change before 0.7 is released). The library is now buildable with Qt 6 as well - any version will do but at least 6.2 is strongly recommended because it's the first Qt 6 LTS version.

With 0.7, libQuotient gains one unconditional and two conditional dependencies:

• QtKeychain; clients previously used it to securely store access tokens, now the library does it for them.
• (only with E2EE) libolm and OpenSSL; both OpenSSL 1.1 and 3.x are supported.

Being experimental, E2EE code has only been tried on Linux. Please let us know (via GitHub issues) if you face problems on Windows and/or macOS.

Version 0.6.11

Just after 0.6.10 was out another nasty bug has been discovered, preventing from proper usage of URLs with an unescaped double-hash (##), which may come when a matrix.to URL is generated for a bridged IRC channel (and it even got a very nice number #512). Aside from this fix, quotest will not return with success code 0 any more if it failed to even start testing for some reason (#496).

Version 0.6.10

Yet another maintenance release of the stable branch, fixing an issue (#510) with invites not showing up in some cases (especially on less active/smaller accounts) - thanks to @TobiasFella for spotting that.

Version 0.6.9

Here's one more stable branch release of libQuotient, fixing a rare assertion failure due to uninitialised read marker when new events arrive and all history is already loaded. Thanks to @ptman for his help with pinning this down.