Cardano SL 2.0.0

Cardano SL 2.0.0

Features

• Reduce number of files on disk

  Blund files (containing blocks and their undos) for older epochs (from zero up to current
  epoch minus two) are now consolidated into epoch/index file pairs. That means that the number
  of files to store the block chain for a single epoch is reduced from 21600 (one blund file for
  each slot in an epoch) to an epoch/index file pair. Consolidation happens on-the-fly in a
  background process.

• Add bouncing and throttling to the API

  Previously, exchanges could accidentally overload their wallet servers. We
  have added configurable throttling to the API service to prevent this
  problem. To configure this, setting, view the changes in the
  configuration.yaml file under the wallet section. The API will now return
  a 429 error containing the microseconds to wait until retry. (CBR-179, #3431)

• We can force an NTP-check when getting node-info via the API (?force_ntp_check query flag)

  • CO-325, #3372, #3461, #3607
  • CBR-427 #3586

• The API provides an endpoint to retrieve basic statistics on the UTxO distribution of a wallet (/api/v1/wallets/{walletId}/statistics) (CO-347, #3402)

  • CO-388, #3610
  • CO-389, #3620, #3658

• cardano-sl exposes a new package x509 with tooling for defining a PKI infrastructure from pure Haskell. This is basically an export of the internals of the tool cardano-sl-x509-generate

  • DEVOPS-992, #3367
  • CO-388, #3610
  • CO-389, #3620, #3658

• Structured logging (CBR-97 #3483 #3645, CBR-207, #3476 #3477, CBR-211 #3507, CBR-213, #3481, DEVOPS-1097, #3764, #3395, #3443, DEVOPS-1109, #3785, CBR-275 #3533 #3534 #3655, CBR-345 #3526 #3613 #3632 #3633 #3709, CBR-348 #3523, CBR-430 #3603, CBR-423 #3609, RCD-42 #3816)

• New data layer for wallet (CBR-150, #3245,
  CBR-227,
  #3393
  )

• Enable new data layer in Docker images for exchanges (DEVOPS-1037, #3545, DEVOPS-1046, #3594).

• Address Discrimination

  • Set up supporting tests (CDEC-500: #3403, #3405)
  • Fix JSON golden test helpers (CDEC-506: #3415)
  • Adjust the address format to distinguish testnet and mainnet addresses (CO-410: #3715, #3733, #3756, #3775)

• New cluster package with utility and CLI to start a full-fledged cluster of nodes

  • CO-423, #3732
  • CO-371, #3605
  • CO-385, #3608
  • CO-390, #3629

• Support query against some fields of the Account resource (balance, addresses) enabling client to fetch only the data they need (CO-324, #3210)

• Integration with the new data-layer

  • CBR-437, #3621
  • CBR-446, #3650
  • CBR-462 #3704
  • CBR-464 #3710
  • CBR-471 #3786
  • CO-367 #3661
  • CBR-366 #3469
  • RCD-44 #3875
  • RCD-45 #3875
  • RCD-46 #3875
  • RCD-47 #3876
  • CBR-495 #3943
  • CBR-496 #3947

• Finalize port of API V0 to V1

  • CO-334 #3197

• Expose ntp client api for makeing forceful ntp checks and a review of the ntp client code base CDEC-355

  • #3323
  • #3264

Fixes

• Make productionReporter more robust

  Add exception handling code in reporting exception handler, to prevent IOExceptions from killing
  the main thread. This was noticed when the network connection was interrupted, and the reporter
  died when it tried to report over the down network. (CDEC-470, #3365)

• Improve type safety (and as a consequence, API documentation) of account indexes (CBR-306, #3086)

• The Swagger specification had names with illegal characters. These names
  where changed to be URL friendly. PR #3595

• The creation of mnemonic doesn't throw anymore when provided words outside of the BIP39 English dictionnary.
  Instead, it returns an error value gracefully (CO-325)

• Response from JSONValidationError are now also encoded inline (instead of a pretty-encoding with newlines) (DDW-318, #3619)

• [API BREAKING CHANGE] The behavior of /api/v1/addresses/{address} has been adjusted to reflect more accurately
  the meaning of ownership regarding addresses.
  The previous version of this endpoint failed with an HTTP error when the given address was unknown to the wallet.
  This was misleading since an address that is unknown to the wallet may still belong to the wallet. To reflect this,
  the V1 endpoint does not fail anymore as it used to when an address is not recognised and returns instead a new field
  'is-ours' which indicates either that an address is ours, or that it is 'not-recognised'. (CBR-401, #3646)

• [API BREAKING CHANGE] A DELETE request to /api/v1/wallets/{wallet} now correctly fails with 404 if the wallet doesn't exist. Previously it incorrectly responded with 204.

• Fix commitAndReleaseBuffer: invalid argument (invalid character) error in Docker image (DEVOPS-877, #3173)

• Fix deadlock triggered by update/shutdown (CBR-424 #3731)

• Fix space leak during network syncing (CBR-454 #3679)

• Fix benchmark on OSX (CSL-2517 #3348)

• Fix logger implementation, enabling pure logger to be used without side-effects (CO-409, #3697)

• Crash host node when the underlying wallet dies (CBR-263, #3584)

• Ensure correct file permissions are set when generate x509 certificates (CBR-470, #3773)

• Fix checksum verification in BIP-39 implementation (CO-298, #3013)

• Fix wallet starting bug introduced by CDEC-509 (CBR-400: #3486)

• Fix Haddock errors (CDEC-585: #3614)

• Fix restoration ignoring new accounts in legacy data layer (DEVOPS-1153: #3911)

• Tweaks to Cardano Explorer for Testnet (DEVOPS-1094, #3817, DEVOPS-1121, #3831, RCD-48, #3883)

• Fix bug in Windows launcher where upgrading to 2.0.0 would terminate any other Daedalus wallets that were running (RCD-52, #3926)

Improvements

• Friendly error mistakes from deserializing invalid addresses instead of brutal 500 (CBR-283)

• [API BREAKING CHANGE] Add walletId to WalletAlreadyExists WalletLayerError (CBR-254)

• Small refactor of wallet Errors implementation to be more maintainable (CBR-26, #3429)

• Content-Type parser is now more lenient and accepts application/json, application/json;charset=utf-8 and no Content-Type at all (defaulting to application/json) (CO-369, #3596)

• The codebase now relies on the package cryptonite (instead of ed25519) for Ed25519 implementation (CO-325)

• [API BREAKING CHANGE] Improve diagnostic for NotEnoughMoney error (CBR-461, #3702)

• When Content-Type's main MIME-type cannot fall back to 'application/json' then UnsupportedMimeTypeError is returned (CO-416, #3727)

• Add cardano-node --no-tls option to wallet (DEVOPS-879, #3074)

• Improve error reporting when a worker thread in cardano dies (DEVOPS-1063, #3664)

• Add failure injection options to wallet for Daedalus testing (DEVOPS-1086, #3787)

• Benchmarking support (PB-20 PB-95 #3670)

• Reorganize and clean up core packages

  • Move code from infra to sinbin so that infra can be moved "up" in the dependency graph (CDEC-416: #3185, #3202, #3209)
  • Move code from networking to core, so that networking can be moved "up" (CDEC-432: #3238, #3261, #3266)
  • Remove unnecessary Mockable typeclass (CDEC-451: #3285)
  • Switch uses of pipes to conduit, for consistency (CDEC-464: #3305)
  • Remove partial-function record accessors using deriveIndexedBi TH function (CDEC-385: #3153)
  • Create a script to visualize the package dependency graph within cardano-sl (CDEC-429: #3227)
  • Remove Blockchain class and clean up Block modules (CDEC-333: #3615)
  • Remove HasProtocolConstants reflection constraint in favour of explicit parameters (CDEC-369: #3482)
  • Remove remaining reflection constraints from core configuration (CDEC-509: #3437, #3505, #3522, #3549, #3550, #3570)
  • Add stylish-haskell enforcement in CI to keep code conformant (CDEC-383: #3142)
  • Move Block datatypes from core to chain (CDEC-485: #3351)
  • Move chain-related core types to chain package (CDEC-505: #3412, #3593, #3600, #3601, #3611)
  • Add golden tests for Undo type (CDEC-623: #3735)
  • Reunite orphan instances
    • in txp (CDEC-418: #3179)
    • in delegation (CDEC-423: #3200)
    • in chain (CDEC-484: #3357)
    • in lib - SafeCopy instances (CDEC-377: #3135)
  • Remove -fno-warn-orphans from ghc-options in update-test (CDEC-455: #3296)
  • Move Arbitrary instances from wallet to wallet-test (CDEC-437: #3259)
  • Move Pos.Core.Genesis.Canonical to Pos.Util.Json.Canonical (CDEC-513: #3445)
  • Weed out unused package dependencies (CDEC-425: #3240)

• Improve readability and execution of various integration tests

  • CO-398, #3678
  • CO-400, #3680
  • CO-436, #3791
  • CBR-408, #3571
  • CBR-417, #3568
  • CO-357, #3573, #3639

• Add integration tests to test redemption of certificates (CBR-398, #3525)

• Review implementation of the BIP39 (Mnemonic Words) implementation

  • CBR-288 #3128
  • CBR-289 #3043

• Add a test which checks if the configuration can be correctly parsed

  • CDEC-405 #3175

Documentation

• Make an inventory of existing wallet errors and exceptions (CBR-307)

• Various API documentation / guides fixes

  • CO-327, #3215
  • CO-328, #3212
  • CO-351 #3391

• Documentation updates for Nix 2.0 (DEVOPS-976, #3311, #3343)

Continuous Integration (CI)

• Add a nightly test which syncs mainnet from scratch (DEVOPS-1016, #3487, DEVOPS-1052, #3626)
• CI speed improvements (DEVOPS-1032, #3544)
• Fixes for regeneration of pkgs/default.nix (DEVOPS-1045, #3677)
• Ensure the nix-shell environment is built and cached by Hydra (DEVOPS-1059, #3653)
• Add support for building cardano-sl with cabal new-build (DEVOPS-1061, #3662, #3729, #3734, DEVOPS-1060, #3675)
• Refactor nix files, add comments and documentation (DEVOPS-1083, #3728, #3760, #3761, #3763, DEVOPS-1004, #3400, DEVOPS-1067, #3690)
• Add stylish-haskell tests to the Hydra build (DEVOPS-936, #3166, #3189, #3222)
• Improve nixpkgs pinning (DEVOPS-779, #3180)
• Fix caching of nix builds (DEVOPS-810, #3174)
• Ensure CI builds all targets (including benchmarks), and runs all tests (DEVOPS-908, #3130, #3150)
• Build fixes for cardano-sl-explorer-frontend (DEVOPS-916, #3137, #3146, DEVOPS-999, #3472)
• Fix nix-build when run from a git worktree (DEVOPS-949, #3691)
• Windows build fixes (DEVOPS-957, #3272, DEVOPS-1003, #3398)
• macOS build fixes (clang: Argument list too long) (DEVOPS-1005, #3432, DEVOPS-1050, #3606)
• Improve wallet integration tests (DEVOPS-980, #3325, #3380, DEVOPS-988, #3346)
• Demo cluster and launch script fixes (DEVOPS-985, #3342, DEVOPS-1062, #3665, DEVOPS-1101, #3769)
• Add timing information to nix builds (DEVOPS-1013, #3457, DEVOPS-1027, #3509, DEVOPS-1048, #3706)
• Better code linting in CI (DEVOPS-1031, #3527, DEVOPS-1057, #3649, DEVOPS-1100, #3762)