From 28ed56f5d710e79252137f997cbd4e48c7d45716 Mon Sep 17 00:00:00 2001 From: Peter Taoussanis Date: Wed, 26 Jul 2023 16:24:40 +0200 Subject: [PATCH] [nop] Update README template --- .gitignore | 1 + FUNDING.yml | 2 +- LICENSE => LICENSE.txt | 0 README.md | 459 ++++------------------------------ navicat-logo.png | Bin 13372 -> 0 bytes wiki/1-Further-usage.md | 154 ++++++++++++ wiki/2-Message-queue.md | 31 +++ wiki/3-Community-resources.md | 8 + wiki/Home.md | 170 +++++++++++++ 9 files changed, 407 insertions(+), 418 deletions(-) rename LICENSE => LICENSE.txt (100%) delete mode 100644 navicat-logo.png create mode 100644 wiki/1-Further-usage.md create mode 100644 wiki/2-Message-queue.md create mode 100644 wiki/3-Community-resources.md create mode 100644 wiki/Home.md diff --git a/.gitignore b/.gitignore index 27d94bae..bc0a2d4f 100644 --- a/.gitignore +++ b/.gitignore @@ -14,3 +14,4 @@ pom.xml* /doc/ .idea/ *.iml +/wiki/.git diff --git a/FUNDING.yml b/FUNDING.yml index dc3d4d22..964e36a0 100644 --- a/FUNDING.yml +++ b/FUNDING.yml @@ -1,2 +1,2 @@ github: ptaoussanis -custom: "https://www.taoensso.com/clojure/backers" +custom: "https://www.taoensso.com/clojure" diff --git a/LICENSE b/LICENSE.txt similarity index 100% rename from LICENSE rename to LICENSE.txt diff --git a/README.md b/README.md index 3eb842ef..f72a8a87 100644 --- a/README.md +++ b/README.md @@ -1,440 +1,65 @@ - -Taoensso open-source +Taoensso open source +[**Documentation**](#documentation) | [Latest releases](#latest-releases) | [Get support][GitHub issues] -**[CHANGELOG][]** | [API][] | current [Break Version][]: +# Carmine -```clojure -[com.taoensso/carmine "3.3.0-RC1"] ; Upcoming, see CHANGELOG for details -[com.taoensso/carmine "3.2.0"] ; Stable -``` +### [Redis](https://en.wikipedia.org/wiki/Redis) client + message queue for Clojure -> See [here][backers] if to help support my open-source work, thanks! - [Peter Taoussanis][Taoensso.com] +Redis and Clojure are individually awesome, and **even better together**. -# Carmine: Clojure [Redis][] client & message queue +Carmine is a mature Redis client for Clojure that offers an idiomatic Clojure API with plenty of **speed**, **power**, and **ease-of-use**. -Redis and Clojure are individually awesome, and **even better together**. +## Latest release/s -Carmine is a mature Redis client for Clojure that offers an idiomatic Clojure API with plenty of **speed**, **power**, and **ease of use**. +- `2023-07-18` `3.3.0-RC1` (dev): [changes](../../releases/tag/v3.3.0-RC1) +- `2022-12-03` `3.2.0` (stable): [changes](../../releases/tag/v3.2.0) -## Features - * Fast, simple **all-Clojure** library. - * Fully documented [API][] with support for the **latest Redis commands**. - * Production-ready **connection pooling**. - * Auto **de/serialization** of Clojure data types via [Nippy][]. - * Fast, simple **message queue** API. - * Fast, simple **distributed lock** API. - * Support for **Lua scripting**, **Pub/Sub**, etc. - * Support for custom **reply parsing**. - * Includes **Ring session-store**. +[![Main tests][Main tests SVG]][Main tests URL] +[![Graal tests][Graal tests SVG]][Graal tests URL] -An major upcoming (v4) release will include support for RESP3, Redis Sentinel, Redis Cluster, and more. +See [here][GitHub releases] for earlier releases. -## Quickstart +## Why Carmine? -Add the necessary dependency to your project: +- High-performance **pure-Clojure** library +- [Fully documented API](#documentation) with support for the **latest Redis commands and features** +- Easy-to-use, production-ready **connection pooling** +- Auto **de/serialization** of Clojure data types via [Nippy](https://www.taoensso/nippy) +- Fast, simple **message queue** API +- Fast, simple **distributed lock** API -```clojure -Leiningen: [com.taoensso/carmine "3.3.0-RC1"] ; or -deps.edn: com.taoensso/carmine {:mvn/version "3.3.0-RC1"} -``` +## Documentation -And setup your namespace imports: +- [Full documentation][GitHub wiki] (**getting started** and more) +- Auto-generated API reference: [Codox][Codox docs], [clj-doc][clj-doc docs] -```clojure -(ns my-app - (:require [taoensso.carmine :as car :refer [wcar]])) -``` +## Funding -You'll usually want to define a single **connection pool**, and one **connection spec** for each of your Redis servers, example: +You can [help support][sponsor] continued work on this project, thank you!! 🙏 -```clojure -(defonce my-conn-pool (car/connection-pool {})) ; Create a new stateful pool -(def my-conn-spec-1 {:uri "redis://redistogo:pass@panga.redistogo.com:9475/"}) - -(def my-wcar-opts - {:pool my-conn-pool - :spec my-conn-spec-1}) -``` - -This `my-wcar-opts` can then be provided to Carmine's `wcar` ("with Carmine") API: - -```clojure -(wcar my-wcar-opts (car/ping)) ; => "PONG" -``` - -`wcar` is the main entry-point to Carmine's API, see its [docstring](http://ptaoussanis.github.io/carmine/taoensso.carmine.html#var-wcar) for more info on pool and spec opts, etc. - -You can create a `wcar` partial for convenience: - -```clojure -(defmacro wcar* [& body] `(car/wcar ~my-wcar-opts ~@body)) -``` - -### Command pipelines - -Calling multiple Redis commands in a single `wcar` body uses efficient Redis [pipelining][] under the hood, and returns a pipeline reply (vector) for easy destructuring: - -```clojure -(wcar* - (car/ping) - (car/set "foo" "bar") - (car/get "foo")) ; => ["PONG" "OK" "bar"] (3 commands -> 3 replies) -``` - -If the number of commands you'll be calling might vary, it's possible to request that Carmine _always_ return a destructurable pipeline-style reply: - -```clojure -(wcar* :as-pipeline (car/ping)) ; => ["PONG"] ; Note the pipeline-style reply -``` - -If the server replies with an error, an exception is thrown: - -```clojure -(wcar* (car/spop "foo")) -=> Exception ERR Operation against a key holding the wrong kind of value -``` - -But what if we're pipelining? - -```clojure -(wcar* - (car/set "foo" "bar") - (car/spop "foo") - (car/get "foo")) -=> ["OK" # "bar"] -``` - -### Serialization - -The only value type known to Redis internally is the [byte string][]. But Carmine uses [Nippy][] under the hood and understands all of Clojure's [rich datatypes][], letting you use them with Redis painlessly: - -```clojure -(wcar* - (car/set "clj-key" - {:bigint (bigint 31415926535897932384626433832795) - :vec (vec (range 5)) - :set #{true false :a :b :c :d} - :bytes (byte-array 5) - ;; ... - }) - - (car/get "clj-key")) - -=> ["OK" {:bigint 31415926535897932384626433832795N - :vec [0 1 2 3 4] - :set #{true false :a :c :b :d} - :bytes #}] -``` - -Types are handled as follows: - -Clojure type | Redis type ------------------------- | -------------------------- -Strings | Redis strings -Keywords | Redis strings -Simple numbers | Redis strings -Everything else | Auto de/serialized with [Nippy][] - -You can force automatic de/serialization for an argument of any type by wrapping it with `car/freeze`. - -### Command coverage and documentation - -Carmine uses the [official Redis command spec][commands.json] to generate its own command API and documentation: - -```clojure -(clojure.repl/doc car/sort) - -=> "SORT key [BY pattern] [LIMIT offset count] [GET pattern [GET pattern ...]] [ASC|DESC] [ALPHA] [STORE destination] - -Sort the elements in a list, set or sorted set. - -Available since: 1.0.0. - -Time complexity: O(N+M*log(M)) where N is the number of elements in the list or set to sort, and M the number of returned elements. When the elements are not sorted, complexity is currently O(N) as there is a copy step that will be avoided in next releases." -``` - -Each Carmine release will use the latest command spec. But if a new Redis command is available that hasn't yet made it to Carmine, you can always use the [redis-call](http://ptaoussanis.github.io/carmine/taoensso.carmine.html#var-redis-call) function to issue **arbitrary commands**. - -This is also useful for **Redis module** commands, etc. - -### Lua scripting - -Redis offers powerful [Lua scripting](http://redis.io/commands/eval/) capabilities. - -As an example, let's write our own version of the `set` command: - -```clojure -(defn my-set - [key value] - (car/lua "return redis.call('set', _:my-key, 'lua '.. _:my-val)" - {:my-key key} ; Named key variables and their values - {:my-val value} ; Named non-key variables and their values - )) - -(wcar* - (my-set "foo" "bar") - (car/get "foo")) -=> ["OK" "lua bar"] -``` - -Script primitives are also provided: `eval`, `eval-sha`, `eval*`, `eval-sha*`. - -### Helpers - -The `lua` command above is a good example of a Carmine *helper*. - -Carmine will never surprise you by interfering with the standard Redis command API. But there are times when it might want to offer you a helping hand (if you want it). Compare: - -```clojure -(wcar* (car/zunionstore "dest-key" 3 "zset1" "zset2" "zset3" "WEIGHTS" 2 3 5)) -(wcar* (car/zunionstore* "dest-key" ["zset1" "zset2" "zset3"] "WEIGHTS" 2 3 5)) -``` - -Both of these calls are equivalent but the latter counted the keys for us. `zunionstore*` is another helper: a slightly more convenient version of a standard command, suffixed with a `*` to indicate that it's non-standard. - -Helpers currently include: `atomic`, `eval*`, `evalsha*`, `info*`, `lua`, `sort*`, `zinterstore*`, and `zunionstore*`. See their docstrings for more info. - -### Commands are (just) functions - -In Carmine, Redis commands are *real functions*. Which means you can *use* them like real functions: - -```clojure -(wcar* (doall (repeatedly 5 car/ping))) -=> ["PONG" "PONG" "PONG" "PONG" "PONG"] - -(let [first-names ["Salvatore" "Rich"] - surnames ["Sanfilippo" "Hickey"]] - (wcar* (mapv #(car/set %1 %2) first-names surnames) - (mapv car/get first-names))) -=> ["OK" "OK" "Sanfilippo" "Hickey"] - -(wcar* (mapv #(car/set (str "key-" %) (rand-int 10)) (range 3)) - (mapv #(car/get (str "key-" %)) (range 3))) -=> ["OK" "OK" "OK" "OK" "0" "6" "6" "2"] -``` - -And since real functions can compose, so can Carmine's. By nesting `wcar` calls, you can fully control how composition and pipelining interact: - -```clojure -(let [hash-key "awesome-people"] - (wcar* - (car/hmset hash-key "Rich" "Hickey" "Salvatore" "Sanfilippo") - (mapv (partial car/hget hash-key) - ;; Execute with own connection & pipeline then return result - ;; for composition: - (wcar* (car/hkeys hash-key))))) -=> ["OK" "Sanfilippo" "Hickey"] -``` - -### Listeners & Pub/Sub - -Carmine has a flexible **Listener** API to support persistent-connection features like [monitoring](http://redis.io/commands/monitor) and Redis's fantastic [Pub/Sub](http://redis.io/topics/pubsub) facility: - -```clojure -(def listener - (car/with-new-pubsub-listener (:spec server1-conn) - {"foobar" (fn f1 [msg] (println "Channel match: " msg)) - "foo*" (fn f2 [msg] (println "Pattern match: " msg))} - (car/subscribe "foobar" "foobaz") - (car/psubscribe "foo*"))) -``` - -Note the map of message handlers. `f1` will trigger when a message is published to channel `foobar`. `f2` will trigger when a message is published to `foobar`, `foobaz`, `foo Abraham Lincoln`, etc. - -Publish messages: - -```clojure -(wcar* (car/publish "foobar" "Hello to foobar!")) -``` - -Which will trigger: - -```clojure -(f1 '("message" "foobar" "Hello to foobar!")) -;; AND ALSO -(f2 '("pmessage" "foo*" "foobar" "Hello to foobar!")) -``` - -You can adjust subscriptions and/or handlers: - -```clojure -(with-open-listener listener - (car/unsubscribe) ; Unsubscribe from every channel (leave patterns alone) - (car/psubscribe "an-extra-channel")) - -(swap! (:state listener) assoc "*extra*" (fn [x] (println "EXTRA: " x))) -``` - -**Remember to close the listener** when you're done with it: - -```clojure -(car/close-listener listener) -``` - -Note that subscriptions are *connection-local*: you can have three different listeners each listening for different messages, using different handlers. This is great stuff. - -### Reply parsing - -Want a little more control over how server replies are parsed? You have all the control you need: - -```clojure -(wcar* - (car/ping) - (car/with-parser clojure.string/lower-case (car/ping) (car/ping)) - (car/ping)) -=> ["PONG" "pong" "pong" "PONG"] -``` - -### Binary data - -Carmine's serializer has no problem handling arbitrary byte[] data. But the serializer involves overhead that may not always be desireable. So for maximum flexibility Carmine gives you automatic, *zero-overhead* read and write facilities for raw binary data: - -```clojure -(wcar* - (car/set "bin-key" (byte-array 50)) - (car/get "bin-key")) -=> ["OK" #] -``` - -### Message queue - -Redis makes a great [message queue server](http://antirez.com/post/250): - -```clojure -(:require [taoensso.carmine.message-queue :as car-mq]) ; Add to `ns` macro - -(def my-worker - (car-mq/worker {:pool {} :spec {}} "my-queue" - {:handler (fn [{:keys [message attempt]}] - (println "Received" message) - {:status :success})})) - -(wcar* (car-mq/enqueue "my-queue" "my message!")) -%> Received my message! - -(car-mq/stop my-worker) -``` - -Guarantees: - * Messages are persistent (durable) as per Redis config - * Each message will be handled once and only once - * Handling is fault-tolerant: a message cannot be lost due to handler crash - * Message de-duplication can be requested on an ad hoc (per message) basis. In these cases, the same message cannot ever be entered into the queue more than once simultaneously or within a (per message) specifiable post-handling backoff period. - -See the relevant [API][] docs for details. - -### Distributed locks - -```clojure -(:require [taoensso.carmine.locks :as locks]) ; Add to `ns` macro - -(locks/with-lock - {:pool {} :spec {}} ; Connection details - "my-lock" ; Lock name/identifier - 1000 ; Time to hold lock - 500 ; Time to wait (block) for lock acquisition - (println "This was printed under lock!")) -``` - -Again: simple, distributed, fault-tolerant, and _fast_. - -See the relevant [API][] docs for details. - -## Tundra (beta) - -Redis is a beautifully designed datastore that makes some explicit engineering tradeoffs. Probably the most important: your data _must_ fit in memory. Tundra helps relax this limitation: only your **hot** data need fit in memory. How does it work? - - 1. Use Tundra's `dirty` command **any time you modify/create evictable keys** - 2. Use `worker` to create a threaded worker that'll **automatically replicate dirty keys to your secondary datastore** - 3. When a dirty key hasn't been used in a specified TTL, it will be **automatically evicted from Redis** (eviction is optional if you just want to use Tundra as a backup/just-in-case mechanism) - 4. Use `ensure-ks` **any time you want to use evictable keys** - this'll extend their TTL or fetch them from your datastore as necessary - -That's it: two Redis commands, and a worker! Tundra uses Redis' own dump/restore mechanism for replication, and Carmine's own message queue to coordinate the replication worker. - -Tundra can be _very_ easily extended to **any K/V-capable datastore**. Implementations are provided out-the-box for: **Disk**, **Amazon S3** and **DynamoDB**. - -```clojure -(:require [taoensso.carmine.tundra :as tundra :refer (ensure-ks dirty)] - [taoensso.carmine.tundra.s3]) ; Add to ns - -(def my-tundra-store - (tundra/tundra-store - ;; A datastore that implements the necessary (easily-extendable) protocol: - (taoensso.carmine.tundra.s3/s3-datastore {:access-key "" :secret-key ""} - "my-bucket/my-folder"))) - -;; Now we have access to the Tundra API: -(comment - (worker my-tundra-store {} {}) ; Create a replication worker - (dirty my-tundra-store "foo:bar1" "foo:bar2" ...) ; Queue for replication - (ensure-ks my-tundra-store "foo:bar1" "foo:bar2" ...) ; Fetch from replica when necessary -) -``` - -Note that this API makes it convenient to use several different datastores simultaneously (perhaps for different purposes with different latency requirements). - -See the relevant [API][] docs for details. - -## Performance - -Redis is probably most famous for being [fast](http://redis.io/topics/benchmarks). Carmine hold up its end and usu. performs w/in ~10% of the official C `redis-benchmark` utility despite offering features like command composition, reply parsing, etc. - -## Thanks to Navicat - -![Navicat-logo](https://github.com/ptaoussanis/carmine/blob/master/navicat-logo.png) - -Carmine's SQL-interop features (forthcoming) were developed with the help of [Navicat](http://www.navicat.com/), kindly sponsored by PremiumSoft CyberTech Ltd. - -## Thanks to YourKit - -Carmine was developed with the help of the [YourKit Java Profiler](http://www.yourkit.com/java/profiler/index.jsp). YourKit, LLC kindly supports open source projects by offering an open source license. - -## This project supports the ![ClojureWerkz-logo](https://raw.github.com/clojurewerkz/clojurewerkz.org/master/assets/images/logos/clojurewerkz_long_h_50.png) goals - - * [ClojureWerkz](http://clojurewerkz.org/) is a growing collection of open-source, **batteries-included Clojure libraries** that emphasise modern targets, great documentation, and thorough testing. - -## Community tools, etc. - -Link | Description --------------------------- | ----------------------------------------------------- -[@lantiga/redlock-clj][] | Distributed locks for uncoordinated Redis clusters -[@danielsz/system][] | PubSub component for system -[@oliyh/carmine-streams][] | Higher order stream functionality including consumers and message rebalancing -Your link here? | **PR's welcome!** - -## Contacting me / contributions +## License -Please use the project's [GitHub issues page][] for all questions, ideas, etc. **Pull requests welcome**. See the project's [GitHub contributors page][] for a list of contributors. +Copyright © 2014-2023 [Peter Taoussanis][]. +Licensed under [EPL 1.0](LICENSE.txt) (same as Clojure). -Otherwise, you can reach me at [Taoensso.com][]. Happy hacking! + -\- [Peter Taoussanis][Taoensso.com] +[GitHub releases]: ../../releases +[GitHub issues]: ../../issues +[GitHub wiki]: ../../wiki -## License +[Peter Taoussanis]: https://www.taoensso.com +[sponsor]: https://www.taoensso.com/sponsor -Distributed under the [EPL v1.0][] \(same as Clojure). -Copyright © 2012-2022 [Peter Taoussanis][Taoensso.com]. + - -[Taoensso.com]: https://www.taoensso.com -[Break Version]: https://github.com/ptaoussanis/encore/blob/master/BREAK-VERSIONING.md -[backers]: https://taoensso.com/clojure/backers +[Codox docs]: https://taoensso.github.io/carmine/ +[clj-doc docs]: https://cljdoc.org/d/com.taoensso/carmine/ - -[CHANGELOG]: https://github.com/ptaoussanis/carmine/releases -[API]: http://ptaoussanis.github.io/carmine/ -[GitHub issues page]: https://github.com/ptaoussanis/carmine/issues -[GitHub contributors page]: https://github.com/ptaoussanis/carmine/graphs/contributors -[EPL v1.0]: https://raw.githubusercontent.com/ptaoussanis/carmine/master/LICENSE -[Hero]: https://raw.githubusercontent.com/ptaoussanis/carmine/master/hero.png "Title" +[Clojars SVG]: https://img.shields.io/clojars/v/com.taoensso/carmine.svg +[Clojars URL]: https://clojars.org/com.taoensso/carmine - -[Redis]: http://www.redis.io/ -[Nippy]: https://github.com/ptaoussanis/nippy -[@lantiga/redlock-clj]: https://github.com/lantiga/redlock-clj -[@danielsz/system]: https://github.com/danielsz/system -[@oliyh/carmine-streams]: https://github.com/oliyh/carmine-streams -[pipelining]: http://redis.io/topics/pipelining -[byte string]: http://redis.io/topics/data-types -[rich datatypes]: http://clojure.org/datatypes -[commands.json]: https://github.com/redis/redis-doc/blob/master/commands.json +[Main tests SVG]: https://github.com/taoensso/carmine/actions/workflows/main-tests.yml/badge.svg +[Main tests URL]: https://github.com/taoensso/carmine/actions/workflows/main-tests.yml +[Graal tests SVG]: https://github.com/taoensso/carmine/actions/workflows/graal-tests.yml/badge.svg +[Graal tests URL]: https://github.com/taoensso/carmine/actions/workflows/graal-tests.yml \ No newline at end of file diff --git a/navicat-logo.png b/navicat-logo.png deleted file mode 100644 index e6aa90ba34ff237e6490593091eff107ab548405..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 13372 zcmb8WbyQSe^f-EFhE5TrVF(cwq`L-0x)yN1KKtxA_pE*PUU#35n(7L~1hfPI01zuFzS0H&&^-tk;Njdq zX{Vnl-oLPHWz=KD2cU-q^o3fE%c(idnikbD3MYT3B=WI=kId1Av6D z*gfcM4L4`1;jGNoX$A`;@pUc(5mYY{pRFs>CkDHJ0#XZFfPd^v9 zx$g@XPuBm%@ z_XcwR*Tc=r#l!u7c;63|_zx?l;bCumFZn;0aovn_X$6Ji-Xzv13B%czF=HDLMaOFtLhv+>k&!Pd4dJGJ|p8-=-Z%oce zOlChj{#x8|?0?kKO%UmCWWI&Ru`f6o=^sZ1d#XnJwg2ZtxGfnUffdDr({t2YBpw=P zcxqy|YDRLW#0I5YTw>sSHj!4+>|KBrRB{>mSV|kF9K?%94Z^gKYUzpD6@4KQd2*FW zL*6UqEgw>&ot;JlEOlZpO-=1rY|nF)*i6lWF&^grj>iq11O&>oz`-|O=MM?}Ii-C+ zU6Q|$q}w#fXQuUwclYhnVDk|OXoR>}05NPq&J%jY3z%@-1B6M@t=L4VBJzM)m7fK) zNw#~mfPkcvJ!QB$r9FDH!+UAPrr+=mXYYI7ctDGeVPwqK#j~;|Pt;330(wP?O@Ewt zu%^j`Q1RS#g6@F<=UGsMz~x&o?W)fMsxt2c_IF>tz0@+@G4gKOmz+&iir!C^)9^+; zJ*+5@o}vYjA5Kl_8%-p2>D60@jH8`N#^1FEv2a;lZ3nXR)qTmxF13l8&~vF=S+HDl z ze(S+TvU%q(bs&iK-DLmwkLwGcs@1+VdkOw_zK9ervAJ+uVQ=b^br5`=YFo{UNeidq zr~Q)T$47vme>!|c#FTfe@8m!ETv&=z%wFQHfHYryjpySaslI@Ex#tBr+bwUdwr{X8 z`qq)uWZPRdVN!Y(=NF?m$|W1+{1p4fPfUp`%cH?iQ35y@M7mN1Y&p+udXn9JjZ$7A zeW(?d+c1=L`OFaV+6hRQA$A079~k+o0w&d~&Pq2|;5*HU>UTt+RfKuNjwf115?9UC zgc;MyYEg|dP?CDm( z7_m|2DE7J|@w!2|*x(|5)bR;ep3SEHI@C=o_=VK-UCZD47}Hwb-W(q4SL^qdS3MgR zC1L^vJFijBUbQiN9Si3Ef&Q9PHuZbIZre;idyJ4kfXJ>`McTN23;XZ$q^Hi@6V-;5 zP&#Z?wtC|L^CI`CJwx|o{;)X#bEE#g6>AC+;92>BED{T?dbDSVIndLuW~RP=towVA z`4hIbsY*O2pF`6wPf}UQj75OZ9Q!vZ#ign5EHk0k-0Q8d;PujT3JT)!a+}gGW#Zlb zaiiaJtS*QB-@J*A2ID@#KK}#)SYocMB^RsFQ{X#VzZ*j8dAy*_ZIP0P(r@&Kh1V8DUe$UA0(~*>rooZbq|BCJ1acto^|Nx)dtGI; zn|w;pq2m~zu%>3NXEd@t2VFc6{CR~%nIA(THL0is#rt0n#Y_U?%<4bPSn-roT!k^hzRc;Yy^nkY~ zD`ET)4?sy(-3Bt`Ej$(^4J#7saL~M!h1k9F_-Otw972s16>c~&8BpRdE3hXcqBlap zZHeW~grHoL&XMk+21YfDf+uRGs(UxM3Q?@s(yc8m^mAMfOQjLtT630!9;x2yP2J0D!7)(%Jnp8)CkY4)@ zE0X3oyaR$D0UR3QVjl@7-B}px3uHCu-#pRSQTi_!4ffXp;I{O)bXKX{4GnOq3F9`s z{ZO?}`WP)=c+vrEq8oQz6zZb~3A>V4+L#6b-ol$quan&J#yoSvQBxe1_qb-{-~K#W z^vgXJ|Dt7T_lRyC7iU=Y)4W>D&&D`LY?e!(y%}rAsnTjjN&o51k(`pcy4yo1*PU7Y zmn>^VKAc~UcDZQ@9sleG>E7CN^+Hm_WV7?ETb1S=`}7H`d~V9$V@sFTA9CR$7{@Fz zr{b9%0pY~h?>hvEzUDwwOGgR*EjJE-aR)0V*4O9&e_Cu&i~+Z2QzbR>?W!hrH|Kla z1H3)(dcEQ1H^6J-!kIP4{V5Ydms9%J!Q*$gULVayr3nc99VE(#CF&5=s^cfe6MX22 zePscJ-c{aq$wnQRM>vXXR<2Q=3R?7v^#he4kpXR=m;6J*wt=ce9HS0+V z*TU>4c#DEsElld*LfOj$dqvo!#ic2Rnz!A1cU;KtY+`g&Tb6axVal5rIaF2=yr#$S zufnqC@g~-?%<2ZhY^TU@W9JTE=Qtz0OBJ&2Ua@cV_VXEraU!s>HTqOPTx&+W&H^;o zA>Q)YdOz?3c{q^;6Lv=~cH{T?ceWydA7*Guw|vVBTS}}cC~+7UM^2qz6mYBrQtYJX z%YL;VNlKa-D+`7N95h|*mUoc+>WfsKt@kWA+x`1hjl;ySCv$cPzzTYd$vpctK4R^B z;SL+EwZyK?eC|QH-;iLG1Fvur02Up7zL(y&FqDYQZMeb7n+P5~0m{S^q{cH;we{8e zo^#?}KZbS@w#Ehhl_J2|QvAi1*~0zmxN&riyf&~jKfik+_%5)xWKE)T*gNz7>o{6l zTRRUC;a4KAJUE^|=R2ks^~fcr6B-cwm?UPIM!5O?0}yfDbbCG!`eh_#}MDdNb_if=NzNF4BF`n;^+;2Stk!m%sLi|ks|`h^u{W)v4N31iD1 znhN^H`f`CWF4=f)<<~xDH>B|}nAz|lmBzz}Zwe+80hJJ^suihK#c^{gajA@31sM8M z$+g>;k*h)(HLn%aWZ9Bx(?gGNZwd{YLSgKGv6@ch9P-kg4M-3a{Q z$IL`a*W7Pi$D{-Y7Xmj-exJLRRex3&Q~D*mM^`WOr^cue5&DQj&p`K!lianwvG(qO zI9SS*&1@eL5FO03cgb+lwfD2Uh8KyY@`x_$SQ&fk8U75-(8lnY%+9%DWT`I3s0$RK zqz1Kp&t_=4WP}KKu1Xn1LU48hsSYKrW|U_x$$fi68W{jBt!vb%e|1(?GM#M?o8+J` z@-QfqHmsM=z5u|t1kq2rl@pG`# z>=)l2O9eggQ(*AZu0k5;u(mo**|9FN|AZed8YnG(6x#`|nw+~(eJlt~zx=TaNZq?y zibzijd#JS9)pK5gK^%&Ugrx71_(VqkyvzKhaRbq91Oe$~OvIqhSH`~;`2lOiXK(dr zH@7w}7c8IEu$ze=$ulTWQoSUrbInLRA9q*;Q7(UdX%F*IOdw-bn3g4#u(HkYN+O0z zHL7KVNo+gp{A@7%P4DS^#7TMyd-Uj$&`ro+?`2!As@g^W+5B{oJT+%2P1?yaR~Vh3 zeLqM_Z=d@Z~7 zt4kRNM0QpO4%(@i-zMYN8Pln$;!1m?IIRlJbNMss@Y)4GJ283ZyPkn>=%!b zqYqs@d|EQEOlP&ehH+k8@B9s~;Moq&n3?8TVJUEO+!1iHyh2+(g7(_AGZep^ls6}rGM2%|} zke0*v!mFN9x5Xxe5;bSl)NO3jB^hDII`X`P!&iBA^HCq262a&A$36kb#%Y`%3V{_o zgnVF#*{V7SHAHU)rWKyR=d9jIoVw9#Qp{LvYAo4;O=!9f(tGG zm7lRTChx5EXqEsoDG;`wpj?rTPyVEi_b2PPQ|SxdgQR08E@v?{qht3rrZ#PjEVcR? z#IM(?zxUbY3lr;ar5x zpn)7w68HL|p)`LNumd~dXg^|%=|4#FF6)qmOZbWi^F$xf0WovRtqdUt7@XFgI?yG7 z#>xgdca>EuoGXjKIN&!4K$8oRDjRXdGcLKh*%Is8q@wjkMP#lE~>^<>}=kTBQ|nGvmsF|7Fw)mMB}la*3{4 zJaD!ExX$MN)uWi?jl{*54od7RQztzyQ9lCvA6$g*H>5 zADJ^sBr(D4FU-SP{ygj_k)iLZ>zC|J4pM*1E}M5)deSTb{46To2F) zR!~jgtT*azTu0;VJugoK268=HQ!=~zJ&L!~|#LtIANw1=@cdpH8$DPOW)y{WJ?`HNupP0#qu8Fpf zu28`XwuE1EboJlYegsLk0Rmd1;{J?)KnZi$1oz#zp5)`hyYXu%=HhEqGP6q$OGtbw z*5e@BOVH1lHqye^dL;jdkuHFj?_w*SwmVCx+xklpBG$w$%ew}T*{g-M>!Uc=KT?On zl`4&9vx^w0jbNo9g}rm|Eo0tCGne|o0V(*GoYm!g)zpLbZo#$jBp_JaW8A!wEo?c{@w>x zE7&JG0pIw+M%(;Zb|Kdpr=7{tU*sARyux{FpA0YgT^A&N-^BUMej_KX{?e~MQ^0*y z)q{gKqrEgCvz26c?7}Meju6}uyWmks3c<^cz5p;O!owS> zzDEQ>rez0X?~WZCIENkxj6drn-FZaOv-;$yQMJh@Cft;&S$ZrjxTZvk@GOT*s^CGdG(HD)0Qle{ogsvenaY96TOQ;J8&1RFV8%7!rF~ zr0X=I;S9n&%q@EDctFwD(MBDjp2VB$Cxblq2Agjn+Da#(7}dujbH22`|**;SLfw) zz~iHVf0&zt+A_o%M`;6}4)4v#Ura1>r=!Z-UmfiBMb$J^JCQ}r=+HIq&QA)LJXYIh zI-b%6w&fU*V0}>ndH&Tl5fQll9r-fqyGe+%-q5JsW!Cy!2)- zK(b$2nve8%CJK_zo;qLJ2v!Tqo2XxQ;(KpdC(ndc;(Pbj$s&G`+iOTpmYKQ|QgK#Y z(FT2!JCxjSJW-$WrDloYrgPJ%$kr^Z}$mfeK^{FR5IYlKd&#TuoSFihzxoFKXD2P&K(6|@HPR% zWc9GCo|pk9Y;+iYZpeM~E2Y$c=Q8hk)dhc+|FHjqS1++J{SsROI(a%aZ^K^1t!x*3 z*HhE{HYZEKC|4*>WrlupnWY;>78EWv^U;6;7}GW7@}@+7j6|R6Xap2ke0}ifdd9KL zxn>g~gq$v<(L*1hg5l2?WwN+)Dl7T)lU?iAD3hx=U-_Gl5SC&peQxia6?{De^ywA{ zC3W-D#CQ2UmpiM$*wyGVzU_dN*)Sk1){fAOm|gh+E4p+Ix~0Qqzz8}{);)P=lDj2O z3w+3LKi!JrZnP5)-mrl_?5xzhx!aozK6yh)ZR{C)bm!i~uyG!HpY2Z3aF3#^r`~$f z;orQdbMs`(TK!7yYz^%h*C`N6-U}jh4a8e&<*bs5FlL==%)@YU)WmD z=Q=S5Yddya6LXRMc{rREbiMVKUF^mvIIutIh0$bg)l!VtE3xL&Q57j0h57Ef-paMf z*DEQQzqXlYys$OrnYuqiQa4VvVaP0r3f}QTUf^f&@26Qy!dM?62N57c|ZcL2~Z3&s60(am!l4m-O+(txiC0J3V?1B8zvQ zlS#fnl9&&vrjy0{C}ydAj#5Dzcz^gj#`qzJc1-}+9`ari=KOo1#keDh)M;s?MN`WV z>;2JK86E;nz2VTgROR|j;1lf)@($0<`s>tD=1@m7Pz8*;M;it7T}$A1ln8Z%p@~Vy6`>)+^E3KYJve)oM`grO>}B+xk%z$@FZC!RlEUX&t00Nq zkDpV9xTjJ>k|2s;uL$*a?G!0Lp6V~qWD&rw_Vr&hDa_sN$i&mS$NMN~bNHK{&|9!N zE$L#bE&XdQuB5HZ@hk)N=7zlQUOSdMM6Jc-Dbfcc{JFF(PI67LUq0>X7G7xBN4nPR z8%I&p7;+riyoobkOn9}M_pGHL>{I#eWXSXKH6l-hRR|yMBvv-rG~+&?9%_gc+p5tz zsI|bD1PMMRPbPi1Pplg5A^Z0e&3m!otvU)*oc8w76Q>&^L#D?NyOxf99dYPba@SKl zNVezJyNdfbr|}u*N5>}dA&}wD<#SG9JuBMnT^ifp-y#S3NMV=dfpy2S2MFgs5shwY z;*O^U2a1wT*l_P~dPQStdHO6H!Z$QHBI*@wA8@2o^g5oaH^MUx`7__1qFLe;5To@V z?n0de>tDk8JJi2eM^ag=!jY+DeIT>J`sq>wD0-yJcA?96OZ>>}^)T{T&*q1Y%Eh?v zj$JtMF4|^b$#B(*1Q2X${cCmeRep^{Q)`RP($q6?G47U{l~;EY=|y|ATxRxDug$Pp zASlqsM(h;)B@byi?Dz4Fs|%gb#je9meJM0yG%`Su?u7JU)I5=HEX)>-CRJC(LQ<)b zkzZ2L!O0eeJ9Dl zJsHEZiQk*r-*c$&`%O-{^NmmzTL?4D8I8;9UHjp8qgR0xnX_$xU2JoWl%}~mEass8 ziJ(i)`ZR-{1>PgNeX6I|&zgmf>)<$Mh&di1M?lmpZW$?XMA#Xk)XB5|?-PjzTIt$& zCPkr?bLBl>Z-;p4O`+6sqlV;7jty?4eALK%o}6q}P0)-&`C0WH1%97zv3Ra5WST3y zr_n1r=2b-I?@2lQh_>ByPgKQ5R^Y8s zh@v>1EKERNSes{F-%va+{`9H#b(rHW|GgP&rfkj{6)xit)Mq|e0-ktbpNc#su%i00 zP;Fed_*0X!>Bm2$7$B^8`FJ3t?>El^iWeKLRjo049NYM=fiVS)sz-bH<@JLn+75i~ zUMn><4K;RXSYEA?`u+pp^`zK+Fm!`J>cbu0V;fzs!bAvzB#vcq<};%n=jGRCqu!89 z+4iIfLqXvLaE%z3T%Dr~{-0zD;Jx?q84*+nOP zK~h?&c?P224wj-~+h&fSw_wjeB!trSi}PbsU%FMOcJC6A$`e7CksaR1>)5EPx`I@g zqTenwovf8VI=o?dCZRdiDdWxj>0la~CrNn0w_~<%7l`l^8PiF2W6iFT)sz4J-Y;#F z1FAKpPRvxJO*}Mf1NQPCEUm4EM9w32D`>7`%77KOyB4R`6*k^Wq1!F5^4cKkf{Ar78C4Pk3 zA-0Si*p6l(!7Pnh%A(GZ#sK}7F2v@v*z~q=2(HSc++H+WyxD z2}ZZc(qRCbR#ekWq%vyie0lNB!N|r;_A^(zCsxTdyrjA%Nuv-S)^ac}8+< zFG9Ru)o*B+|kIs{+G9haWx^!{oqX4*i{E6I#IBCJb`P2 zWRCVx3!z=*7ch5<9f`MYw*&EDZ%3?NIwq}YPsw&eD=+4P9dpM6j*q>!Xu8`c-cq%p zzUR-nGz~sc%lwzX_@uWzpF_&?eHkxsctvjrTI=>+i}K8ls1C*(R*n)0DsKL2|5EBf zQBcPNCZyz#BhapYGUwVLFF{-!Tw^={*Q1y9$6X^KCf>B2I-NPp)|Q7v`54$G+x+7> z)1n`+8Q4INZks`^XJ0o|^IBv4#OEIYxUG{eYkB`kNdJ=$P!>7C@obU zW2L_ifxUzE;3e=PW}qQykf3&^PHKt3cD#2lLe!te2Q18;@g`hs@vMZoZDZ*ysRzQN zP;Ii(eIUfASIHVDS2B;EDyRO;&AXGw{J4LsLljJ4fM%sAnbTCaY+II4bB+7Bfr=MICbYr3-E`7fm;1J0_7 zOB?%A*#~0LEW~v>Z-*A;yEPEU+aHofbNe#(_at!QALP_{G&ScT$Fb0&p(bGX1!M>h zu@6vvJV5aP=;eLsbDyI?z%TwhZ0q>#E813)na{qr=<-HIhueU@dn z1IOa%&(@+Q?E@4bMzv=>e~V3JvNWcbPIWlmUeG>mG@7oyFwTpy4U}Z~ItcsBu5tdf zqC+?adRx*|ASEI+SFRUQC@T=Tkr_O+gFo%2=p4@Iz7LDZOQMcupM+N7OddlxyVr;~ z42z9a`!ni9q)|>-(UD?mDLidE{o=a>6_S@@v8TLoa9>Z9v%QX|{cpa;6KZ6z@1ObS z8T*hF?N)EijocuDf4u5o^mY2UMiuhOK&uRiPA@M3l5`prOr!yV;w&wSGE63>6Klg1 zgJ?b*ADx9w*K*QQH%#_ zndMgto;I!2CD&ew30?t2k0l(z23!N~>$?w(E3H0iln~R1bqZtB-PdXA>+hGVS_Ef| z<3q#%ZTS(7IArHxrr&z%Tz8tm6k?0T&C-u6Rj`BX*6G6ekKC{!fu;6Nl2*u#|hw2gdgKpdBQ`H0|vw(!EY zC(Bb~fzgq|OI;e0L9=CoqTBuBc`3cYWbOmYhJ_~h^bgLH5xMx>14Y@^onC z#zyLv`kO@Z{5oY~a+X`@{Elr%@&ZgC-|dAD2Q z=Dbv2)l{&03X4v##!$ehN`EpgKT~?!;q5`-Vu98-_c80%@wh|x)owp}YQl3$O3LNq zHM-e!9;^nD#z<-(NY+-<>chC16! zuY8n+CgWD}3#0Xc5R(t-_`OA~h}IbA733%1SP;>6>0MoazvWj7!$arGGo~aHQ}~FS zI$hjwOt9D&kjO55Cz2|jU0VV?1NwO=A9VH5ejo4Lb}exZa?*NGA0-g-QQ(0T{>D0e zEA%*CGkYpx9D18P9=F{BgCMA7%z#odyHwYZ{1ge*LIRZ@hb_-a{S2puRyig&Tr&Wx z)vXi}K%a3p@+MBKychBS=uUwxJx(ua(ire-7w2!aaX8Fsxj?yKSuT_Q#%P*=Bo1i$ zd&q-Rf>=@O#+*@?PH$qZN4M_t|MScn4co-wWC?>H6_c?u4FzlwCk}c3W%l9t1|(vC z)${i7wcfGT6Ity1OA(*=E-CJ&Lv5D;uGwPlMvok14suf&58R7q|^tK}^50dvRVzQaqcVV$pY z#XvPz7QCiMZ3MO?SKE~RX3bf0faij6pgD$wrSfEhFC{I-J#pNXQnlcT5ouR zsXM%;Ol*_;4n`LhXwsu_=La}h~^I5QfZESP!i_-<>M9a_~98%WY1w# zVb8t8(uKIiKeAV#>T$Zd!XU+!ov3BNR%9%~oJfa3rNCg^r8+alId`R$fHCuB=MGbl z?kn$Bu|vOb8lFkg3#K!2aimJ1;zL1D3Nkb5+7R{4Hr(mY_b6j5$Y_|3EkLsrxASjS zXV-5dW5aSTQxb0Y>3Fs7d9FyThsB%pc+c7RS*P`Z)bqX-{YHson<|}L27RHS`%)WP z*sIX`3!lZnsknic58Lf5GBe%zgNHJMYEU?4B?XN1_LB)wUb4z>uTxd#`%sLKAO}LK zMb82az$}*Ek^%AyRMvF;6(q8GU(V@XzVGQV!d?(GnvbxE%p!_rChG~vY&qL=rIF1L z^;{R}2qSqL9{!U0MTM=j{^+V1AHS4f$CkL=5)*fa>88d{x-(20X^`13o5ADH9}K)c z{8I?E;2rDLV!bu$5$Zaf_u|%EWqo&EqFOnE`?va-v3Twf7av9T-AUxG?Af&E^A@TS zxY|S@Zjc%M=}XKO2iw_dEY$Tlz0n?_L+X7P5l4;8Lc+|!AT&=X-lZEU(MuFsQ)$m; zhxebyj^z1qZX?$}pb#gUPeEbBP6PLdBn!!U>vyt7zRL262VA;XzQ87!4KAk->0+Qd ziS%hdUQ_bbNp^|jPst}czI;~j!n#Z3lf09g#pb&{hU6ElZEv`d<8eo$atR7VJr8=F zfZ1qwJBnP(rycP`RQ=KS45~RYxM>-_>g8gmrHWJ~`tNa1eI$}*(9^EASY?@QLfhO{ zPGKz@7C9<6Y;>P|6g$ZFHY?ZzBgMhu#F)p?M$*CCnNOfV%o47M%vpg8Xp=YrMuN=( z!RtV_K0v3qPBdj(Ws_jl^NJpZw*QU*B1o0# zLbo)CdkRm_M>+qgwg0+*?6dHoHLqk6bv9C?e%t7iPIAhOL{fYh=E3Ua4NrIAc4Hd= zkUqxSkjtmMpa1~DbIrU=-lB4A)!gz4=Rt=EtH$bzmOk#GPHk??{F!@0aXOxpm=7Y~ zshI`?H}IJ#WL@j@mv8L~7~rU3>yBDJV(d_&njQLmIdF6SF6(p)VHvb_c$rzg@14;D zJp9vZle!|M1J7J|X+H{)llTj|kL#A$F~ISwi3^^`PPFJteC2(vt5r|mN0=I!?TzlU zfZ`iP%n$5KlOos#<2ikhS2EPHlnGJVnmJw|-Px-2~= zL_d25EjW1xS7`4bs&m{ItC!$32QzjjLYEGLzn;;Aj*$M-@$Lp4KY$TlQ)}L{2e90| z>N~Gvf9qw=AM;}^VTU(|k^O5$5$;05PSTC*Hz;$PjgCW4mT@;0$5H_$TrFAO=kNIetc2 zaE#uF8dbBGnd}hb10yO?e+c%RrU_>A?cc2ZEDvOH;?z z`J4enzX$w@vyu`O<@gQ{?t{D~OSzCzbKu#-Jwv~&y|+iA_~TKJ2!=1cC}dj5hS&aF zDeW>cYI7l_kXaQ{LBNnFmLx$;{*gg@@uML7K_=nagASnSeukcIdq>K|RJ>*7u1yF! zO0?`lwQ{pq?OUpca9IJwOfyaXHG6N*Vb~3_VWNXXm)s}c7gi}yM?=uy!{P2=K!kaY z?aGtTFaDbB-dn6wo@Jk(#EQ4-(`FcUat^I$HT8Z3=-;hfesnGh};0H2K9ZQ{O)eZH7+f;1e{ zz5SdDFN=N0g66TjG!0?>M%8_t`;UFdrJI~%@S(zCfo>2{5D(DaD%3&35S*9A&b=ir zr+xRhM3=gN$Jk1OCvD)qM7Zwhup|-i)nW^`)uTFK(&*~C;HSr`tez1j>)q1RCfZ`h z?3KrpJ{hf{vdbIX_H?;HuP<18kJyWv3KB|W9EstI^}el|w@J&(dZTxB{{$A98ClWx z!tw2#3waTBcA_e0SRbJmKgH(9j?>wwRoQx9Z7hm?&JVGB8EA_$ z$Q$;=oz4#=4!ZsfY9_Ph(~u8Idg^Azi)zrN9uKC0h~UBeC;vr*KY{O{Tc&@*oB0RtkNE(AlAQXh(wAoM{x6jk2o3-M diff --git a/wiki/1-Further-usage.md b/wiki/1-Further-usage.md new file mode 100644 index 00000000..46732c6d --- /dev/null +++ b/wiki/1-Further-usage.md @@ -0,0 +1,154 @@ +# Lua scripting + +Redis offers powerful [Lua scripting](https://redis.io/docs/interact/programmability/eval-intro) capabilities. + +As an example, let's write our own version of the `set` command: + +```clojure +(defn my-set + [key value] + (car/lua "return redis.call('set', _:my-key, 'lua '.. _:my-val)" + {:my-key key} ; Named key variables and their values + {:my-val value} ; Named non-key variables and their values + )) + +(wcar* + (my-set "foo" "bar") + (car/get "foo")) +=> ["OK" "lua bar"] +``` + +Script primitives are also provided: [`eval`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-eval), [`eval-sha`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-evalsha), [`eval*`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-eval*), [`eval-sha*`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-evalsha*). + +# Helpers + +The [`lua`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-lua) command above is a good example of a Carmine "helper". + +Carmine will never surprise you by interfering with the standard Redis command API. But there are times when it might want to offer you a helping hand (if you want it). Compare: + +```clojure +(wcar* (car/zunionstore "dest-key" 3 "zset1" "zset2" "zset3" "WEIGHTS" 2 3 5)) +(wcar* (car/zunionstore* "dest-key" ["zset1" "zset2" "zset3"] "WEIGHTS" 2 3 5)) +``` + +Both of these calls are equivalent but the latter counted the keys for us. [`zunionstore*`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-zunionstore*) is another helper: a slightly more convenient version of a standard command, suffixed with a `*` to indicate that it's non-standard. + +Helpers currently include: [`atomic`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-atomic), [`eval*`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-eval*), [`evalsha*`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-evalsha*), [`info*`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-info*), [`lua`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-lua), [`sort*`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-sort*), [`zinterstore*`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-zinterstore*), and [`zunionstore*`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-zunionstore*). + +# Pub/Sub and Listeners + +Carmine has a flexible **listener API** to support persistent-connection features like [monitoring](https://redis.io/commands/monitor/) and Redis's fantastic [Pub/Sub](https://redis.io/docs/interact/pubsub/) facility: + +```clojure +(def listener + (car/with-new-pubsub-listener (:spec server1-conn) + {"foobar" (fn f1 [msg] (println "Channel match: " msg)) + "foo*" (fn f2 [msg] (println "Pattern match: " msg))} + (car/subscribe "foobar" "foobaz") + (car/psubscribe "foo*"))) +``` + +Note the map of message handlers. `f1` will trigger when a message is published to channel `foobar`. `f2` will trigger when a message is published to `foobar`, `foobaz`, `foo Abraham Lincoln`, etc. + +Publish messages: + +```clojure +(wcar* (car/publish "foobar" "Hello to foobar!")) +``` + +Which will trigger: + +```clojure +(f1 '("message" "foobar" "Hello to foobar!")) +;; AND ALSO +(f2 '("pmessage" "foo*" "foobar" "Hello to foobar!")) +``` + +You can adjust subscriptions and/or handlers: + +```clojure +(with-open-listener listener + (car/unsubscribe) ; Unsubscribe from every channel (leave patterns alone) + (car/psubscribe "an-extra-channel")) + +(swap! (:state listener) assoc "*extra*" (fn [x] (println "EXTRA: " x))) +``` + +**Remember to close the listener** when you're done with it: + +```clojure +(car/close-listener listener) +``` + +Note that subscriptions are **connection-local**: you can have three different listeners each listening for different messages, using different handlers. + +# Reply parsing + +Want a little more control over how server replies are parsed? See [`parse`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-parse): + +```clojure +(wcar* + (car/ping) + (car/parse clojure.string/lower-case (car/ping) (car/ping)) + (car/ping)) +=> ["PONG" "pong" "pong" "PONG"] +``` + +# Distributed locks + +See the [`locks`](https://taoensso.github.io/carmine/taoensso.carmine.locks.html) namespace for a simple distributed lock API: + +```clojure +(:require [taoensso.carmine.locks :as locks]) ; Add to `ns` macro + +(locks/with-lock + {:pool {} :spec {}} ; Connection details + "my-lock" ; Lock name/identifier + 1000 ; Time to hold lock + 500 ; Time to wait (block) for lock acquisition + (println "This was printed under lock!")) +``` + +# Tundra + +> **Deprecation notice**: Tundra isn't currently being actively maintained, though it will continue to be supported until Carmine 4. If you are using Tundra, [please let me know](https://www.taoensso.com/contact-me)! + +Redis is a beautifully designed datastore that makes some explicit engineering tradeoffs. Probably the most important: your data _must_ fit in memory. Tundra helps relax this limitation: only your **hot** data need fit in memory. How does it work? + + 1. Use Tundra's [`dirty`](https://taoensso.github.io/carmine/taoensso.carmine.tundra.html#var-dirty) command **any time you modify/create evictable keys** + 2. Use [`worker`](https://taoensso.github.io/carmine/taoensso.carmine.tundra.html#var-worker) to create a threaded worker that'll **automatically replicate dirty keys to your secondary datastore** + 3. When a dirty key hasn't been used in a specified TTL, it will be **automatically evicted from Redis** (eviction is optional if you just want to use Tundra as a backup/just-in-case mechanism) + 4. Use [`ensure-ks`](https://taoensso.github.io/carmine/taoensso.carmine.tundra.html#var-ensure-ks) **any time you want to use evictable keys** - this'll extend their TTL or fetch them from your datastore as necessary + +That's it: two Redis commands, and a worker! + +Tundra uses Redis' own dump/restore mechanism for replication, and Carmine's own [[Message queue|message queue]] to coordinate the replication worker. + +It's possible to easily extend support to **any K/V-capable datastore**. +Implementations are provided out-the-box for: + +- [Disk](https://taoensso.github.io/carmine/taoensso.carmine.tundra.disk.html) +- [Amazon S3](https://taoensso.github.io/carmine/taoensso.carmine.tundra.s3.html) +- [Amazon DynamoDB](https://taoensso.github.io/carmine/taoensso.carmine.tundra.faraday.html) via [Faraday](https://github.com/taoensso/faraday) + +## Example usage + +```clojure +(:require [taoensso.carmine.tundra :as tundra :refer (ensure-ks dirty)] + [taoensso.carmine.tundra.s3]) ; Add to ns + +(def my-tundra-store + (tundra/tundra-store + ;; A datastore that implements the necessary (easily-extendable) protocol: + (taoensso.carmine.tundra.s3/s3-datastore {:access-key "" :secret-key ""} + "my-bucket/my-folder"))) + +;; Now we have access to the Tundra API: +(comment + (worker my-tundra-store {} {}) ; Create a replication worker + (dirty my-tundra-store "foo:bar1" "foo:bar2" ...) ; Queue for replication + (ensure-ks my-tundra-store "foo:bar1" "foo:bar2" ...) ; Fetch from replica when necessary +) +``` + +Note that the [Tundra API](https://taoensso.github.io/carmine/taoensso.carmine.tundra.html) makes it convenient to use several different datastores simultaneously (perhaps for different purposes with different latency requirements). \ No newline at end of file diff --git a/wiki/2-Message-queue.md b/wiki/2-Message-queue.md new file mode 100644 index 00000000..50a2ec26 --- /dev/null +++ b/wiki/2-Message-queue.md @@ -0,0 +1,31 @@ +Carmine includes a simple **distributed message queue** originally inspired by a [post](http://oldblog.antirez.com/post/250) by Redis's original author Salvatore Sanfilippo. + +> **Note**: [Carmine 3.3](https://github.com/taoensso/carmine/releases/tag/v3.3.0-RC1) is introducing major improvements to Carmine's message queue. I plan to update and expand this documentation before the final 3.3 release. - [Peter](https://www.taoensso.com) + +# Example usage + +```clojure +(:require [taoensso.carmine.message-queue :as car-mq]) ; Add to `ns` macro + +(def my-worker + (car-mq/worker {:pool {} :spec {}} "my-queue" + {:handler (fn [{:keys [message attempt]}] + (println "Received" message) + {:status :success})})) + +(wcar* (car-mq/enqueue "my-queue" "my message!")) +%> Received my message! + +(car-mq/stop my-worker) +``` + +The following guarantees are provided: + +- Messages are persistent (durable) as per Redis config. +- Each message will be handled once and only once. +- Handling is fault-tolerant: a message cannot be lost due to handler crash. +- Message de-duplication can be requested on an ad hoc (per message) basis. + + In these cases, the same message cannot ever be entered into the queue more than once simultaneously or within a (per message) specifiable post-handling backoff period. + +See the [message queue API](https://taoensso.github.io/carmine/taoensso.carmine.message-queue.html) for more info. \ No newline at end of file diff --git a/wiki/3-Community-resources.md b/wiki/3-Community-resources.md new file mode 100644 index 00000000..4cdc7333 --- /dev/null +++ b/wiki/3-Community-resources.md @@ -0,0 +1,8 @@ +If you spot issues with any linked resources, please **contact the relevant authors** to let them know! + +Contributor | Link | Description +:-- | :-- | :-- +[@lantiga](https://github.com/lantiga) | [redlock-clj](https://github.com/lantiga/redlock-clj) | Distributed locks for uncoordinated Redis clusters +[@oliyh](https://github.com/oliyh) | [carmine-streams](https://github.com/oliyh/carmine-streams) | Utility functions for working with [Redis streams](https://redis.io/topics/streams-intro) +[@danielsz](https://github.com/danielsz) | [system](https://github.com/danielsz/system/tree/master/src/system/components) | Example `system` components for [Redis Pub/Sub](https://redis.io/docs/interact/pubsub/), etc. +_ | _ | Your link here? [PRs](../wiki#contributions-welcome) welcome! \ No newline at end of file diff --git a/wiki/Home.md b/wiki/Home.md new file mode 100644 index 00000000..d37f7feb --- /dev/null +++ b/wiki/Home.md @@ -0,0 +1,170 @@ +# Getting started + +## Dependency + +Add the [relevant dependency](../#latest-releases) to your project: + +```clojure +Leiningen: [com.taoensso/carmine "x-y-z"] ; or +deps.edn: com.taoensso/carmine {:mvn/version "x-y-z"} +``` + +And setup your namespace imports: + +```clojure +(ns my-app (:require [taoensso.carmine :as car :refer [wcar]])) +``` + +## Configure connection + +You'll usually want to define a single **connection pool**, and one **connection spec** for each of your Redis servers, example: + +```clojure +(defonce my-conn-pool (car/connection-pool {})) ; Create a new stateful pool +(def my-conn-spec {:uri "redis://redistogo:pass@panga.redistogo.com:9475/"}) +(def my-wcar-opts {:pool my-conn-pool, :spec my-conn-spec}) +``` + +This `my-wcar-opts` can then be provided to Carmine's `wcar` ("with Carmine") API: + +```clojure +(wcar my-wcar-opts (car/ping)) ; => "PONG" +``` + +`wcar` is the main entry-point to Carmine's API. See its [docstring](https://taoensso.github.io/carmine/taoensso.carmine.html#var-wcar) for **lots more info** on connection options! + +You can create a `wcar` partial for convenience: + +```clojure +(defmacro wcar* [& body] `(car/wcar ~my-wcar-opts ~@body)) +``` + +## Command pipelines + +Calling multiple Redis commands in a single `wcar` body uses efficient [Redis pipelining](https://redis.io/docs/manual/pipelining/) under the hood, and returns a pipeline reply (vector) for easy destructuring: + +```clojure +(wcar* + (car/ping) + (car/set "foo" "bar") + (car/get "foo")) ; => ["PONG" "OK" "bar"] (3 commands -> 3 replies) +``` + +If the number of commands you'll be calling might vary, it's possible to request that Carmine _always_ return a destructurable pipeline-style reply: + +```clojure +(wcar* :as-pipeline (car/ping)) ; => ["PONG"] ; Note the pipeline-style reply +``` + +If the server replies with an error, an exception is thrown: + +```clojure +(wcar* (car/spop "foo")) +=> Exception ERR Operation against a key holding the wrong kind of value +``` + +But what if we're pipelining? + +```clojure +(wcar* + (car/set "foo" "bar") + (car/spop "foo") + (car/get "foo")) +=> ["OK" # "bar"] +``` + +## Serialization + +The only scalar type native to Redis is the [byte string](https://redis.io/docs/data-types/). But Carmine uses [Nippy](https://github.com/taoensso/nippy) under the hood to seamlessly support all of Clojure's rich data types: + +```clojure +(wcar* + (car/set "clj-key" + {:bigint (bigint 31415926535897932384626433832795) + :vec (vec (range 5)) + :set #{true false :a :b :c :d} + :bytes (byte-array 5) + ;; ... + }) + + (car/get "clj-key")) + +=> ["OK" {:bigint 31415926535897932384626433832795N + :vec [0 1 2 3 4] + :set #{true false :a :c :b :d} + :bytes #}] +``` + +Types are handled as follows: + +Clojure type| Redis type +-- | -- +Strings| Redis strings +Keywords | Redis strings +Simple numbers | Redis strings +Everything else | Auto de/serialized with [Nippy](https://github.com/taoensso/nippy) + +You can force automatic de/serialization for an argument of any type by wrapping it with [`car/freeze`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-freeze). + +## Command coverage + +Carmine uses the [official Redis command spec](https://github.com/redis/redis-doc/blob/master/commands.json) to auto-generate a Clojure function for **every Redis command**. These are accessible from the main Carmine namespace: + +```clojure +;; Example: car/sort is a Clojure function for the Redis SORT command +(clojure.repl/doc car/sort) + +=> "SORT key [BY pattern] [LIMIT offset count] [GET pattern [GET pattern ...]] [ASC|DESC] [ALPHA] [STORE destination] + +Sort the elements in a list, set or sorted set. + +Available since: 1.0.0. + +Time complexity: O(N+M*log(M)) where N is the number of elements in the list or set to sort, and M the number of returned elements. When the elements are not sorted, complexity is currently O(N) as there is a copy step that will be avoided in next releases." +``` + +Each Carmine release will always use the latest Redis command spec. + +But if a new Redis command hasn't yet made it to Carmine, or if you want to use a Redis command not in the official spec (a [Redis module](https://redis.io/resources/modules/) command for example) - you can always use Carmine's [`redis-call`](https://taoensso.github.io/carmine/taoensso.carmine.html#var-redis-call) function to issue **arbitrary Redis commands**. + +So you're **not constrained** by the commands provided by Carmine. + +## Commands are functions + +Carmine's auto-generated command functions are **real functions**, which means that you can use them like any other Clojure function: + +```clojure +(wcar* (doall (repeatedly 5 car/ping))) +=> ["PONG" "PONG" "PONG" "PONG" "PONG"] + +(let [first-names ["Salvatore" "Rich"] + surnames ["Sanfilippo" "Hickey"]] + (wcar* (mapv #(car/set %1 %2) first-names surnames) + (mapv car/get first-names))) +=> ["OK" "OK" "Sanfilippo" "Hickey"] + +(wcar* (mapv #(car/set (str "key-" %) (rand-int 10)) (range 3)) + (mapv #(car/get (str "key-" %)) (range 3))) +=> ["OK" "OK" "OK" "OK" "0" "6" "6" "2"] +``` + +And since real functions can compose, so can Carmine's. + +By nesting `wcar` calls, you can fully control how composition and pipelining interact: + +```clojure +(let [hash-key "awesome-people"] + (wcar* + (car/hmset hash-key "Rich" "Hickey" "Salvatore" "Sanfilippo") + (mapv (partial car/hget hash-key) + ;; Execute with own connection & pipeline then return result + ;; for composition: + (wcar* (car/hkeys hash-key))))) +=> ["OK" "Sanfilippo" "Hickey"] +``` + +# Performance + +Redis is probably most famous for being *fast*. Carmine hold up its end and usually performs within ~10% of the official C `redis-benchmark` utility despite offering features like command composition, reply parsing, etc. + +Benchmarks are [included](https://github.com/taoensso/carmine/blob/master/src/taoensso/carmine/benchmarks.clj) in the Carmine GitHub repo and can be easily run from your own environment. \ No newline at end of file