Missing Commands in StackExchange.Redis #2055

slorello89 · 2022-03-25T14:18:16Z

Hi @NickCraver @mgravell , as we discussed a few weeks ago (with @chayim and @gkorland) I went through the library and compiled a list of what I believe are the missing commands of StackExchange.Redis.

Methodology

I basically diffed RedisCommand.cs with the main Redis Project's commands.json, which resulted in ~180 commands, I went through and purged most of the commands that felt like they were admin/config (except ACL) to preserve the more interactive commands, this left ~70. Then I went through the library to see if the sub-commands that came up (a lot of stuff from XGROUP XINFO and a few others) were already covered by a combination of the command Enum + a literal to filter those out as they are already supported. After that, and collapsing the many sub-commands of FUNCTION and ACL, we're left with 47. I think what follows is mostly correct (in that the thing is ACTUALLY missing, and that it's something that you would probably want to add to the library if it were feasible). I marked the blocking commands, and then applied a bit of my own reason to determine if I thought the command would be addable without a major overhaul of the library. I also added the Redis Version the command was added in (about 40% of the missing commands are 7.0, I don't know what your policy is on adding Release candidate commands, I'm guessing you wouldn't want to :) )

As it stands, there are about 21 commands which are in stable versions of Redis, interactive, and shouldn't require any major architectural changes, call them low-hanging fruit. Seems like there are a few commands that were either just missed, or there's some story behind why they never made it in (BITFIELD, some OBJECT subcommand, SCRIPT KILL, LOLWUT (ok maybe a bit silly, but can always use a bit of levity)). And then I'm guessing there hasn't been a command refresh since 6.0

This is meant as a starting point, for my part, I'm happy to start opening PRs for any commands you'd approve of adding. I understand the reviews can be burdensome and do not want to overwhelm you, so just let me know what you want to be added and I'm happy to get started.

Full List

All the missing commands, including 7.0 commands and things that are going to be a bigger headache to add.

Command Name	PRs		Interface	Proposed names	Notes
ACL (+sub-commands)		6	(TBD)	Skip: Broader discussion
BITFIELD	#2107	3.2	IDatabase	StringBitFieldSet/StringBitFieldGet/StringBitFieldIncrement/StringBitField	This API might be tricky, the command is variadic, so an array of sub-commands may make sense, or the three sub-commands split out, or both could work. Also, might want a clever return-type.
BITFIELD_RO	#2107	6.2	IDatabase	see notes	Consider intelligently interrogating the commands executed to see if they are RO.
BLMOVE		6.2	IDatabase	Skip: Blocking
BLMPOP		7	IDatabase	Skip: Blocking
BZMPOP		7	IDatabase	Skip: Blocking
BZPOPMAX		5	IDatabase	Skip: Blocking
BZPOPMIN		5	IDatabase	Skip: Blocking
~~COPY~~	#2064	6.2	IDatabase	KeyCopy
EVALSHA_RO		7	IDatabase	ScriptEvaluateReadOnly
EVAL_RO		7	IDatabase	ScriptEvaluateReadOnly
~~EXPIRETIME~~	#2083	7	IDatabase	KeyExpireTime
FCALL		7	IDatabase	Skip: Pending functions design
FCALL_RO		7	IDatabase	Skip: Pending functions design
FUNCTION (+sub-commands)		7	IDatabase	Skip: Pending functions design
~~GEOSEARCH~~	#2089	6.2	IDatabase	GeoSearch
~~GEOSEARCHSTORE~~	#2089	6.2	IDatabase	GeoSearchAndStore
~~GETEX~~	#1743		IDatabase	StringGetSetExpiry
~~HRANDFIELD~~	#2090	6.2	IDatabase	HashRandomField
~~LCS~~	#2104	7	IDatabase	LongestCommonSubsequence
~~LMOVE~~	#2065	6.2	IDatabase	ListMove
~~LMPOP~~	#2094	7	IDatabase	Overload ListLeftPop and ListRightPop
~~LPOS~~	#2080	6	IDatabase	ListPosition/ListPositions
~~OBJECT ENCODING~~	#2088	2.2	IDatabase	KeyEncoding
~~OBJECT FREQ~~	#2105	4	IDatabase	KeyFrequency	This can only be used if maxmemory-policy is set to an LFU policy, might be worth some command-map manipulation to interrogate this policy.
~~OBJECT REFCOUNT~~	#2087	2.2	IDatabase	KeyRefCount
~~PEXPIRETIME~~	#2083	7	IDatabase	KeyExpireTime	Overload that specifies to get the expire-time in milliseconds
PUBSUB SHARDCHANNELS		7	IServer	SubscriptionShardChannels
PUBSUB SHARDNUMSUB		7	IServer	SubscriptionShardSubscriberCount
SCRIPT KILL		2.6	IServer	ScriptKill
~~SINTERCARD~~	#2078	7	IDatabase	SetIntersectionLength	This could also be SetCombineLength, I worry though because there's no corresponding union/diff commands yet that this can lead to some confusion.
~~SMISMEMBER~~	#2077	6.2	IDatabase	Overload SetContains
~~SORT_RO~~	#2111	7	IDatabase	SortReadOnly	Another option is to just change the Sort method to use the SORT_RO, but I fear that's a break (and there would need to be some other logic to determine if SORT_RO is available on the Redis Server
SPUBLISH		7	(TBD)	Skip: Pending sharded pub-sub design
SSUBSCRIBE		7	(TBD)	Skip: Pending sharded pub-sub design
SUNSUBSCRIBE		7	(TBD)	Skip: Pending sharded pub-sub design
WAIT		3	(TBD)	Skip: Blocking
~~XAUTOCLAIM~~	#2095	6.2	IDatabase	StreamAutoClaim
~~ZDIFF~~	#2075	6.2	IDatabase	SortedSetCombine
~~ZDIFFSTORE~~	#2075	6.2	IDatabase	SortedSetCombine
~~ZINTER~~	#2075	6.2	IDatabase	SortedSetCombine
~~ZINTERCARD~~	#2075	7	IDatabase	SortedSetIntersectionLength	Same comment as with SINTERLENGTH
~~ZMPOP~~	#2094	7	IDatabase	Overload SortedSetPop
~~ZMSCORE~~	#2082	6.2	IDatabase	Overload SortedSetScore
~~ZRANDMEMBER~~	#2076	6.2	IDatabase	SortedSetRandomMember
~~ZUNION~~	#2075	6.2	IDatabase	SortedSetCombine

The text was updated successfully, but these errors were encountered:

NickCraver · 2022-03-26T12:47:32Z

Hey Steve! This is awesome. My thoughts are:

Yes, I'd like to add these and would welcome PRs.
I'd like to ideally get Nullable Reference Types for main library #2041 in first because it gives us a lot of checks in the compiler for free and in applying it, I recognize a lot of missteps we allowed in result processors e.g. with streams that would have been caught.
Working cross-fork is messy, I'd rather add you as a contributor here (main branch is protected, so no one can mess it up without a PR/check already).

I don't think ground rules for these are an issue, your first PR was fantastic. I propose that we pick a branch naming scheme (e.g. feature/<command>), and ideally lay all of these our as method names since we have the list (to ensure they make sense as a whole). In the past, we added as Redis did and it's a bit organic and not everything matches - IMO it'd be a good exercise to lay these out as the return type and method name(s). This helps because when the return types match for 2 very similar commands, sometimes it's better as an argument. If the return types are different (e.g. long for count vs. actual results), it's obvious they'll never be the same method, and both help inform the naming.

I'm happy if we want to setup a call and go through these or whatever you think works to get those down. IMO the naming is worth an hour to go through ahead of time so we're not figuring it out and conflicts/etc. each PR - much less time overall :)

Thoughts?

NickCraver · 2022-03-27T00:52:51Z

Updating #1743 to get GETEX off the list here :)

slorello89 · 2022-03-27T02:37:00Z

Hey @NickCraver , totally agree, let's come to an agreement on what the API will look like before we get underway. Can set up a call Next week or early (Mon/Tues) the week after.

slorello89 · 2022-03-28T20:11:57Z

@NickCraver, Striking GETEX given #1743 has been merged.

NickCraver · 2022-03-29T01:27:47Z

@slorello89 hmm, at random I went to https://redis.io/commands/zunion/ - the results...aren't what I expect there in the console. General feedback though: the command interface on the new site is nice looking and has good info, but is far less usable. I'm having to go back and re-search when trying to find the command(s) I want, something that was much better in the previous interface. I'm not sure where to put that feedback, but wanted to pass it along!

slorello89 · 2022-03-29T01:40:13Z

Hey @NickCraver, yeah, seeing some of the same with some other commands.

I believe redis-clinterwebz is the home of the app now. @itamarhaber would be the definitive voice on that. Will look at it tomorrow.

slorello89 · 2022-04-04T02:22:40Z

Hey @NickCraver RE laying out what the API will look like, I put together the following table (have a copy in sheets as well) as basically a jumping-off point to kick off this discussion. Happy to go back and forth asynchronously, happy to get on a call too (though I'm going to be out-of-pocket Wed-Fri this week.

EDIT

Moved updated table up to the full table in the first comment.

ttingen · 2022-04-09T18:20:12Z

@slorello89 I'm happy to help with this list if you're interested. @NickCraver sorry for all the nulls you had to fix in streams for #2041, my fault... :/

Avital-Fine · 2022-04-10T07:46:16Z

@slorello89 Did you mean SortedSetCombineAndStore for ZDIFFSTORE command?

NickCraver · 2022-04-10T14:45:23Z

@slorello89 I'm happy to help with this list if you're interested. @NickCraver sorry for all the nulls you had to fix in streams for #2041, my fault... :/

No worries at all! That was me being irked at me for not catching a few things in PR, we'll keep reving this thing better and better and I'm very, very appreciative of the help :) Now we've inboarded the tooling to help us all better as we add more commands and processors so yay. I'm super happy to find some badness in various places with the tooling lit up...give me decent confidence in the wins.

NickCraver · 2022-04-10T14:56:41Z

@slorello89 I went through and updated the list with PR pointers - recommend we collapse your later comment with naming into the original because it's either method names or blocking/punt so that can be 1 column so we can more easily update things to track here - thoughts? Happy to collapse that if agreeable.

slorello89 · 2022-04-11T11:47:57Z

Hey @NickCraver, I collapsed everything as requested, and noted the commands that need more consideration:

ACLs
Functions - the Redis 7 function feature
Shard pub/sub - the Redis 7 Shard Pub-sub feature.

I think probably any command that I've named (with the exception of PUBSUB SHARDCHANNELS and PUBSUB SHARDSUBSCRIBERS) is probably fair game at this point @NickCraver?

I'd also note that we should be careful to separate anything going out in Redis 7, There's a case to be made for having those PRs ready to go (those features are increasingly set), but all the same I don't think any tests written against them are worth anything until Redis 7 is GA.

@Avital-Fine, @ttingen or anyone else working this, I'd say just shout out here what commands you want to work on before getting underway, that way there's no duplication of effort.

I'm going to start on the following today:

GEOSEARCH
GEOSEARCHSTORE
HRANDFIELD
LPOS

PS I met @ttingen at Codestock last week - was nice catching up, he was super excited to hear about the work happening with the library :)

Avital-Fine · 2022-04-11T12:09:09Z

I will start now with EXPIRETIME and PEXPIRETIME commands.
Also, I know that there are new features to EXPIRE, PEXPIRE, EXPIREAT, PEXPIREAT so I'll check it out :)

Avital-Fine · 2022-04-11T12:43:40Z

Btw, after we finish with the commands we need to check for missing features.
For example, I notice that SortedSetAdd (ZADD) doesn't support GT and LT options which were added in 6.2.0

NickCraver · 2022-04-11T12:52:54Z

Heads up: it occurred to me we're not actually testing 7.0-rc bits because of the Docker container version. I remedied that in #2079 which should be followed by a merge of main into any PRs with 7.0 features so we're properly exercising those paths :)

Adds support for https://redis-stack.io/commands/sintercard/ (#2055) Co-authored-by: Nick Craver <[email protected]>

ttingen · 2022-04-12T01:07:11Z

I will start with the following:
XAUTOCLAIM
ZRANDMEMBER

NickCraver · 2022-04-12T01:09:21Z

@ttingen ZRANDMEMBER is up in #2076 ;)

ttingen · 2022-04-12T01:15:29Z

@ttingen ZRANDMEMBER is up in #2076 ;)

Oops, let's make that XAUTOCLAIM and ZMSCORE.

Implements [`HRANDFIELD`](https://redis.io/commands/hrandfield/) as a part of #2055. Co-authored-by: Nick Craver <[email protected]>

Support for https://redis.io/commands/zmscore/ (#2055) Co-authored-by: Nick Craver <[email protected]>

Avital-Fine · 2022-04-14T14:31:43Z

Starting working on LCS

ttingen · 2022-04-14T14:35:21Z

I have XAUTOCLAIM working, just need to write more tests. Should be able to get it out for review tonight.

This PR implements [`GEOSEARCH`](https://redis.io/commands/geosearch/) and [`GEOSEARCHSTORE`](https://redis.io/commands/geosearchstore/) for #2055 To abstract the box/circle sub-options from GEOSEARCH I added a new abstract class `GeoSearchShape` who's children are responsible for maintaining the bounding shape, its unit of measurement, the number of arguments required for the sub-option, and of course the sub-option name. Rather than casting/extracting the arguments, I have it use an IEnumerable state-machine with yield/return. Wasn't sure which was the better option, the IEnumerable seemed cleaner, open to whichever you want. I changed the `GEORADIUS` pattern of having a `GeoSearch(key, member, args. . .)` and a `GeoSearch(key, lon, lat, args. . .)`, and instead have `GeoSearchByMember` and `GeoSearchByCoordinates`. If I'm honest, it was because my IDE was complaining about breaking compatibility rules by having more than 1 override with optional parameters, not sure if you have a strong feeling on this. Co-authored-by: Nick Craver <[email protected]>

Adds support for https://redis.io/commands/expiretime/ https://redis.io/commands/pexpiretime/ (#2055) Co-authored-by: Nick Craver <[email protected]>

Added [XAUTOCLAIM](https://redis.io/commands/xautoclaim/) support as part of #2055 The XCLAIM command has two methods, StreamClaim & StreamClaimIdsOnly. Since the XAUTOCLAIM result is a bit more complex than the XCLAIM command, I opted to consolidate to a single method and single result type. To return just the message IDs in the result, the `idsOnly` parameter should be set to `true`. The caveat here is that the `StreamEntry` instances will only have `Id` populated when `idsOnly` is `true`, the `Values` array will be empty. This behavior is called out in the XML docs for the method. I wasn't sure if this is the proper "feel" you want for this command or not. Let me know if you want to break it up like `StreamClaim`. Co-authored-by: Nick Craver <[email protected]>

Implementing [`ZMPOP`](https://redis.io/commands/zmpop/) and [`LMPOP`](https://redis.io/commands/lmpop/) as part of #2055 Co-authored-by: Nick Craver <[email protected]>

Adds support for https://redis.io/commands/object-freq/ (#2055) Co-authored-by: Nick Craver <[email protected]>

@NickCraver

Introducing [`SORT_RO`](https://redis.io/commands/sort_ro/) as part of #2055 @NickCraver - I modified the path for message creation for the Sort command to point to using `SORT_RO` when possible, this again had to do a version-check against the multiplexer, which I'm not certain is the correct way - thoughts? Also, SORT is considered a write command and will be rejected out of hand by a replica (so I'm moving that as well) ```text 127.0.0.1:6378> SORT test (error) READONLY You can't write against a read only replica. ``` Co-authored-by: Nick Craver <[email protected]>

Adds support for https://redis.io/commands/lcs/ (#2055) Co-authored-by: slorello89 <[email protected]> Co-authored-by: Nick Craver <[email protected]>

Add support for `COMMAND` commands (part of #2055): COMMAND COUNT - https://redis.io/commands/command-count/ COMMAND GETKEYS - https://redis.io/commands/command-getkeys/ COMMAND LIST - https://redis.io/commands/command-list/ Co-authored-by: Nick Craver <[email protected]>

tyler-boyd · 2022-07-20T03:22:22Z

re: sharded pub/sub, is it as simple as changing PUBLISH commands to SPUBLISH etc? Or is there some drawback to the sharded version that requires more thought?

mgravell · 2022-07-20T06:37:03Z

IIRC we already have equivalent routing code, so it shouldn't be a huge change - however, there are non-trivial validations needed here, to ensure that we route correctly for both initial and reconnect scenarios, and to check the routing isn't impacted for non-routed subscriptions.

caoyang1024 · 2022-10-02T07:32:53Z

Hi, can I check the plan on commands (SPUBLISH, SSUBSCRIBE, SUNSUBSCRIBE) for sharded pub-sub.

NickCraver added the ⚙️ area:API label Mar 26, 2022

NickCraver mentioned this issue Mar 27, 2022

Implement GETEX command #1743

Merged

slorello89 mentioned this issue Apr 3, 2022

Support BITFIELD and BITFIELD_RO #2061

Closed

tombatron mentioned this issue Apr 3, 2022

Lolwut #2060

Closed

Avital-Fine mentioned this issue Apr 4, 2022

Support LMOVE #2065

Merged

slorello89 mentioned this issue Apr 10, 2022

"-READONLY You can't write against a read only replica." on ScriptEvaluate(...) #2074

Closed

This was referenced Apr 10, 2022

Sorted set new commands: ZDIFF, ZDIFFSTORE, ZINTER, ZINTERCARD, and ZUNION #2075

Merged

Support ZRANDMEMBER #2076

Merged

Support SMISMEMBER #2077

Merged

Avital-Fine mentioned this issue Apr 11, 2022

Support SINTERCARD #2078

Merged

slorello89 mentioned this issue Apr 11, 2022

LPOS feature #2080

Merged

NickCraver added a commit that referenced this issue Apr 11, 2022

Support SINTERCARD (#2078)

ba352d2

Adds support for https://redis-stack.io/commands/sintercard/ (#2055) Co-authored-by: Nick Craver <[email protected]>

NickCraver added a commit that referenced this issue Apr 14, 2022

HRANDFIELD feature (#2090)

6b37116

Implements [`HRANDFIELD`](https://redis.io/commands/hrandfield/) as a part of #2055. Co-authored-by: Nick Craver <[email protected]>

NickCraver added a commit that referenced this issue Apr 14, 2022

Support the ZMSCORE command. (#2082)

cb571bd

Support for https://redis.io/commands/zmscore/ (#2055) Co-authored-by: Nick Craver <[email protected]>

slorello89 mentioned this issue Apr 14, 2022

ZMPOP and LMPOP #2094

Merged

ttingen mentioned this issue Apr 15, 2022

Support the XAUTOCLAIM command. #2095

Merged

NickCraver added a commit that referenced this issue Apr 16, 2022

Support EXPIRETIME and PEXPIRETIME (#2083)

bcc647b

Adds support for https://redis.io/commands/expiretime/ https://redis.io/commands/pexpiretime/ (#2055) Co-authored-by: Nick Craver <[email protected]>

This was referenced Apr 19, 2022

Support LCS #2104

Merged

Support OBJECT FREQ #2105

Merged

slorello89 mentioned this issue Apr 19, 2022

BITFIELD and BITFIELD_RO feature #2107

Open

NickCraver added a commit that referenced this issue Apr 19, 2022

ZMPOP and LMPOP (#2094)

dbd52f1

Implementing [`ZMPOP`](https://redis.io/commands/zmpop/) and [`LMPOP`](https://redis.io/commands/lmpop/) as part of #2055 Co-authored-by: Nick Craver <[email protected]>

NickCraver added a commit that referenced this issue Apr 19, 2022

Support OBJECT FREQ (#2105)

f0fa79c

Adds support for https://redis.io/commands/object-freq/ (#2055) Co-authored-by: Nick Craver <[email protected]>

slorello89 mentioned this issue Apr 19, 2022

Add support for SORT_RO #2111

Merged

Avital-Fine mentioned this issue Apr 20, 2022

Support SORT_RO + Test all SORT commands #2112

Closed

NickCraver mentioned this issue Apr 27, 2022

Redis 7 support #2121

Closed

NickCraver added a commit that referenced this issue May 16, 2022

Support LCS (#2104)

09a067c

Adds support for https://redis.io/commands/lcs/ (#2055) Co-authored-by: slorello89 <[email protected]> Co-authored-by: Nick Craver <[email protected]>

chayim mentioned this issue May 19, 2022

Support COMMAND [...] #2143

Merged

shacharPash mentioned this issue Jun 15, 2022

Add Support to 'SCRIPT KILL' Command #2158

Closed

tomap mentioned this issue Aug 30, 2022

SignalR Scale-out Doc dotnet/AspNetCore.Docs#7814

Closed

This was referenced Nov 19, 2022

Support Redis Clusters with Redis 7 & SSubscribe/SPublish dotnet/AspNetCore.Docs#27693

Closed

Support Redis Clusters with Redis 7 & SSubscribe/SPublish SignalR/SignalR#4656

Closed

dominikjeske mentioned this issue Feb 2, 2023

Get available ACL from server? #2360

Closed

BrennanConroy mentioned this issue Feb 2, 2023

SignalR Redis Cluster scale-out: Message throughput scalability dotnet/aspnetcore#46408

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Missing Commands in StackExchange.Redis #2055

Missing Commands in StackExchange.Redis #2055

slorello89 commented Mar 25, 2022 •

edited by NickCraver

Loading

NickCraver commented Mar 26, 2022

NickCraver commented Mar 27, 2022

slorello89 commented Mar 27, 2022

slorello89 commented Mar 28, 2022

NickCraver commented Mar 29, 2022

slorello89 commented Mar 29, 2022

slorello89 commented Apr 4, 2022 •

edited

Loading

ttingen commented Apr 9, 2022 •

edited

Loading

Avital-Fine commented Apr 10, 2022

NickCraver commented Apr 10, 2022

NickCraver commented Apr 10, 2022

slorello89 commented Apr 11, 2022

Avital-Fine commented Apr 11, 2022

Avital-Fine commented Apr 11, 2022

NickCraver commented Apr 11, 2022

ttingen commented Apr 12, 2022

NickCraver commented Apr 12, 2022

ttingen commented Apr 12, 2022

Avital-Fine commented Apr 14, 2022

ttingen commented Apr 14, 2022

tyler-boyd commented Jul 20, 2022

mgravell commented Jul 20, 2022

caoyang1024 commented Oct 2, 2022

Missing Commands in StackExchange.Redis #2055

Missing Commands in StackExchange.Redis #2055

Comments

slorello89 commented Mar 25, 2022 • edited by NickCraver Loading

Methodology

Full List

NickCraver commented Mar 26, 2022

NickCraver commented Mar 27, 2022

slorello89 commented Mar 27, 2022

slorello89 commented Mar 28, 2022

NickCraver commented Mar 29, 2022

slorello89 commented Mar 29, 2022

slorello89 commented Apr 4, 2022 • edited Loading

ttingen commented Apr 9, 2022 • edited Loading

Avital-Fine commented Apr 10, 2022

NickCraver commented Apr 10, 2022

NickCraver commented Apr 10, 2022

slorello89 commented Apr 11, 2022

Avital-Fine commented Apr 11, 2022

Avital-Fine commented Apr 11, 2022

NickCraver commented Apr 11, 2022

ttingen commented Apr 12, 2022

NickCraver commented Apr 12, 2022

ttingen commented Apr 12, 2022

Avital-Fine commented Apr 14, 2022

ttingen commented Apr 14, 2022

tyler-boyd commented Jul 20, 2022

mgravell commented Jul 20, 2022

caoyang1024 commented Oct 2, 2022

slorello89 commented Mar 25, 2022 •

edited by NickCraver

Loading

slorello89 commented Apr 4, 2022 •

edited

Loading

ttingen commented Apr 9, 2022 •

edited

Loading