Open
Conversation
…ssages (#2276) In a multiplexed setup we often see a timeout behind a large payload in the process of parsing (or not). This is meant to help better diagnose that case, by recording what the last payload size was that came through this connection. Needs some @mgravell eyes on the record point here and if we want to adjust (or if it's wrong due to buffers in a way I'm not understanding). I realize this won't work on a payload so large enough it blows all buffers completely, but I don't see how to solve that either the way we properly stream parse. I think it'll still improve most cases we see this in.
1. add a tunnel config option, which is used to influence connection creation 2. implement http-tunnel as a well-known tunnel, integrated via the `http:` prefix on the tunnel option tunnels are implemented as concrete subclasses of the `Tunnel` type; an HTTP proxy "connect" implementation is provided as a well-known version that is supported inside `Parse` - but custom 3rd-party tunnel implementations can also be provided via the object-model (not `Parse`) A `Tunnel` allows: - overriding the `EndPoint` used to create `Socket` connections, or to suppress `Socket` creation entirely (by default, the same logical endpoint requested is provided back out) - provide a "before socket connect" twin (mirrors delegate approach, but: async; by default, do nothing) - provide a "before authenticate" injection point, which can a: perform additional handshake operations, and b: subvert the entire `Stream` (by default, nothing is done and no custom stream is returned) Co-authored-by: maksimkim <maksim.kim@gmail.com> Co-authored-by: Nick Craver <nrcraver@gmail.com>
…5"' (#2254) The old wording equated `<httpRuntime targetFramework="4.5" />` to `<add key="aspnet:UseTaskFriendlySynchronizationContext" value="false" />`, when the linked citation makes it clear that the effect is the opposite: `<httpRuntime targetFramework="4.5" />` is inferred as adding the setting `<add key="aspnet:UseTaskFriendlySynchronizationContext" value="true" />`. > Second, `<httpRuntime targetFramework="4.5" />` is a shortcut that allows the ASP.NET runtime to infer a wide array of configuration settings. If the runtime sees this setting, it will expand it out just as if you had written the following: > ``` > ... > <add key="aspnet:UseTaskFriendlySynchronizationContext" value="true" /> > ``` https://devblogs.microsoft.com/dotnet/all-about-httpruntime-targetframework/ Co-authored-by: Nick Craver <nrcraver@gmail.com>
This aligns all tests to *Tests.cs (and class names), moves a few heleprs to Helpers\ and general cleanup - no net significant changes. Next: finally renaming the wrapper classes to be sensible :)
This renames the various wrappers to IMO more intuitive "KeyPrefixedX" - just couldn't get used to the previous model when touching N files for an API addition, hoping this makes it a little clearer overall, also has the benefit of batching them in UI due to common prefix.
The original purpose of this was to get system-level CPU to help advise people filing issues that their machine overall was under too much load and that's why we were seeing timeouts. However, due to ecosystem changes and shifts the actual reporting of this counter has dropped off so dramatically it's not actually doing what it's supposed to be doing and giving us signal data to help. Given it's a dependency chain that also depends ultimately on some problematic packages now (e.g. System.Drawing.Common) and isn't cross-platform correctly...let's just remove it. It's not a net win anymore. Fixes #2283.
#2284) From my reading of code, the sync execute path uses `syncTimeout` and applies it [instantaneously through](https://github.com/StackExchange/StackExchange.Redis/blob/main/src/StackExchange.Redis/ConnectionMultiplexer.cs#L1879) `Monitor.Wait`. Looks like the heartbeatInterval only applies to async commands? https://github.com/StackExchange/StackExchange.Redis/blob/main/src/StackExchange.Redis/PhysicalConnection.cs#L690
Meant to help address #1520, #1660, #2074, and #2020. I'm not 100% sure about this because if there is a MOVED happening (e.g. bad proxy somewhere) this would just continually re-run...but only once every 5 seconds. Overall though, we linger in a bad state retrying moves until a discovery happens today and this could be resolved much faster.
Right now we don't pay attention to fail state (PFAIL == FAIL) and continue trying to connect in the main loop. I don't believe this was intended looking at the code, we just weren't handling the flag appropriately. Added now. Docs at: https://redis.io/commands/cluster-nodes/
Fixing a few requests here.
…nected (#2300) More info to help us advise and debug timeouts for users! Overall adds for timeout messages: - `Sync-Ops`: A count of synchronous operation calls - `Async-Ops`: A count of asynchronous operation calls - `Server-Connected-Seconds`: How long the bridge in question has been connected ("n/a" if not connected)
…umerator (#2337) So we aren't limited *just* to `ReadOnlySpan<ServerEndpoint>`, and don't need the allocatey `ToArray()` Implemented as custom iterator which allows `async` and LINQ to work directly' existing code still uses span for efficiency, with the new API used in limited scenarios only Intent here: 1. Provide an efficient basis for filtered "all server's matching X" functionality 2. Provide the `Where(CommandFlags)` basis for upcoming broadcast work 3. Avoid some unnecessary allocations
…ver finishes (#2338) What happens here is that `PhysicalConnection` attempts to connect but never hears back, so it's stuck in `State.ConnectedEstablishing` state, which is handled in the heartbeat code. In the situation where the "last write seconds ago" passes in the heartbeat and we haven't heard back, we fire an `PhysicalBridge.OnDisconnected()` which clears out `physical` and orphans the socket listening forever. This now properly disposes of that `PhysicalConnection` mirroring like we do in `State.Connecting` which will properly fire `OnDisconnected()` and clean up the orphan socket. The situation manifests where we establish a TCP connection, but not a Redis connection. All of the cases in the memory dump we're analyzing are some bytes sent and 0 received. Likely a Redis server issue, but the client is then handling it incorrectly and leaking. I nuked the unused `RemovePhysical` method just to prevent further oops here. Addresses a new case of #1458.
Adds: Support for `EVAL_RO` and `EVALSHA_RO` via `IDatabase.ScriptEvaluateReadOnly`/`IDatabase.ScriptEvaluateReadOnlyAsync` Co-authored-by: Nick Craver <nrcraver@gmail.com>
1. Found a few inconsistencies when doing https://github.com/StackExchange/StackExchange.Redis/releases - mirroring on the markdown. 2. Adjust the project URL for NuGet to https://stackexchange.github.io/StackExchange.Redis/ so that people can go to "main page" and repo (rather than the repo linked twice) from NuGet.org as intended.
* fix #2368 - implement Length() for other encodings (using format layout) - unify format code - switch to C# 11 for u8 strings (needed a few "scoped" modifiers adding) - tests for format and Length * tweak langver * cleanup double format * use 7.0.101 SDK (102 not yet on ubuntu?) * tweak SDK in CI.yml; add CI.yml to sln * We need Redis 6 runtime for tests, so let's grab both * Add release notes --------- Co-authored-by: Nick Craver <nrcraver@gmail.com>
…2378) * fix #2376 1. to fix the immediate scenario: don't hold the queue lock when we abort things - only hold it when fetching next 2. to avoid similar not yet seen: in GetHeadMessages, don't blindly wait forever also standardise on TryPeek/TryDequeue * ExecuteSyncImpl: don't hold the lock-obj when throwing for timeout * use placeholder message when unable to query the connection queue * release notes
* AppVeyor: move to VS 2022 image The update blog post (https://www.appveyor.com/updates/2023/02/24/) doesn't match the inventory (https://www.appveyor.com/docs/windows-images-software/), primarily: .NET 6 SDK which is LTS is no longer on the 2019 image which seems like an oops. Bypassing the issue for now by getting on the 2022 image. * Okay...install both I guess Cloud images are fun! * bahhhhhhhhh * 4th time's a charm * Maybe 5th * Force it * 7.0.200 * trying explicitly named packages * Splitting lines --------- Co-authored-by: slorello89 <steve.lorello@redis.com>
* expose IAsyncEnumerable on ChannelMessageQueue fix #2400 * PR number * move ChannelMessageQueue.GetAsyncEnumerator to shipped
Update the 'Configuration' documentation page with the correct default 'ReconnectRetryPolicy' value. Reference: - [src/StackExchange.Redis/ConfigurationOptions.cs](https://github.com/SonnyRR/StackExchange.Redis/blob/improvement/configuration-docs-retry-policy/src/StackExchange.Redis/ConfigurationOptions.cs#L466) - [src/StackExchange.Redis/Configuration/DefaultOptionsProvider.cs](https://github.com/SonnyRR/StackExchange.Redis/blob/improvement/configuration-docs-retry-policy/src/StackExchange.Redis/Configuration/DefaultOptionsProvider.cs#L152)
* Update README to include Azure Managed Redis * Removed ACR
* add new KeyNotification API - KeyNotification wraps channel+value, exposes friendly parsed members (db, type, etc) - KeyNotificationType is new enum for known values - add TryParseKeyNotification help to ChannelMessage (and use explicit fields) * clarifications * RedisChannel creation API * assert non-sharded in tests * simplify database handling for null and zero * Add API for KeyEvent usage with unexpected event types * nits * optimize channel tests * nit * assertions for multi-node and key-routing logic * prevent publish on multi-node channels * naming * split Subscription into single/multi implementation and do the necessary * initial tests; requires CI changes to be applied * enable key notifications in CI * implement alt-lookup-friendly API * improve alt-lookup logic * Bump tests (and CI etc) to net10, to allow up-to-date bits * queue vs handler tests * docs; moar tests * docs * fix routing for single-key channels * Consider keyspace and channel isolation * Update KeyspaceNotifications.md * Much better API for handling keyspace prefixes in KeyNotification * clarify docs * docs are hard * words * Fix incorrect routing of pub/sub messages on cluster when using channel prefix * simplify channel-prefix passing * - reconnect RESP3 channel subscriptions - EnsureSubscribedToServer[Async] can now *remove* subscriptions in the multi-node case * runner note * make SubscriptionsSurviveConnectionFailureAsync more reliable; Ping is not routed to the channel, and can use primary or replica, so Ping is not a reliable test *unless* we demand master * rem net8.0 tests * - allow single-node subscriptions to follow relocations - prevent concurrency problem on TextWriterOutputHelper * improve subscription recovery logic when using key-routed subscriptions * docs typo * Update docs/KeyspaceNotifications.md Co-authored-by: Philo <philon@microsoft.com> * Update docs/KeyspaceNotifications.md Co-authored-by: Philo <philon@microsoft.com> * Update version.json --------- Co-authored-by: Philo <philon@microsoft.com>
* Options: Split out AMR to its own options provider This breaks out AMR into its own default options provider to specify default versions but also allow us to do things like specify default RESP later on. * Add AMR to BuildInProviders * Add new cloud endpoints for OSS Azure Redis * Fix/add DefaultOptions tests * Move Enterprise domain root from OSS to AMR --------- Co-authored-by: Philo <philon@microsoft.com>
* - XADD [IDMP|IDMPAUTO] - XINFO new fields - fix delta in XTRIM in 8.6 * - XCFGSET * - add [Experimental] - add docs
* propose API for HOTKEYS * stubs * untested stab at message impl * untested result processor * basic integration tests * more integration tests * release notes and [Experimental] * github link * sample ration default is 1, not zero * - RESP3 - don't expose raw arrays - expose API-shaped ms/us accessors - reuse shared all-slots array * validate/fix cluster slot filter * validate duration * docs; more tests and compensation * make SharedAllSlots lazy; explicitly track empty cpu/network/slots * More docs * "wow"? * more words * update meaning of count * expose a bunch of values that are conditionally present * tests on the sampled/slot-filtered metrics * - naming in HotKeysResult - prefer Nullable<T> when not-always-present * pre-empt typo fix * CI: use internal 8.6 preview build * additional validation on conditional members * CI image update * stabilize CI for Windows Server * be explicit about per-protocol/collection on cluster
* test * docs * Revert "test" This reverts commit ecd1189. * Update Authentication.md * Update Authentication.md * Update index.md * prefer async
…iption; fix inappropriate async void (#3013)
* Propose CAS/CAD docs * internal link * clarify usage of LockExtendAsync * reuse local * Update docs/CompareAndSwap.md Co-authored-by: Philo <philon@microsoft.com> --------- Co-authored-by: Philo <philon@microsoft.com>
* Implement VRANGE support (8.4) * docs * using lease for VRANGE * release notes * space * Update docs/VectorSets.md Co-authored-by: Philo <philon@microsoft.com> * Update docs/VectorSets.md Co-authored-by: Philo <philon@microsoft.com> --------- Co-authored-by: Philo <philon@microsoft.com>
* Handle MOVED error pointing to same endpoint by triggering reconnection before retrying the request. * Better stimulate proxy/LB * Increase timeout * Fixed key name to prevent collisions * Add NeedsReconnect flag to defer reconnection to reader loop for MOVED-to-same-endpoint * drop unstable test --------- Co-authored-by: Marc Gravell <marc.gravell@gmail.com>
* - add test infrastructure for rich in-proc dummy servers, for simulating connection state - migrate the -MOVED/self test to the new infrastructure, and use better server/cluster/etc impl * log client id
* wip * Multi-node test infrastructure * respect CROSSSLOT in the toy server * extensively use "in" in the toy server; lots of stack values * implement decrby (for desting from CI) * implement basic TTL to support basic-ops test in primary tests * basic resp3 * - simplify connecting to the test server - include all commands and endpoints when connecting to the test server - better RESP3 aggregate support * run basic tests on RESP2+3 * Fix remaining RESP3 handshake snafus * test fixture for in-proc; run with basic tests * implement everything in multi/exec except the serialize/deserialize (d'oh!) * also force attributes to be pairs (same semantics as maps) * don't buffer any of the transaction commands
* likely looking pub/sub * pub/sub test fixes * cleanup on outbound write * fix publish return val * ping/subs * fix pub/sub command check * disable InProcPubSubTests for now, server is brittle
* Track the number of live multiplexers, and report it in the error dump if non-trivial * fight the CI on a CS9336 mixup * simplify counting
…ophic failure: System.InvalidOperationException : End position was not reached during enumeration.`
* minimal pre-v3 consolidation: - add RESPite to the solution, but *DO NOT* reference from SE.Redis - update eng tooling and migrate field parsing from FastHash to AsciiHash - update the toy server *completely* to the v3 specification * - backport LCS and RedisType.VectorSet changes from v3 * CI * tidy TFMs * nix TestHarness * deal with TFM-specific warnings from .NET 10 * in-proc test enhancements * fix Select * fix "fromBytes" logic to be more formal, and more importantly: to keep Nick happy * make DebugAssertValid logic #if DEBUG * cleanup a great many #if * join NET7 => NET8 * join net9 with net10 * fix high integrity token check on AUTH/HELLO * borked a #endif * Use a better way of refing the build project * clarify public API files * exclude Build.csproj from refing the analyzer project! * - improve "moved" test for "did the client already know the new node?" - fix where RESP3 default is specified for the toy server * make test server output more terse * toy server: don't pretend SENTINEL exists * rev minor * bump * Include endpoint data in log, for example `[127.0.0.1:6379 #1] CLIENT => +OK`
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.