Skip to content

Bucket Shadowing Design Notes

Jump to bottom
Jens Alfke edited this page Jan 8, 2014 · 1 revision

OLD DESIGN NOTES — 18 Oct 2013

This is a third proposed solution to the problem of letting Couchbase Server apps coexist with the Sync Gateway (the dreaded issue #7.) I think it's better, and probably easier to implement, than either of the two others we've talked about.

The Big Idea

Rather than try to share a bucket with a Couchbase Server app, have the Sync Gateway use a separate bucket. The gateway's bucket will operate pretty much as it does today, but the gateway will also watch the app's bucket (using Tap or XDCR) and apply the changes to its own copies of the documents. Mobile clients will replicate from/to the gateway's bucket as they already do. The gateway can propagate changes made by mobile clients back to the corresponding documents in the app bucket.

Details are below, in the appropriately-named "More Details" section.

Benefits

  • This should work with existing Couchbase 2.x servers; no need to request new features.
  • Only minor slowdown of the app bucket: just the overhead of sending the Tap notifications.
  • Doesn't mess with the original bucket by adding funny metadata fields. Doesn't even require write access to the bucket, if the app doesn't need mobile changes propagated back to it.
  • The Couchbase client code in the app continues to talk directly to Couchbase Server without having to know about the gateway at all.
  • We can subset the original bucket if desired, by applying a fast filter to the change notifications from Tap; that way the gateway only has to scale to the number of documents that are relevant to it, which might be a small fraction of what's in the app.
  • It's very easy to detect changes made by the app and incorporate them into the document's revision history.
  • We can be one-way if desired, never propagating any changes back to the app bucket. Or we can propagate only certain changes; the app can be in charge of accepting them, e.g. via a custom REST endpoint it runs.
  • The change-watching process can be abstracted enough to make it a plug-in, and it could even read from some other type of server or data source entirely, like SAP. The only requirements are that it needs to provide a change feed and a document change-id property (like a CAS).

Drawbacks

  • Requires an extra bucket.
  • Duplicates the document bodies. (We could avoid this, but it would make the algorithm more complex, slow down document access, and add overhead to the app bucket.)
  • Requires a task to monitor the app bucket's Tap feed. This can be done within a gateway process, but there should be only one, so if there are multiple gateway nodes they'd have to agree on who did it (with failover, etc.)
  • Introduces some latency before changes made by the app show up to mobile clients; it should be very little, but it depends on whether the Tap feed and incoming-changes handlers can keep up.

More Details

Setup

  1. Create a new empty bucket for the gateway's use (as today).
  2. Create a JSON config file including the URL of the gateway bucket (as today).
  3. Add a new property to the config: the URL of the app bucket.
  4. Start the gateway.

Startup

(Written from the gateway's perspective)

  1. When opening a database, read a lastTapCount property from a special key in its bucket. This is the last Tap count that's been processed from the app bucket. If missing, it defaults to zero.
  2. Create a Tap feed, including backfill, starting from the saved count.

Note that on the first run of a new database, the Tap backfill will cause the database to be populated with the current contents of the app bucket as the Incoming Changes algorithm (q.v.) runs.

Incoming Changes

When a Tap notification arrives from the app bucket:

  1. Optional: test it against an app-provided quick filter. For example, we might only pay attention to docs whose ID starts with "mobile:". (This filter is going to be run at high volume so it probably shouldn't use JavaScript. Maybe just a regex match on the ID.)
  2. Get the doc with the matching ID in the gateway bucket (the "gateway doc"). If there is none, start with a JSON object initialized to an empty state.
  3. Compare the upstreamCAS property of the gateway doc to the CAS value in the Tap notification; if they're equal, ignore the notification (it must be a dup or something.)
  4. Get the upstreamRevID property of the gateway doc, if there is one; this will be considered the parent revision ID.
  5. Insert the document body from the Tap notification as a new revision with the given parent ID, as though it were coming in from a push replication. This will run it through the sync function, assign channels, push it to _changes listeners, etc. (Question: If there's a CAS conflict with the insert, do I need to go back to step 3 instead of just retrying the stuff in step 5?)
  6. Update the gateway document's upstreamCAS and upstreamRevID properties as part of saving the new revision.
  7. Update the bucket's lastTapCount value.

Outgoing Doc Changes

When a document is updated by a replication client or the gateway REST API (but not by a Tap notification):

  1. Optional: test it against an app-provided filter to see if it should be copied back to the app server.
  2. Check whether the document's default/winning revision's ID is the same as upstreamRevID. If so, there's nothing more to do.
  3. Store the default revision's body to the app bucket, but only if the doc in the app bucket still has a CAS that matches the upstreamCAS. Otherwise, get the app document again and go back to the previous step. (Note: This could be a direct Put to the app bucket, or it could go through a web-hook in the app server.)
  4. Update the upstreamCAS and upstreamRevID in the gateway doc to correspond to the revision sent to the app bucket. (Note: There are probably race conditions here I haven't considered yet.)

Questions

  • Is Tap or XDCR a better protocol to use? Use Tap.
  • Do we need to store our own copies of the doc bodies, or is the metadata sufficient?
  • Do we need to store the entire revision tree, every change made in the original bucket, or can we collapse them?
  • Do we have to be careful about waiting for persistence of changes from the original bucket? (If so, the upcoming UPR features will help.)
  • If we do write changes back to the original bucket, what happens when there are conflicts?
Clone this wiki locally