Skip to content

Commit

Permalink
update license and docs
Browse files Browse the repository at this point in the history
  • Loading branch information
mrdimidium committed Apr 29, 2024
1 parent 02ffd7a commit 19d670a
Show file tree
Hide file tree
Showing 4 changed files with 25 additions and 20 deletions.
22 changes: 17 additions & 5 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,9 +1,21 @@
Copyright 2024 Victor Grishchenko
MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
Copyright 2024 Victor Grishchenko & dRPC.org

This codebase shall not be used for training AI models of any kind.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the “Software”), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
File renamed without changes.
7 changes: 2 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

Chotki is a syncable store with really fast counters.
Internally, it is [pebble db][p] running CRDT natively, using
the [Replicated Data Interchange][r] format (RDX). Chotki is
the [Replicated Data Interchange](./rdx) format (RDX). Chotki is
sync-centric and causally consistent. That means, Chotki
replicas can sync master-master, either incrementally in
real-time or stay offline to diff-sync periodically.
Expand Down Expand Up @@ -113,12 +113,10 @@ go get -u github.com/drpcorg/chotki

## Inner workings

Internally, Chotki is [pebble db][p] using [RDX][r] merge operators.
Internally, Chotki is [pebble db][p] using [RDX](./rdx) merge operators.
See the RDX doc for further details on its serialization format
(type-length-value) and a very predictable choice of CRDTs.

[r]: https://github.com/learn-decentralized-systems/Chotki/tree/main/rdx

## Comparison to other projects

Overall, RDX is the final version of RON (Replicated Object
Expand Down Expand Up @@ -177,4 +175,3 @@ busywork. Updates are lamport-timestamped, there is a re-sync protocol for
newly joining and re-joining replicas. Overall, we take every shortcut to make
the store lightweight and fast while focusing on our specific usecase
(distributed counters, mainly).

16 changes: 6 additions & 10 deletions rdx/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ offline-first and peer-to-peer replication, with no central
server required, as any two *replicas* can merge their data. By
installing RDX data types as merge operators in an LSM database
(leveldb, RocksDB, pebble, Cassandra, etc) one can effectively
have a CRDT database (which [Chotki][c] basically is).
have a CRDT database (which [Chotki](./ARCHITECTURE.md) basically is).

We will implement *unified* CRDTs able to synchronize using
operations, full states or deltas. Types may imply [causal
Expand All @@ -31,7 +31,7 @@ Our objects can have fields of the following CRDT types. Each
type is named by a letter.

1. last-write-wins variables (`I` for int64, `S` for string, `F`
is float64, and `R` is [id64][i])
is float64, and `R` is [id64](./rdx/id.go#12))
2. counters, `N` increment-only uint64 and `Z` two-way int64
3. maps (M), like key-value maps, where keys and values are `FIRST`
4. sets (E), contain arbitrary `FIRST` elements
Expand All @@ -52,13 +52,13 @@ let's see how a bare (no TLV envelope) `I` int64 `-11` would
look like, assuming it is the 4th revision of the register
autored by replica #5. The TLV would look like: `32 08 05 15`
(hex) where `0x15` is a [zig-zag][g] encoded and zipped `-11`,
while `32 08 05` is a tiny [ToyTLV][t] record for a zipped pair
while `32 08 05` is a tiny [ToyTLV](./protocol/tlv.go) record for a zipped pair
of ints, 4 (signed, zig-zagged, so `08`) and 5 (unsigned, so
`05`). If we add a ToyTLV envelope, that becomes `69 04 32 08 05
15` (type of record `I`, length 4, then the bare part).

String `S` values are simply UTF-8 strings. Int64 `I`, float64
`F` and id64 `R` values get compressed using [`zip_int`][z]
`F` and id64 `R` values get compressed using [`zip_int`](./rdx/zipint.go)
routines. Overlong encodings are forbidden both for strings and
for zip-ints!

Expand Down Expand Up @@ -222,7 +222,7 @@ here we imply `I` last-write-wins int64.

## Serialization format

We use the [ToyTLV][t] format for enveloping/nesting all data.
We use the [ToyTLV](./protocol/tlv.go) format for enveloping/nesting all data.
That is a bare-bones type-length-value format with zero
semantics. What we put into ToyTLV envelopes is integers,
strings, and floats. Strings are UTF-8, no surprises. Floats are
Expand All @@ -231,7 +231,7 @@ as a compressed pair of integers.

A note on integer compression. From the fact that protobuf
has about ten integer types, one can guess that things can
be complicated here. We use [ZipInt][z] routines to produce
be complicated here. We use [ZipInt](./rdx/zipint.go) routines to produce
efficient varints in a TLV format (differently from protobuf
which has a separate bit-level [LEB128][b] coding for ints).

Expand All @@ -245,15 +245,11 @@ which has a separate bit-level [LEB128][b] coding for ints).
id64 and logical timestamps get packed as pairs of uint64s. All
zip codings are little-endian.

[c]: https://github.com/learn-decentralized-systems/Chotki/blob/main/ARCHITECTURE.md
[x]: https://en.wikipedia.org/wiki/Causal_consistency
[v]: https://en.wikipedia.org/wiki/Version_vector
[r]: https://www.educative.io/answers/how-are-vector-clocks-used-in-dynamo
[j]: https://en.wikipedia.org/wiki/RDX
[p]: https://en.wikipedia.org/wiki/Remote_procedure_call
[z]: https://github.com/learn-decentralized-systems/Chotki/blob/main/zipint.go
[g]: https://protobuf.dev/programming-guides/encoding/
[t]: https://github.com/learn-decentralized-systems/toytlv
[b]: https://en.wikipedia.org/wiki/LEB128
[i]: https://github.com/learn-decentralized-systems/Chotki/blob/main/id.go#L12
[m]: https://en.wikipedia.org/wiki/Merge_sort

0 comments on commit 19d670a

Please sign in to comment.