Skip to content
This repository has been archived by the owner on Oct 18, 2023. It is now read-only.

Add Hrana 3 #548

Merged
merged 9 commits into from
Aug 23, 2023
Merged

Add Hrana 3 #548

merged 9 commits into from
Aug 23, 2023

Conversation

honzasp
Copy link
Contributor

@honzasp honzasp commented Jul 25, 2023

This PR introduces and implements version 3 of the Hrana protocol. The changes relative to Hrana 2 are as follows:

  • Added Protobuf encoding in addition to the existing JSON encoding. The encoding is negotiated in the same way as the protocol version.
  • Added cursors. Previously, results from a statement or a batch of statements had to be loaded into memory and serialized into a single large message. Cursors support incremental reading of results (with row granularity) from executing a batch.
  • Added request and batch condition to query the autocommit status. This is necessary to correctly implement transactions in libsql clients, to make sure that if the transaction is implicitly rolled back (for example by using OR ROLLBACK conflict resolution strategy), we won't execute follow up statements in autocommit mode.

These additions stretch the Hrana protocol near its breaking point. At the beginning, Hrana was small and simple protocol, but it has gotten quite bloated: there are two protocols (WebSocket and HTTP), two encodings (JSON and Protobuf) and multiple ways to execute statements (execute request, batch request and cursors). Future versions of Hrana should simplify the protocol.

Fortunately, clients can be simpler, because they need to implement just a single combination of protocol and encoding (typically WebSocket and Protobuf).


I also rewrote the Hrana spec, so Hrana 3 is described in a single document (and not as a set of changes to the previous versions, like Hrana 2).

@honzasp honzasp marked this pull request as ready for review August 17, 2023 11:57
Copy link
Collaborator

@MarinPostma MarinPostma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, few minor nits

let state = self.state.lock().await;
Ok(match *state {
State::Txn => false,
State::Init | State::Invalid => true,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can't assume that auto-commit is true from an invalid state

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was hoping that the invalid state would not occur in practice :) Assuming autocommit is safer, because it is more dangerous to execute a statement that is intended to be in transaction outside of a transaction. (I.e., it is better if the client aborts a transaction if the server reaches the invalid state.)

sqld/src/hrana/cursor.rs Outdated Show resolved Hide resolved
sqld/src/hrana/protobuf.rs Show resolved Hide resolved
@MarinPostma MarinPostma added this pull request to the merge queue Aug 23, 2023
Merged via the queue into libsql:main with commit 10b0591 Aug 23, 2023
7 checks passed
@honzasp honzasp deleted the hrana3 branch August 23, 2023 11:43
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants