feat(61/STATUS-Community-History-Archives): add Status Community History #610
Conversation
There was a problem hiding this comment.
Content LGTM 🚀, but there are a few semantic changes required (not exhaustive) -
- semantic line breaks - https://sembr.org/
- reduction in the use of contractions
- Casing in sub-headings to be the same (Capitalized case? cc: @kaiserd for confirmation on this)
|
|
||
| Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). | ||
|
|
||
| # References |
There was a problem hiding this comment.
Maybe add references to the original spec here? or the implementation?
There was a problem hiding this comment.
This is a tough one. There's no original spec, or are you referring to the PRs that lived in status/specs?
Implementation lives in multiple places in status-go, how would you go about linking to that?
Listing links to lines of code of some commit hash (to ensure it doesn't break with forward changes)?
There was a problem hiding this comment.
Also for the other specs we are porting: the old status spec page is meant to shut down.
Imo, it is OK not to have this link. We can mention, it moved.
For other specs, where we have a previously published spec, we could add a copy (on rfc.vac.dev) to keep as version history for specs that were in draft state (or higher).
(This process still has to be included in the COSS, but has been used here https://rfc.vac.dev/spec/12/ ; see previous version link.)
|
@rymnc thank you so much for this thorough review! |
| 2. Control node requests messages from store nodes for the missed time range for all channels in their community | ||
| 3. All missed messages are stored into control node's local message database | ||
| 4. If 7 or more days have elapsed since the last message history torrent was created, the control node will perform step 6 and 7 of [Serving community history archives](#serving-community-history-archives) for every 7 days worth of messages in the missed time range (e.g. if the node was offline for 30 days, it will create 4 message history archives) | ||
|
|
There was a problem hiding this comment.
If a control node is off for more than 30 days, can it get the info from other control nodes?
There was a problem hiding this comment.
The only thing it can do is try to grab messages from store nodes. If store nodes don't store messages for longer than 30 days, then this is the limit.
So one implicit requirement is that the control node comes online at least once every 30 days (in practice it's much more often though.)
| This is in addition to their database of application messages. | ||
| This is required to provide confidentiality, authenticity, and integrity of message data distributed via the BitTorrent layer, and later validated by community members when they unpack message history archives. | ||
|
|
||
| Control nodes SHOULD remove those messages from their local databases once they are older than 30 days and after they have been turned into message archives and distributed to the BitTorrent network. |
There was a problem hiding this comment.
But they would still seed the archives after that time?
There was a problem hiding this comment.
Yes, they will still seed.
Generally, control nodes always seed one torrent per community. The torrent itself contains the entire data.
If the messages have been removed from the local database, they still exist in the archive data in the torrent.
That's why, in fact, a control node needs to be able to store a message thrice: once as WakuMessage, once as unwrapped application message (so it can be rendered on screen), and once as part of the archive in the torrent.
Only thing to note here is, if the Waku messages are removed from the database, the control node cannot recreate archives for those, unless it's able to recover those messages from store nodes (which is unlikely if the messages are older than 30 days).
Hence, the spec says "SHOULD", as it's more memory efficient, but it comes at a cost of course.
Archives spec
Part of #608