Prod Release 18/06/24 #806
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
morgsmccauley
commented
Jun 18, 2024
morgsmccauley
commented
- refactor: Merge sync logic to provide fine-grained life-cycle control (refactor: Merge sync logic to provide fine-grained life-cycle control #762)
- chore: Add more logs to Coordinator (chore: Add more logs to Coordinator #798)
- feat: Implement Block Height Stream (feat: Implement Block Height Stream #776)
- add CI to frontend (feat: add CI to frontend (lint ,test, typecheck, build) #799)
…#762) To synchronise Block Streams/Executors we compare the latest registry with the rest of the system. Components are started/stopped as needed, but we lack control over life-cycle events such as registration/deletion. As we build towards Coordinator managing provisioning, we need a way to hook in to these life-cycle events, so we can provision/de-provision accordingly. This PR refactors the current synchronisation logic to provide greater control over these life-cycle events. Existing Block Stream/Executor sync logic have been merged to to a single `Synchronisation` struct, which now manages new/existing/deleted indexers as a whole, providing the required life-cycle hooks described above. Merging the sync logic allows us to control _when_ Block Streams/Executors are started/stopped, allowing us to do things before and after, specifically, de-/provisioning. I wanted to do this refactor first to make reviewing easier, I'll follow this up with the provisioning changes next. To achieve the above, each Indexer now has it's own `IndexerState` object in Redis. After synchronising a newly registered Indexer we write its state object, and after removing it, we delete the state object. This means we have the following states, allowing us to handle each accordingly: - Non-existent State/Existing Config - New Indexer - Existing State/Exisiting Config - Existing Indexer - Exisiting State/Non-existent Config - Deleted Indexer The synchronisation logic is essentially the same, expect that now it is driven off the persistent Redis state, rather than the current Block Stream/Executors state.
I've implemented a struct which is capable of returning an async stream which yields block heights matching a particular contract filter starting from a specified block height. This should serve as a replacement for the existing method of getting block heights by querying Delta Lake through the Delta Lake Client. Some differences between Delta Lake and this new stream is that it yields a SINGLE block height rather than all block heights. In addition, we now query the bitmap indexer's tables for block heights for one day, instead of for all days. This should improve time to get a block as well as reduce the memory footprint.
Added CI.yml for lint, typecheck, build, Test is currently commented (as it will give a false pos since we recreate the functions as oppose to importing them) but will be added in a later PR once we find a compatible setup with jest. Other files are linted with ts-config, prettier, and eslintrc. manually Removed dead files.
|
Want to get #762 migration out before merging follow ups :) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.