-
Notifications
You must be signed in to change notification settings - Fork 227
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(core): Add support for Tantivy based time series index #1852
Commits on Jul 17, 2024
-
misc(build): Add support for Rust JNI libraries (#1815)
Add build integration to build Rust librarie that can be used for native methods in a module: * Opt in plugin that enables Rust integration * compile, test, lint, package steps * Support for multi-arch build allowing the final jar to support more than one platform No libraries are using this as of this commit, follow ups will add Rust code usage
Configuration menu - View commit details
-
Copy full SHA for 55f521d - Browse repository at this point
Copy the full SHA 55f521dView commit details
Commits on Jul 23, 2024
-
refactor(core): Move common index logic to abstract base class (#1816)
In preparation for new index types separate common logic from Lucene specific logic. No functional changes are made as part of this commit, just moving code around and refactoring to accept multiple index implementations. In most cases this amounts to moving code out of PartKeyLuceneIndex and into PartKeyIndexRaw without further changes. In a few cases things that were instaniating Lucene specific types now use callbacks or other approaches to allow instantiating index specific types.
Configuration menu - View commit details
-
Copy full SHA for 9779a31 - Browse repository at this point
Copy the full SHA 9779a31View commit details
Commits on Aug 1, 2024
-
feat(core): Add open/close support for Tantivy Index (#1817)
Commit 1 of adding Tantivy index support. This PR is broken down into a small subset of the overall Tantivy logic to allow for easier partial review. The index code is not usable end to end until all parts are committed. This adds the basic Rust project skeleton and supports opening, applying schema, and closing the index. Many methods are unimplemented and will be added in follow up PRs. End to end testing is not available in this PR as the index test suite requires ingestion and query support for verification.
Configuration menu - View commit details
-
Copy full SHA for 6148ee4 - Browse repository at this point
Copy the full SHA 6148ee4View commit details
Commits on Aug 2, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 6e45ca3 - Browse repository at this point
Copy the full SHA 6e45ca3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 50b2484 - Browse repository at this point
Copy the full SHA 50b2484View commit details -
feat(core): Add ingestion support for Tantivy
This implements enough of the interface to add documents to the Tantivy index. This is part 2 of the series and is not usable end to end until additional PRs complete. Document additional and removal is covered in this PR. Documents are created by calling a single native method that has many of the built-in fields, such as start and end time, passed as method parameters. Dyanmic fields are passed to the Rust code via a simple binary encoding to reduce JNI overhead.
Configuration menu - View commit details
-
Copy full SHA for b886661 - Browse repository at this point
Copy the full SHA b886661View commit details -
Configuration menu - View commit details
-
Copy full SHA for bf797b5 - Browse repository at this point
Copy the full SHA bf797b5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 572ac53 - Browse repository at this point
Copy the full SHA 572ac53View commit details
Commits on Aug 13, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 2b90c68 - Browse repository at this point
Copy the full SHA 2b90c68View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4cc2606 - Browse repository at this point
Copy the full SHA 4cc2606View commit details
Commits on Aug 14, 2024
-
feat(core): Add ingestion support for Tantivy #1827
This implements enough of the interface to add documents to the Tantivy index. This is part 2 of the series and is not usable end to end until additional PRs complete. Document additional and removal is covered in this PR. Documents are created by calling a single native method that has many of the built-in fields, such as start and end time, passed as method parameters. Dyanmic fields are passed to the Rust code via a simple binary encoding to reduce JNI overhead. **Pull Request checklist** - [X] The commit(s) message(s) follows the contribution [guidelines](CONTRIBUTING.md) ? - [ ] Tests for the changes have been added (for bug fixes / features) ? - [ ] Docs have been added / updated (for bug fixes / features) ? **New behavior :** Part 2 of Tantivy index support. The functionality is still incomplete and not ready for usage yet.
Configuration menu - View commit details
-
Copy full SHA for 4ef88b0 - Browse repository at this point
Copy the full SHA 4ef88b0View commit details
Commits on Aug 26, 2024
-
feat(core): Add Rust side of Tantivy query support (#1835)
This contains the Rust code needed to support all query patterns for FiloDB. This is part 3 of the series and is not usuable end to end until additional PRs complete. This PR splits the logic into a common tantivy library, where collectors and query extensions are implemented, and the filodb library, where specific JVM bridge methods and business logic exists. The majority of this PR are custom collectors and caching to better match the performance of Lucene in the FiloDB use cases. The glue logic to connect these collectors to JVM methods are for the most part very straight forward - parse a query, run it, serialize results.
Configuration menu - View commit details
-
Copy full SHA for 8dfe6f2 - Browse repository at this point
Copy the full SHA 8dfe6f2View commit details
Commits on Sep 6, 2024
-
feat(core): Add Scala side of Tantivy query support (#1843)
This contains the Scala side of query support. This completes the basic end to end flow, so it also includes regression and performance tests. There will be some small cleanup PRs after this for things like additional metrics, but with this PR the Tantivy index can be successfully activated via config.
Configuration menu - View commit details
-
Copy full SHA for 38de69d - Browse repository at this point
Copy the full SHA 38de69dView commit details
Commits on Sep 10, 2024
-
feat(core): Add tantivy metrics (#1845)
Add two new metrics for commit processing time and cache hit rates. These will be used for monitoring as we roll out and turn the feature on for testing.
Configuration menu - View commit details
-
Copy full SHA for a5e1e50 - Browse repository at this point
Copy the full SHA a5e1e50View commit details -
Configuration menu - View commit details
-
Copy full SHA for 54bf773 - Browse repository at this point
Copy the full SHA 54bf773View commit details -
Configuration menu - View commit details
-
Copy full SHA for ffe8818 - Browse repository at this point
Copy the full SHA ffe8818View commit details
Commits on Sep 11, 2024
-
Merge pull request #1847 from rfairfax/rfairfax/feat-index-rust-merge…
…-9-10 feat(core): Merge latest develop
Configuration menu - View commit details
-
Copy full SHA for faaa9c0 - Browse repository at this point
Copy the full SHA faaa9c0View commit details -
feat(core): Optimize Rust code by default (#1848)
Previously the code only optimized if a flag was passed during sbt invocation or if we're building multi-arch. This has created confusing results where perf drifts because debug mode was used. After doing testing and development it was found that release is a suitable default - it does incremental build, includes symbols for debugging, and while not as fast as debug builds is still fast enough for an inner loop. You can still build with debug by setting rust.optimize=false for any cases where it is needed during local development.
Configuration menu - View commit details
-
Copy full SHA for c21e4ff - Browse repository at this point
Copy the full SHA c21e4ffView commit details