-
Notifications
You must be signed in to change notification settings - Fork 0
Home
Documentation page of ElixirBench service. A tour to our features, goals and more!
✔️ Setup a YAML config file to have your benchmark scripts running
✔️ Configure database and other dependencies easily using docker containers syntax like
✔️ See the performance metrics of your project over the time in beautiful and interactive graphs
✔️ Reliable and powerful hardware that runs one benchmark job at a time
✔️ Automatic runs for each push and pull request events
See our getting started guide and learn how to setup your project.
The primary goal of ElixirBench is to improve the Elixir and Erlang ecosystem of open source libraries.
We're maintainers of couple open source packages, and we know how difficult it sometimes is, to guard against performance regressions. While test suites are popular, benchmark suites are not - which is a great shame. Poor performace is a bug like every other yet it is often overlooked.
To solve this problem, we decided to create this project that will help mainainers of libraries in providing consistent performance of their code through constant monitoring.
The project is inspired by similar projects from other ecosystems:
Unlike those projects, though, that are focused on some fixed packages, ElixirBench looks to provide a common service, similar to a continuous integration system, available for all packages published on Hex.
The project consists of several components:
- the API server - powers everything and is responsible for scheduling execution of benchmarks. The server provides a public GraphQL API for exploring the results of the benchmarks and a private JSON API for communication with the runner server.
- the runner server - runs on separate infrastructure and is responsible for consistent execution of benchmarks jobs.
- the runner container - scripts to build docker container which fetches project source and executes benchmarks.
- the front-end - is a website that leverages the GraphQL API and facilitates exploration of the results
Questions, discussions and announcements are made in the #elixir-lang group on Slack! You can join us at the #elixir-bench channel
We aim to deliver soon new features to the Benchmark lovers:
- Create a GitHub marketplace app that would ease integration to the level of most common CI tools;
- More benchmark tools which are easier to use and require less manual setup - maybe some generators;
- Track job status pending/running/finished/failed
- Save/Download docker-compose file from job
- Be able to search repository
- Detect and notify performance regressions and a lot more!