-
Notifications
You must be signed in to change notification settings - Fork 58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BALANCER] Improve balancer-related code execution speed #1544
[BALANCER] Improve balancer-related code execution speed #1544
Conversation
This function is designed for `ClusterLogAllocation`, a data structure been deleted for a while.
This reverts commit 8a70e25.
common/src/main/java/org/astraea/common/balancer/tweakers/ShuffleTweaker.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
整體看起來不錯,幾個小建議請看一下
common/src/main/java/org/astraea/common/balancer/tweakers/ShuffleTweaker.java
Show resolved
Hide resolved
common/src/main/java/org/astraea/common/balancer/tweakers/ShuffleTweaker.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@garyparrot 感謝更新,我覺得就工程面來說 balancer 有越來越簡潔,幾個建議請看一下
common/src/main/java/org/astraea/common/balancer/tweakers/ShuffleTweaker.java
Outdated
Show resolved
Hide resolved
common/src/main/java/org/astraea/common/balancer/tweakers/ShuffleTweaker.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM 做得很好
Context: #1476 (comment), #1541
目前 Balancer 在大量 Partition 下 (約 10000 Partition) 會出現程式碼執行的效能議題,這個 PR 對 Balancer 相關的程式碼做一些實作上的修正,以增進執行速度。
主要的變更包含:
ShuffleTweaker
,現在重複利用一個ClusterInfoBuilder
來建立最後的結果。ClusterInfo.Optimized#replicas()
系列函數使用 Lazy 的 grouping 結果,避免用default
的實作(透過replicaStream
來重新建議一樣的 list)AllocationTweaker
改進前
30 秒進行 963 次嘗試
改進後
30 秒進行 3006 次嘗試