Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is for #540. The core code modification is about 300 lines and many others are generated by thrift.
main points:
1.1 zk add :topology-version and :update-duration-sec fields to StormBase:status map
1.2 zk add :topology-version to executor heartbeat
1.3 worker local state add versions dir to storm worker's current running topology-version
1.4 nimbus add updateTopology interface
1.5 add topology-version field to storm.thrift three struct: TopologySummary ExecutorSummary TopologyInfo
2.1 storm client run "storm jar xxxx -c topology.update=true" to invoke topology update process
2.2 storm client upload new jar file to nimbus
2.3 storm client call nimbus updateTopology interface
2.4 nimbus check the new topology and replace stormdist/storm-id dir
2.5 nimbus update StormBase in zk, set :topology-version(for destination version) and :update-duration-sec(for all workers update process duration) fields to StormBase:status map
2.6 supervisors check zk StormBase and do the update work if topology's local version is not the same with zk version
2.6.1 sync-supervisor download the latest code from nimbus to local stormdist/topology-version dir
2.6.2 each supervisor schedule the topology's worker update at a rand(expect-max-update-time) time point
2.6.3 sync-process check local worker version, if it is not the same with sync-supervisor downloaded version and update time point reached, set worker state to a new :update state
2.6.4 sync-process kill workers in :update state as normally
2.6.5 sync-process restart killed worker as normally, expect that read topology and conf from stormdist/topology-version dir
2.6.6 new worker heartbeat to zk with new topology-version, it can be displayed on web ui to check update progress