Roadmap

Performance & Scalability

• Ability to execute arbitrarily large workflows
  • Workflows with a large number of tasks will be handled seamlessly
  • There will be no performance bottlenecks/impact on the system
• Efficiently scale the system to handle large number of concurrent workflow executions
  • Decider queue implementation can elastically handle load
• Robust and scalable event handling framework
  • Provide at-least-once guarantees for event processing
  • Event handlers can elastically handle load
  • Event handlers can be scaled independently of other components in the system

Usability

• Provide a managed tasks experience
  • Conductor manages the task lifecycle
  • Scaling of resources for such tasks will be handled by Conductor
  • System will provide pluggable interfaces for integrating with external systems such as serverless functions (e.g. AWS Lambda).
• Deprecate and discontinue support for DynoQueues and Dynomite
  • Implement a Redis based QueueDAO
  • Full support for all Redis configurations
• Improved developer experience
  • Provide support for debug mode (breakpoints, input/output changes) workflow execution
  • Improved workflow authoring
  • Workflow as code - allow the developers to implement dynamic workflows with code (Java to start with)
  • New UI that provides revamped experience for users to configure, manage, monitor and debug workflows, with improved performance and usability.
• Tenant isolation
  • Noisy neighbors should not affect system SLO
  • Configurable resource limits per tenant
• Type safety
  • Schematized payloads for workflow and tasks
  • Validation for workflow and task payloads with configurable actions on validation failure
• Granular notifications
  • Declaratively embed notifications
  • Notifications for task state transitions
• Event registry
  • Ability to multiplex events to effect workflow/task actions
  • Provide an event store with a configurable TTL

Security

• Pluggable authentication and authorization
  • Ability to enable/disable AuthN and AuthZ
  • Configurable rules

Testing

• Workflow dry runs
  • Able to evaluate a workflow definition without running tasks
• Provide toolings to aid local development and automated testing of workflows and workers