Create nodes synchronization without input and outputs depency #816
acnazarejr
started this conversation in
Idea
Replies: 1 comment 1 reply
-
I believe this is not (easily) possible without forcing the order of execution by defining dependency between the two nodes with a dataset. Note that you don't have to make this dataset the actual real table in question; you can make a new dummy one: # catalog.yml
dummy_dataset:
type: MemoryDataSet And then define your nodes: node(node_A, inputs, outputs=["dummy_dataset"])
node(node_B, inputs=["dummy_dataset"], outputs) |
Beta Was this translation helpful? Give feedback.
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I have two nodes. The first (NODE A) creates a table on my data warehouse and the second (NODE B) consumes this table. However, the first node (A) outputs isn't an input for the second node (B).
So, it's possible to force the NODE B to wait for the NODE A execution to finish, when I use the ParallelRunner?
Beta Was this translation helpful? Give feedback.
All reactions