Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure deterministic graph calculation with consistent layer, node, and edge ordering in Kedro-Viz #2185

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

rashidakanchwala
Copy link
Contributor

@rashidakanchwala rashidakanchwala commented Nov 12, 2024

Description

Resolves #2057.

Based on @astrojuanlu's comment, I realized that Kedro-Viz might be causing the randomness. To investigate, I checked if the inputs to graph calculations—nodes, edges, and layout—were consistent. While the order of nodes and layout remained stable, the order of edges varied with each backend data request. To address this, we now sort the layers, the edges and nodes to ensure consistency.

Update -
As @ravi-kumar-pilla noted, there was an issue with layer ordering: changing a node name could alter the layer order, especially for layers with identical dependencies, like model_input and feature in the example he shared. This issue stemmed from the layer_dependencies dictionary in services/layers.py, where layers with the same dependencies weren’t consistently ordered. To fix this, I added alphabetical sorting for layers with identical dependencies to ensure stability in toposort.

For nodes and edges, I now sort them immediately upon loading from the backend API, ensuring they are consistently ordered in the Redux state.

For testing, I initially considered using Cypress/backend e2e testing with screenshot comparison of the flowchart, but it proved too complex. Instead, I created a new mock dataset, reordered_spaceflights, with reordered nodes and edges. I added tests in normalise-data-test and actions/graph-test. The first test verifies that nodes and edges are consistently sorted by their ids, regardless of backend order. The second test compares the x and y coordinates in the flowchart, confirming that the graph layout is the same between the two mocks.

QA notes

  • make build
  • pip install -e package
  • kedro viz --autoreload

Make changes to the files, doc changes, and see the layout is consistent.

Checklist

  • Read the contributing guidelines
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added new entries to the RELEASE.md file
  • Added tests to cover my changes

Signed-off-by: rashidakanchwala <[email protected]>
Signed-off-by: rashidakanchwala <[email protected]>
rashidakanchwala and others added 6 commits November 12, 2024 13:08
Signed-off-by: rashidakanchwala <[email protected]>
Signed-off-by: rashidakanchwala <[email protected]>
Signed-off-by: rashidakanchwala <[email protected]>
Signed-off-by: rashidakanchwala <[email protected]>
Signed-off-by: rashidakanchwala <[email protected]>
@jitu5
Copy link
Contributor

jitu5 commented Nov 12, 2024

@rashidakanchwala I tested with both your branch and main, I made the changes roughly 5 to 7 times and took screenshot every time and compared, it look exactly same on your branch and on main its different every time.

Its working for me.

Copy link
Contributor

@jitu5 jitu5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rashidakanchwala Really cool, Not easy to fix such bugs. 🎉

@ravi-kumar-pilla
Copy link
Contributor

Hi @rashidakanchwala , The fix looks promising. I tried without expanding modular pipelines and the nodes seem to position similarly. However, upon expanding modular pipelines, there seems to be some random positioning like below -

Run 1 -

image

Run 2 -

image

Observation: create_feature_importance node has changed its position, so are the layers model_input and feature

Steps to reproduce:

  1. Follow QA steps
  2. Expand all modular pipelines, observe layout of feature_engineering
  3. Change node name of apply_types_to_companies to apply_types_to_companies_test inside data_ingestion
  4. Observe the layout with the change

Some questions:

  1. Can we try sorting nodes and layers as well for consistency ?
  2. Is there any overhead or lag you observed due to sorting as graphNew is an expensive operation ?
  3. Do we think of adding any tests to be sure of the positioning ?

Thank you

Signed-off-by: rashidakanchwala <[email protected]>
Signed-off-by: rashidakanchwala <[email protected]>
Signed-off-by: rashidakanchwala <[email protected]>
Signed-off-by: rashidakanchwala <[email protected]>
@rashidakanchwala rashidakanchwala changed the title Ensure deterministic graph calculation with consistent edge ordering in Kedro-Viz Ensure deterministic graph calculation with consistent layer, node, and edge ordering in Kedro-Viz Nov 14, 2024
Signed-off-by: rashidakanchwala <[email protected]>
Copy link
Contributor

@ravi-kumar-pilla ravi-kumar-pilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really good @rashidakanchwala , tested few times and the order seems the same. Great work !! 💯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Spike: Try to reproduce non-deterministic rendering
4 participants