Add nova 3 cell DT #401

jamepark4 · 2024-09-17T20:38:22Z

Standard deployments currently deploy cell0 and cell1. Create a DT that deploys cell0, cell1, and cell2.

openshift-ci · 2024-09-17T20:38:26Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

openshift-ci · 2024-09-17T20:38:28Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jamepark4
Once this PR has been reviewed and has the lgtm label, please assign leifmadsen for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

hjensas · 2024-10-07T20:52:34Z

examples/dt/nova/nova-multiplecells/values.yaml

+    config.kubernetes.io/local-config: "true"
+
+data:
+  ssh_keys:


I am not a fan of hacing ssh_keys set in two values files in this PR. The nodeset in the library will create the same secret dataplane-ansible-ssh-private-key-secret for both nodesets. If a user looks at this they may be led to believe they can use different keys for the (I assume pre-provisioned nodes?) when using this DT - but that will not work since applying the second nodeset will modify the same secret instead of creating a new one.

Ack I can removed these.

I am not sure you can remove them, if you cannot - then this change should not be blocked by this comment from me.

I was trying to point out an issue with the library not supporting multiple nodesets in a good way. Essentilly you have two nodesets referencing lib/dataplane/nodeset/kustomization.yaml#L6-L8 - so you end up applying two separate files defining the same secret resource.

gibizer

I have couple of question / comments inline.

gibizer · 2024-11-13T08:07:56Z

examples/dt/nova/nova-three-cells/control-plane/service-values.yaml

+          cellMessageBusInstance: rabbitmq-cell2
+          conductorServiceTemplate:
+            replicas: 3
+          hasAPIAccess: true


Just a note for the future. We can spice this setup up by defining cell2 without api DB access (i.e. no upcall support).

examples/dt/nova/nova-three-cells/control-plane/service-values.yaml

gibizer · 2024-11-13T08:10:50Z

examples/dt/nova/nova-three-cells/edpm/nodeset2/values.yaml

+      - ovn
+      - neutron-metadata
+      - libvirt
+      - nova


wondering how this connects the nodes to cell2 if this still refers to the default nova service.

right, that won't work, please see my WIP for adoption, perhaps you may find somthing to reuse https://github.com/bogdando/data-plane-adoption/blob/f61537f9701f6585183810b20e2142423ae4f8d0/docs_user/modules/proc_adopting-compute-services-to-the-data-plane.adoc#L237 , then how we use it in the services list https://github.com/bogdando/data-plane-adoption/blob/f61537f9701f6585183810b20e2142423ae4f8d0/docs_user/modules/proc_adopting-compute-services-to-the-data-plane.adoc#L381

It's replaced here: https://github.com/openstack-k8s-operators/architecture/pull/401/files#diff-20af9459981c1a0588160942c3490649f5cf4fd222526cea08c3244208fb48ecR19. But I can just define it directly in the nodeset.

oh indeed. replacing via kustomize works for me

sorry, yeah kustomize is OK to me too.

gibizer · 2024-11-13T08:13:20Z

examples/dt/nova/nova-three-cells/README.md

+| Role              | Machine Type | Count |
+| ----------------- | ------------ | ----- |
+| Compact OpenShift | vm           |   3   |
+| OpenStack Compute | vm           |   4   |


Does this DT really deploys 4 EDPM nodes as computes? I only see edpm-compute-0 and edpm-compute-1 referenced in the NodeSets.

Because this is completely virtualized its unfortunately not very clear based on purely this DT. This DT works in conjunction with templates in CIFMW [1] that splits the total computes between each nodeset. It also needs a scenario file similar to [2] to define the number of VMs needed (the one I'm using for testing this is defined downstream). Virtualized setups now have unique hostnames that are recorded during setup and that information is what [1] uses to allocate how the hosts are split. With baremetal setups where I can define the hostnames in the jobs, I can specifically define how the EDPM Hosts are allocated per nodeset within the context of the DT.

[1] https://github.com/openstack-k8s-operators/ci-framework/pull/2423/files
[2] https://github.com/openstack-k8s-operators/ci-framework/blob/main/scenarios/reproducers/va-nfv-ovs-sriov.yml

gibizer · 2024-11-13T08:15:00Z

examples/dt/nova/nova-three-cells/README.md

+(cell1 and cell2) in addition to cell0. Computes will be allocated to both 
+cells


Would be nice to describe the mapping. Which compute will be connected to which cell.

examples/dt/nova/nova-three-cells/README.md

bogdando · 2024-11-13T13:55:12Z

examples/dt/nova/nova-three-cells/README.md

+  containerImageFields:
+  - NovaComputeImage
+  - EdpmIscsidImage
+  dataSources:


here we should add that nova-cell<N>-metadata-neutron-config, if running metadata service in a cell instead of top scope

ack I'll try for per cell deployment.

bogdando · 2024-11-13T14:01:47Z

examples/dt/nova/nova-three-cells/values.yaml

+      - ovn
+      - neutron-metadata
+      - libvirt
+      - nova


ditto, use nova-<cell> in <cell> specific nodesets

perhaps this is also replaced via kustomize?

I was only updating cell2 and keeping the default for cell1 so this isn't replaced, but if it improves overall clarity I can update this and kustomize to have nova-cell-1 and nova-cell-2 respectively instead.

abays · 2024-11-18T13:20:24Z

@jamepark4 Would you mind addressing the errors reported in CI when you have a chance?

Standard deployments currently deploy cell0 and cell1. Create a DT that deploys cell0, cell1, and cell2.

softwarefactory-project-zuul · 2024-11-18T15:15:34Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/dfe0fa56e83c475db9c67b866bab6145

✔️ noop SUCCESS in 0s
❌ rhoso-architecture-validate-nova-three-cells FAILURE in 3m 30s

openshift-ci bot added the do-not-merge/work-in-progress label Sep 17, 2024

jamepark4 force-pushed the add_multicell_dt branch 9 times, most recently from 364ebe9 to 38d7c4b Compare September 24, 2024 18:34

jamepark4 force-pushed the add_multicell_dt branch 6 times, most recently from 19005c6 to 0f33d8c Compare October 2, 2024 13:44

hjensas reviewed Oct 7, 2024

View reviewed changes

jamepark4 force-pushed the add_multicell_dt branch from 0f33d8c to 9cadc05 Compare October 8, 2024 17:55

hjensas mentioned this pull request Oct 8, 2024

Introduce DCN DT #392

Merged

jamepark4 force-pushed the add_multicell_dt branch 5 times, most recently from 5bcd2c7 to 4e22901 Compare October 11, 2024 13:24

jamepark4 force-pushed the add_multicell_dt branch from 4e22901 to 01ef803 Compare October 23, 2024 16:54

jamepark4 marked this pull request as ready for review November 12, 2024 20:24

openshift-ci bot removed the do-not-merge/work-in-progress label Nov 12, 2024

openshift-ci bot requested a review from karelyatin November 12, 2024 20:24

openshift-ci bot requested a review from raukadah November 12, 2024 20:24

jamepark4 requested review from gibizer and bogdando November 12, 2024 20:25

jamepark4 changed the title ~~Add multi-cell DT~~ Add nova 3 cell DT Nov 12, 2024

jamepark4 force-pushed the add_multicell_dt branch from 4df3930 to 28677c1 Compare November 12, 2024 22:08

gibizer requested changes Nov 13, 2024

View reviewed changes

openshift-ci bot assigned gibizer Nov 13, 2024

bogdando reviewed Nov 13, 2024

View reviewed changes

examples/dt/nova/nova-three-cells/README.md Show resolved Hide resolved

bogdando reviewed Nov 13, 2024

View reviewed changes

jamepark4 force-pushed the add_multicell_dt branch from 28677c1 to e1cff34 Compare November 18, 2024 15:02

Add nova 3 cell DT

9d16f03

Standard deployments currently deploy cell0 and cell1. Create a DT that deploys cell0, cell1, and cell2.

jamepark4 force-pushed the add_multicell_dt branch from e1cff34 to 9d16f03 Compare November 18, 2024 15:11

jamepark4 mentioned this pull request Nov 18, 2024

Add nova multiplecells nodeset templates openstack-k8s-operators/ci-framework#2423

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add nova 3 cell DT #401

Add nova 3 cell DT #401

jamepark4 commented Sep 17, 2024 •

edited

Loading

openshift-ci bot commented Sep 17, 2024

openshift-ci bot commented Sep 17, 2024

hjensas Oct 7, 2024

jamepark4 Oct 7, 2024

hjensas Oct 8, 2024

gibizer left a comment

gibizer Nov 13, 2024

gibizer Nov 13, 2024

bogdando Nov 13, 2024

jamepark4 Nov 13, 2024

bogdando Nov 13, 2024

gibizer Nov 14, 2024

gibizer Nov 13, 2024

jamepark4 Nov 13, 2024 •

edited

Loading

gibizer Nov 13, 2024

jamepark4 Nov 13, 2024

bogdando Nov 13, 2024

jamepark4 Nov 13, 2024

bogdando Nov 13, 2024

bogdando Nov 13, 2024

jamepark4 Nov 13, 2024

abays commented Nov 18, 2024

softwarefactory-project-zuul bot commented Nov 18, 2024

		(cell1 and cell2) in addition to cell0. Computes will be allocated to both
		cells

Add nova 3 cell DT #401

Are you sure you want to change the base?

Add nova 3 cell DT #401

Conversation

jamepark4 commented Sep 17, 2024 • edited Loading

openshift-ci bot commented Sep 17, 2024

openshift-ci bot commented Sep 17, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gibizer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jamepark4 Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abays commented Nov 18, 2024

softwarefactory-project-zuul bot commented Nov 18, 2024

jamepark4 commented Sep 17, 2024 •

edited

Loading

jamepark4 Nov 13, 2024 •

edited

Loading