Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sumac Sandbox & Release Preparation #1121

Open
49 tasks done
DawoudSheraz opened this issue Sep 13, 2024 · 33 comments
Open
49 tasks done

Sumac Sandbox & Release Preparation #1121

DawoudSheraz opened this issue Sep 13, 2024 · 33 comments

Comments

@DawoudSheraz
Copy link
Contributor

DawoudSheraz commented Sep 13, 2024 •
edited
Loading

This issue is for tracking the items to prepare https://github.com/overhangio/openedx-release-demo for Sumac cutoff.

Sumac cutoff is expected to take place on October 23, 2024. Once the branches are out on upstream, the community will be looking forward to sandbox provided by Edly. tutor and its plugins will require sumac branches with appropriate changes for the sandbox to complete. The necessary information is already present on https://edlyio.atlassian.net/wiki/spaces/openedx/pages/3013148726/Building+Open+edX+major+releases. A note: once the branches are created, any changes to nightly would need to be manually added to the sumac branch. Therefore, a rebase on nightly whenever nightly changes would be required to ensure the sumac branch is always up-to-date

This issue is meant for tracking the statuses of various repositories needed for sandbox. It will act as a todo list to ensure that tutor and its plugins have sumac branches created from nightly before the cutoff date (even though there will be potential failures on CI against it) and have a PR against master.
For sanity checks, make sure to run image build and init task on local tutor <local/dev> do init --limit=pluginnname.

The sandbox script expects the branches to be available on the main repository, not the fork. If you are assigned a repository but you cannot push to it directly, please get in touch with @DawoudSheraz or the maintainer of the repository with patch file so that they can create the PR in the appropriate repository with your provided changes

Tutor & Plugins Todo List

Available on launch

Available within a week

Not needed for sandbox

Sandbox Repo Todo List

Context

Every six months, Tutor maintainers sync up with the Build/Test/Release working group to create the next version, both of Open edX and Tutor.

In the scenario below, we are upgrading Open edX from fictional “Delta” to “Epsilon”. Tutor will upgrade from v4 to v5 (“d”=4, “e”=5).

(these instructions are pulled and adapted from this old discussion topic)

Create release branches

In Tutor core and all plugins, we must create “epsilon” branches off of the nightly branches. For each repo, this looks like the following:

git checkout nightly

git pull

git checkout -b epsilon

Push the created “epsilon” branch to the upstream repo: git push

Upgrade Tutor core and plugins

  • Modify about.py:
    • bump the version number to 5.0.0.
  • in Tutor core, set the version_suffix to an empty string.
  • In plugins’ setup.py, bump the version of the “tutor” package that the plugin depends on:
install_requires=["tutor>=5.0.0,<6.0.0"],
extras_require={"dev": ["tutor[dev]>=5.0.0,<6.0.0"]},
  • Create a changelog entry with make changelog-entry: "💥[Feature] Upgrade to Epsilon. (by @YourGithubUsername)"
  • Replace all instances of “[dD]elta” that make sense (i.e: not in the CHANGELOG.md!)
  • Go through the Dockerfile templates and manually upgrade all the 3rd-party requirements that you can find (dockerize, ipdb, etc.). Sometimes it’s not desirable to upgrade some pieces of software: for instance node and python are expected to run a specific version. Those required version numbers are specified in the edx/configuration 5 repo. (look for SERVICENAME_VERSION variables)
  • Make sure that the plugin images (if any) are built correctly: tutor images build all.
  • Make sure that the plugin can be correctly initialized: tutor local do init --limit=pluginnname.
  • If you need to make some changes to the docker-compose*.yml files and patches, make sure to backport these changes to the k8s-* patches, for compatibility with Kubernetes.
  • Make sure that the plugins work correctly by doing some basic usage testing. Pay close attention to any log (warning or error) that come out of the plugin containers as well as the lms container.
  • git commit -a -m v5.0.0
  • Push the created “epsilon” branch to the upstream repo: git push

The release branches should be updated regularly to take into account the latest changes from the nightly branch. During the release process, it is frequent that changes are pushed to the master and nightly branches, and the latest “espilon” branch must have those changes as well. Be prepared to push --force the “espsilon” branches frequently.

⚠️ Plugin tests will fail in the release branches, because the plugins can’t find tutor>=5.0.0 on pypi. To resolve that, we should update the test scripts to install tutor from source (pip install -e --config-settings editable_mode=compat https://github.com/overhangio/tutor@epsilon#egg=tutor). But it’s inconvenient to do that for each and every plugin. So maybe we should consider migrating to a centralized “tutor-test-plugin” GitHub action. This action would be versioned according to the Tutor major version (@5).
@DawoudSheraz DawoudSheraz changed the title Sumac Preparation - Nightly Build Testing Sumac Sandbox Preparation Oct 11, 2024
@DawoudSheraz
Copy link
Contributor Author

@jfavellar90 Hi, letting you know since you are the maintainer for notes and xqueue.

@ghassanmas Hi, letting you know since you are the maintainer for forum.

@MoisesGSalas Hi. I am not sure if you are the contact person for codejail but since you did it last time for redwood, I have assigned the sumac to you. Let me know if this is not the case.

Thank you all. Let me know if you have any questions

@MoisesGSalas
Copy link
Contributor

Hi @DawoudSheraz, that's right, I will take care of the codejail plugin.

@DawoudSheraz
Copy link
Contributor Author

Hi @DawoudSheraz, that's right, I will take care of the codejail plugin.

Awesome, thank you.

@jfavellar90
Copy link

@DawoudSheraz I'm taking care of notes and xqueue :)

@DawoudSheraz
Copy link
Contributor Author

@DawoudSheraz I'm taking care of notes and xqueue :)

Thank you.

@angonz
Copy link
Contributor

angonz commented Oct 21, 2024

Hi!
There are a number of tests for Aspects. Shall we include the aspects plugin in the list? Who is in charge?

@DawoudSheraz
Copy link
Contributor Author

Hi! There are a number of tests for Aspects. Shall we include the aspects plugin in the list? Who is in charge?

Hi, I am in touch with BTR to get a confirmation on this. Thanks

@angonz
Copy link
Contributor

angonz commented Oct 21, 2024

Hi @bmtcril! Can you please check if the aspects plugin is ready for the summac sandbox?

@bmtcril
Copy link
Contributor

bmtcril commented Oct 21, 2024

@angonz @DawoudSheraz we're working on the last couple of Aspects bug fixes, but expect to have a release ready in the next day or so.

@hinakhadim
Copy link
Contributor

Tutor-mfe and tutor-indigo Sumac PRs are ready. I've tested them on local and sandboxdev using tutor Sumac PR (with nightly)

@Faraz32123
Copy link
Contributor

Faraz32123 commented Oct 23, 2024 •
edited
Loading

tutor-discovery sumac PR is ready.

  • Is nightly local image build working?
  • Is init working?

tutor-ecommerce sumac PR is ready.

  • Is nightly local image build working?
  • Is init working?

tutor-credentials sumac PR is ready.

  • Is nightly local image build working?
  • Is init working?

@bmtcril
Copy link
Contributor

bmtcril commented Oct 23, 2024

@angonz @DawoudSheraz Aspects v1.2.0 is on PyPI and ready for testing, thanks for your patience!

@crathbun428
Copy link

crathbun428 commented Oct 23, 2024 •
edited
Loading

@angonz @DawoudSheraz - Linking the configuration information for Libraries for the testing sandbox:here

Thank you!

@jfavellar90
Copy link

jfavellar90 commented Oct 24, 2024 •
edited
Loading

tutor-notes sumac PR is ready

  • Is nightly local image build working?
  • Is init working?

tutor-xqueue sumac PR is ready

  • Is nightly local image build working?
  • Is init working?

@DawoudSheraz
Copy link
Contributor Author

@angonz @DawoudSheraz Aspects v1.2.0 is on PyPI and ready for testing, thanks for your patience!

Cool. I will update sandbox script to point to v1.2.0 tag for contrib-aspects. Thanks

@DawoudSheraz
Copy link
Contributor Author

@angonz @DawoudSheraz - Linking the configuration information for Libraries for the testing sandbox:here

Thank you!

Hi. meilisearch related PRs are not merged yet, they are under review. I have not looked at the entire linked issue but it seems we need meilisearch before enabling the flags. @kdmccormick Is this correct?

@ghassanmas
Copy link
Member

ghassanmas commented Oct 24, 2024 •
edited
Loading

Tutor forum PRs are ready I have tested (v1) with discussion MFE.

Note: The discussion MFE and forum cs_comments_service would crash when responding to a thread; if your disk is almost full, this because Elasticsearch blocks writing in such case.

@MoisesGSalas
Copy link
Contributor

Tutor contrib codejail: eduNEXT/tutor-contrib-codejail#59, still wip but is working.

  • Is nightly local image build working?
  • Is init working?

@DawoudSheraz
Copy link
Contributor Author

Tutor forum PRs are ready I have tested (v1) with discussion MFE.

Note: The discussion MFE and forum cs_comments_service would crash when responding to a thread; if your disk is almost full, this because Elasticsearch blocks writing in such case.

Thanks

@Faraz32123
Copy link
Contributor

tutor-minio sumac PR is ready.

  • Is init working?

@Danyal-Faheem
Copy link
Contributor

tutor-cairn sumac PR is ready.

  • Is nightly local image build working?
  • Is init working?

@angonz
Copy link
Contributor

angonz commented Oct 25, 2024

Hi team! Can we confirm that the basic sandbox is running, so we can start with the tests?

@angonz
Copy link
Contributor

angonz commented Oct 25, 2024

The announcement is in the forum.

@Abdul-Muqadim-Arbisoft
Copy link
Contributor

tutor-jupyter sumac PR is ready.

  • Is nightly local image build working?
  • Is init working?

tutor-webui sumac PR is ready.

  • Is nightly local image build working?
  • Is init working?

@angonz
Copy link
Contributor

angonz commented Oct 29, 2024

Hi @DawoudSheraz,
I got some complaints that testers don't receive emails. There is a reported issue with some email providers, probably due to AWS not allowing port 25 out of the server.
Fayyaz Ahmed reported that emails from arbisoft.com don't work either. CC @mariajgrimaldi who also worked on that. Can we review this? Which SMTP server is using the sandbox?

@DawoudSheraz
Copy link
Contributor Author

Hi @DawoudSheraz, I got some complaints that testers don't receive emails. There is a reported issue with some email providers, probably due to AWS not allowing port 25 out of the server. Fayyaz Ahmed reported that emails from arbisoft.com don't work either. CC @mariajgrimaldi who also worked on that. Can we review this? Which SMTP server is using the sandbox?

Hello. This happened with redwood as well. Port 25 is blocked on the instance, this is by default by AWS. We did not update any spam-related records because the sandbox is public and anyone can send emails from that (by sign up, activation, etc.). It risks impacting the domain authority.
In Redwood testing, it was decided to update testing instructions to highlight using self hosted email accounts for getting emails openedx/wg-build-test-release#350 (comment).

@regisb Please add any context that I might have missed.

@mariajgrimaldi
Copy link
Contributor

mariajgrimaldi commented Oct 29, 2024 •
edited
Loading

@angonz @DawoudSheraz: As mentioned in the thread, emails won't work for email providers like Google, Yahoo, etc. I'll try with the protonmail account that I used during the Redwood release; if it doesn't work, I'll use a colleague with a self-hosted email account to test.

@Abdul-Muqadim-Arbisoft
Copy link
Contributor

tutor-android sumac PR is ready.

  • Is nightly local image build working?
  • Is init working?

@DawoudSheraz
Copy link
Contributor Author

@angonz @DawoudSheraz: As mentioned in the thread, emails won't work for email providers like Google, Yahoo, etc. I'll try with the protonmail account that I used during the Redwood release; if it doesn't work, I'll use a colleague with a self-hosted email account to test.

@mariajgrimaldi Hi, can you verify this again? It should be set now.

@mariajgrimaldi
Copy link
Contributor

mariajgrimaldi commented Oct 30, 2024 •
edited
Loading

@DawoudSheraz it works with my protomain account! Thank you.

FYI @angonz

@jmakowski1123
Copy link

The new Libraries feature is not showing up in the sandbox - these are the configuration requirements, please let me know if I can help. openedx/frontend-app-authoring#1334

@DawoudSheraz
Copy link
Contributor Author

The new Libraries feature is not showing up in the sandbox - these are the configuration requirements, please let me know if I can help. openedx/frontend-app-authoring#1334

meilisearch PRs are not part of sumac branches yet.

@DawoudSheraz
Copy link
Contributor Author

Some updates on sandbox:

  • Meilisearch PRs have been merged and deployed as part of today's sandbox re-run. Libraries v2 interface is now accessible on Studio. cc: @jmakowski1123 @crathbun428
  • tutor-forum has been disabled temporarily because of some merge conflicts issues of forum_v2 branch temp: disable forum openedx-release-demo#65. Once those issues are resolved, tutor-forum will be re-enabled. cc: @angonz

Let me know if there are any questions or concerns.

@DawoudSheraz DawoudSheraz changed the title Sumac Sandbox Preparation Sumac Sandbox & Release Preparation Nov 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
Tutor project management
Status: In Progress
Milestone
No milestone
Development

No branches or pull requests