Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove calls to ElasticReadOnlyCommand and disable EsReadOnlyMonitorJob #30468

Open
dsilvam opened this issue Oct 28, 2024 · 6 comments · Fixed by #30471 or #30639
Open

Remove calls to ElasticReadOnlyCommand and disable EsReadOnlyMonitorJob #30468

dsilvam opened this issue Oct 28, 2024 · 6 comments · Fixed by #30471 or #30639

Comments

@dsilvam
Copy link
Contributor

dsilvam commented Oct 28, 2024

Parent Issue

No response

Problem Statement

Both the job and the command had their purpose when the dotCMS setup included an embedded ES and the disks would fill up. Now that dotCMS uses a managed ES, these command and job are not needed and in fact they are failing on our servers because of lacking of permissions to get the cluster settings.

[22/10/24 17:00:00:392 GMT] ERROR core.JobRunShell: Job dotcms_jobs.EsReadOnlyMonitorJob threw an unhandled Exception: 
org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=security_exception, reason=no permissions for [cluster:monitor/term] and User [name=corpsites-es, backend_roles=[], requestedTenant=null]]
	at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:187) ~[?:?]
	at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1911) ~[?:?]
	at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1888) ~[?:?]
	at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1645) ~[?:?]
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1602) ~[?:?]
	at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1572) ~[?:?]
	at org.elasticsearch.client.ClusterClient.getSettings(ClusterClient.java:98) ~[?:?]
	at com.dotcms.content.elasticsearch.business.ElasticsearchUtil.getClusterSettings(ElasticsearchUtil.java:158) ~[?:?]
	at com.dotcms.content.elasticsearch.business.ElasticsearchUtil.getClusterUpdateInterval(ElasticsearchUtil.java:136) ~[?:?]
	at com.dotcms.content.elasticsearch.business.ElasticReadOnlyCommandImpl.loadTimeToWaitAfterWriteModeSet(ElasticReadOnlyCommandImpl.java:118) ~[?:?]
	at com.dotcms.content.elasticsearch.business.ElasticReadOnlyCommandImpl.executeCheck(ElasticReadOnlyCommandImpl.java:87) ~[?:?]
	at com.dotmarketing.quartz.job.EsReadOnlyMonitorJob.execute(EsReadOnlyMonitorJob.java:20) ~[?:?]

Steps to Reproduce

Reproducible from different actions:

  • When the job runs every five minutes
  • When checkin in content

Acceptance Criteria

The error doesn't happen again upon any action on the CMS, in particular when saving content.
The EsReadOnlyMonitorJob does not run and it's not present in the list of system jobs from the maintenance tab.

dotCMS Version

trunk_cad5532

Proposed Objective

Core Features

Proposed Priority

Priority 2 - Important

External Links... Slack Conversations, Support Tickets, Figma Designs, etc.

No response

Assumptions & Initiation Needs

No response

Quality Assurance Notes & Workarounds

No response

Sub-Tasks & Estimates

No response

@dsilvam dsilvam moved this from New to Current Sprint Backlog in dotCMS - Product Planning Oct 28, 2024
@freddyDOTCMS freddyDOTCMS self-assigned this Oct 28, 2024
@freddyDOTCMS freddyDOTCMS moved this from Current Sprint Backlog to In Progress in dotCMS - Product Planning Oct 28, 2024
freddyDOTCMS added a commit that referenced this issue Oct 28, 2024
@jcastro-dotcms
Copy link
Contributor

INTERNAL QA: PASSED

  • Docker image: trunk_7408571

The code using the ElasticReadOnlyCommand class has been removed. If there's no easy way for you to generate an ES read only status in your local, you check the following:

  • Go to the Roles & Tools portlet, and add the Job Scheduler portlet.
  • This is the data table BEFORE the fix form this ticket -- notice the presence of the EsReadOnlyMonitorJob class:
before
  • And this is the data table AFTER the fix:
after

@jcastro-dotcms jcastro-dotcms moved this from Internal QA to QA - Backlog in dotCMS - Product Planning Nov 1, 2024
@josemejias11
Copy link
Contributor

Approved: Tested on trunk_0fe6d13, Docker, macOS 14.5, FF v126.0.1

@dsilvam
Copy link
Contributor Author

dsilvam commented Nov 12, 2024

Failing on the headless dotCMS server with the following stacktrace:
https://gist.github.com/dsilvam/395de332cb5b9cbde95452561afff41b

@dsilvam dsilvam moved this from Done to Current Sprint Backlog in dotCMS - Product Planning Nov 12, 2024
@freddyDOTCMS freddyDOTCMS self-assigned this Nov 12, 2024
@freddyDOTCMS freddyDOTCMS moved this from Current Sprint Backlog to In Progress in dotCMS - Product Planning Nov 12, 2024
@dsilvam dsilvam moved this from In Progress to Internal QA in dotCMS - Product Planning Nov 13, 2024
@dsilvam
Copy link
Contributor Author

dsilvam commented Nov 14, 2024

Passed IQA: Feedback on both local docker instance and the headless dotcms.com

@dsilvam dsilvam assigned dsilvam and unassigned freddyDOTCMS Nov 14, 2024
@dsilvam dsilvam moved this from Internal QA to QA - Backlog in dotCMS - Product Planning Nov 14, 2024
@josemejias11
Copy link
Contributor

Approved: Tested on trunk_0dfb0a3, Docker, macOS 14.5, FF v126.0.1

@josemejias11 josemejias11 moved this from QA - In Progress to Done in dotCMS - Product Planning Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment