Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PMM-13426: Fix PBM collector memory leak #928

Merged
merged 8 commits into from
Oct 11, 2024

Conversation

idoqo
Copy link
Contributor

@idoqo idoqo commented Oct 9, 2024

PMM-13426 (optional, if ticket reported)

Closes #922 and #924


Below we provide a basic checklist of things that would make it a good PR:

  • Make sure to sign the CLA (Contributor License Agreement).
  • Make sure all tests pass.
  • Keep current with the target branch and fix conflicts if necessary.
  • Update jira ticket description if necessary.
  • Attach screenshots and/or console output to the jira ticket to confirm new behavior, if applicable.
  • Leave notes to the reviewers if you need to focus their attention on something specific.

Once all checks pass and the code is ready for review, please add pmm-review-exporters team as the reviewer. That would assign people from the review team automatically. Report any issues on our Forum.

@idoqo idoqo requested a review from a team as a code owner October 9, 2024 20:15
@idoqo idoqo requested review from BupycHuk and JiriCtvrtka and removed request for a team October 9, 2024 20:15
Comment on lines 252 to 259
pbmc, err := newPbmCollector(ctx, client, e.opts.URI, e.opts.Logger)
if err != nil {
e.logger.Errorf("Cannot create PBM collector: %v, collector will now be disabled.", err)
e.opts.EnablePBMMetrics = false
requestOpts.EnablePBMMetrics = false
} else {
registry.MustRegister(pbmc)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we create these objects on each request, so it won't help much

Comment on lines 56 to 83
func newPbmCollector(ctx context.Context, client *mongo.Client, mongoURI string, logger *logrus.Logger) (*pbmCollector, error) {
// we can't get details of other cluster from PBM if directConnection is set to true,
// we re-write it if that option is set (e.g from PMM).
if strings.Contains(mongoURI, "directConnection=true") {
mongoURI = strings.ReplaceAll(mongoURI, "directConnection=true", "directConnection=false")
}
pbmClient, err := sdk.NewClient(ctx, mongoURI)
if err != nil {
logger.Errorf("failed to initialize PBM client from uri %s: %s", mongoURI, err.Error())
return nil, err
}

defer func() {
err := pbmClient.Close(ctx)
if err != nil {
logger.Errorf("failed to close PBM client: %v", err)
}
}()
_, err = pbmClient.GetConfig(ctx)
if err != nil {
logger.Errorf("failed to get PBM configuration during initialization: %s", err.Error())
return nil, err
}
return &pbmCollector{
ctx: ctx,
mongoURI: mongoURI,
base: newBaseCollector(client, logger.WithFields(logrus.Fields{"collector": "pbm"})),
}
}, nil
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's revert it because of message above

@@ -112,7 +95,7 @@ func (p *pbmCollector) collect(ch chan<- prometheus.Metric) {

pbmConfig, err := pbmClient.GetConfig(p.ctx)
if err != nil {
logger.Errorf("failed to get PBM configuration: %s", err.Error())
logger.Warnf("failed to get PBM configuration: %s", err.Error())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be Info level if we expect that pbm config might not exist there

@idoqo idoqo requested a review from BupycHuk October 10, 2024 09:44
@BupycHuk BupycHuk merged commit 686b313 into main Oct 11, 2024
7 checks passed
@BupycHuk BupycHuk deleted the fix-pbm-collector-memory-leak branch October 11, 2024 09:28
BupycHuk pushed a commit that referenced this pull request Oct 11, 2024
* add benchmark

* disable pbm collector if there's an error during init

* drop benchmark code

* close pbm client

* format imports

* drop error log

* drop log level to info

(cherry picked from commit 686b313)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0.41.1 Connections didn't closed
3 participants