Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: merge upstream form v0.4.0 beta10 #4

Closed
wants to merge 33 commits into from

Conversation

zjg555543
Copy link

vcastellm and others added 30 commits October 29, 2024 11:00
* feat: use metadata field on certificate

* fix: lint and UT

* fix: comments
Use input_parser.star from kurtosis
* feat use sqlite on lastgersync

* apply requests

* rm tree migrations

* Update lastgersync/processor.go

Co-authored-by: Goran Rojovic <[email protected]>

---------

Co-authored-by: Goran Rojovic <[email protected]>
* feat use sqlite on claimsponsor

* wip

* pass UTs

* fix identation

* fix identation

* rm cover.out

* rm tree migrations

* make err a var
…ents-for-the-version-command-from-combinations-files

refactor: retrieve and parse versions at buildtime
* wip

* implementation

* fix tests

* wip

* mdbx is gone

* increase coverage

* remove ifElseChain from golangci

* remove ifElseChain from golangci

* remove ifElseChain from golangci

* increase coverage

* increase coverage

* identation

* identation

* identation

* fix kurtosis config
…pessimistic proof branch (0xPolygon#165)

* fix: certificate with no importedBridges set '[]' instead of 'null'

* fix: certificate with no importedBridges set '[]' instead of 'null'

* feat: adapt to kurtosis-cdk pp

* feat: change para SaveCertificatesToFiles to SaveCertificatesToFilesPath

* fix: get candidate and proven certificates as well

* fix: remove test

* fix: small changes

* fix: db tx rollback

* fix: replace existing certificate

* fix: lint and coverage

* feat: check for nil fields in certificate

* feat: no claims test

* fix: comments

* fix: lint

* fix: shallow copy imported bridge exits and bridge exits

* fix: local_config for debug

* fix: cdk-erigon-node-001 rename to cdk-erigon-rpc-001

* feat: add logs to check cert

* feat: store hash as text, add logs

* fix: lint

* fix: bump kurtosis-cdk version to 0.2.18

* fix: comments

* fix: string conversion error on BridgeExit

* fix: lint

* fix: update minter key

* fix: e2e

* fix: e2e tests

---------

Co-authored-by: joanestebanr <[email protected]>
Co-authored-by: Victor Castell <[email protected]>
* feat: created and updated timestamps

* feat: save raw certificate to db

* fix: raw to signed_certificate

* fix: indentation
* chore: bump kustoris
* Adapt to changes in services names
* fix: update minter key
* Apply feedback

Co-authored-by: Stefan Negovanović <[email protected]>
---------
Co-authored-by: Stefan Negovanović <[email protected]>
* fix: var zkevm_path_rw_data is defined in kurtosis/main but not yet on 0.2.8, try to override it
* fix: bump kurtosis 0.2.19 to have the new variable
* feat: sync UpdateL1InfoTreeV2

* fix linter

* use common hash instead of bytes 32

* imporve

* imporve

* imporve

* cover verify trusted aggregator event

* cover halted queries

* rm coverage file

* increase coverage

* moar coverage

* remove files that shouldnt be there

* do not cover smart contracts (generated bindings)

* feat: increase coverage (0xPolygon#159)

* apply pr suggestions

* add context done in handle newblock

* add context done in handle newblock

* add context done in handle newblock

* add context done in handle newblock

---------

Co-authored-by: Joan Esteban <[email protected]>
* feat: unpack and log agglayer errors (0xPolygon#158)

* feat: unpack and log agglayer errors

* feat: agglayer error unpacking

* fix: lint and UT

* feat: epoch notifier (0xPolygon#144)

- Send certificates after a percentage of epoch
- Require epoch configuration to AggLayer
- Change config of `aggsender` adding: `BlockFinality` and `EpochNotificationPercentage`

* refact: GetSequence method (0xPolygon#169)

* feat: remove sanity check (0xPolygon#178) (0xPolygon#179)

---------

Co-authored-by: Goran Rojovic <[email protected]>
Co-authored-by: Rachit Sonthalia <[email protected]>
Co-authored-by: Toni Ramírez <[email protected]>
ToniRamirezM and others added 2 commits November 15, 2024 16:03
* ensure oldAccInputHash is ready

* feat: updata sync lib

* feat: acc input hash sanity check

* feat: check acc input hash -1

* feat: refactor

* feat: refactor

* fix: batch1 acc input hash

* fix: timestamp in input prover

* fix: timestamp in input prover

* fix: timestamp

* feat: remove test

* fix: test

* fix: test

* fix: comments

* fix: comments
elapsedTime := time.Now().UTC().Sub(time.UnixMilli(certificate.CreatedAt))
a.log.Debugf("aggLayerClient.GetCertificateHeader status [%s] of certificate %s elapsed time:%s",
certificateHeader.Status,
certificateHeader.String(),

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information

[Sensitive data returned by an access to Password](1) flows to a logging call.

Copilot Autofix AI 2 days ago

To fix the problem, we need to ensure that sensitive information such as passwords is not logged in clear text. The best way to fix this is to remove the sensitive information from the string representation methods and ensure it is not included in any logging statements.

  1. Modify the String method of the Config struct in aggsender/config.go to exclude the AggsenderPrivateKey.Password field.
  2. Ensure that the String method of the CertificateHeader struct in agglayer/types.go does not include any sensitive information.
  3. Update the logging statements in aggsender/aggsender.go to avoid logging sensitive information.
Suggested changeset 3
aggsender/aggsender.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/aggsender/aggsender.go b/aggsender/aggsender.go
--- a/aggsender/aggsender.go
+++ b/aggsender/aggsender.go
@@ -510,3 +510,3 @@
 			certificateHeader.Status,
-			certificateHeader.String(),
+			certificateHeader.CertificateID.String(),
 			elapsedTime)
@@ -515,3 +515,3 @@
 			a.log.Infof("certificate %s changed status from [%s] to [%s] elapsed time: %s",
-				certificateHeader.String(), certificate.Status, certificateHeader.Status, elapsedTime)
+				certificateHeader.CertificateID.String(), certificate.Status, certificateHeader.Status, elapsedTime)
 
@@ -528,3 +528,3 @@
 			a.log.Infof("certificate %s is still pending, elapsed time:%s ",
-				certificateHeader.String(), elapsedTime)
+				certificateHeader.CertificateID.String(), elapsedTime)
 			thereArePendingCerts = true
EOF
@@ -510,3 +510,3 @@
certificateHeader.Status,
certificateHeader.String(),
certificateHeader.CertificateID.String(),
elapsedTime)
@@ -515,3 +515,3 @@
a.log.Infof("certificate %s changed status from [%s] to [%s] elapsed time: %s",
certificateHeader.String(), certificate.Status, certificateHeader.Status, elapsedTime)
certificateHeader.CertificateID.String(), certificate.Status, certificateHeader.Status, elapsedTime)

@@ -528,3 +528,3 @@
a.log.Infof("certificate %s is still pending, elapsed time:%s ",
certificateHeader.String(), elapsedTime)
certificateHeader.CertificateID.String(), elapsedTime)
thereArePendingCerts = true
agglayer/types.go
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/agglayer/types.go b/agglayer/types.go
--- a/agglayer/types.go
+++ b/agglayer/types.go
@@ -553,9 +553,4 @@
 func (c CertificateHeader) String() string {
-	errors := ""
-	if c.Error != nil {
-		errors = c.Error.String()
-	}
-
-	return fmt.Sprintf("Height: %d, CertificateID: %s, NewLocalExitRoot: %s. Status: %s. Errors: [%s]",
-		c.Height, c.CertificateID.String(), c.NewLocalExitRoot.String(), c.Status.String(), errors)
+	return fmt.Sprintf("Height: %d, CertificateID: %s, NewLocalExitRoot: %s. Status: %s",
+		c.Height, c.CertificateID.String(), c.NewLocalExitRoot.String(), c.Status.String())
 }
EOF
@@ -553,9 +553,4 @@
func (c CertificateHeader) String() string {
errors := ""
if c.Error != nil {
errors = c.Error.String()
}

return fmt.Sprintf("Height: %d, CertificateID: %s, NewLocalExitRoot: %s. Status: %s. Errors: [%s]",
c.Height, c.CertificateID.String(), c.NewLocalExitRoot.String(), c.Status.String(), errors)
return fmt.Sprintf("Height: %d, CertificateID: %s, NewLocalExitRoot: %s. Status: %s",
c.Height, c.CertificateID.String(), c.NewLocalExitRoot.String(), c.Status.String())
}
aggsender/config.go
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/aggsender/config.go b/aggsender/config.go
--- a/aggsender/config.go
+++ b/aggsender/config.go
@@ -40,3 +40,2 @@
 		"AggsenderPrivateKeyPath: " + c.AggsenderPrivateKey.Path + "\n" +
-		"AggsenderPrivateKeyPassword: " + c.AggsenderPrivateKey.Password + "\n" +
 		"URLRPCL2: " + c.URLRPCL2 + "\n" +
EOF
@@ -40,3 +40,2 @@
"AggsenderPrivateKeyPath: " + c.AggsenderPrivateKey.Path + "\n" +
"AggsenderPrivateKeyPassword: " + c.AggsenderPrivateKey.Password + "\n" +
"URLRPCL2: " + c.URLRPCL2 + "\n" +
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
certificate.Status = certificateHeader.Status
if certificateHeader.Status != certificate.Status {
a.log.Infof("certificate %s changed status from [%s] to [%s] elapsed time: %s",
certificateHeader.String(), certificate.Status, certificateHeader.Status, elapsedTime)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information

[Sensitive data returned by an access to Password](1) flows to a logging call.

Copilot Autofix AI 2 days ago

To fix the problem, we need to ensure that sensitive information such as passwords is not logged in clear text. The best way to fix this without changing existing functionality is to remove the logging of the sensitive information or obfuscate it before logging. In this case, we will remove the logging of the AggsenderPrivateKey.Password from the String method in the Config struct.

Suggested changeset 1
aggsender/config.go
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/aggsender/config.go b/aggsender/config.go
--- a/aggsender/config.go
+++ b/aggsender/config.go
@@ -40,3 +40,2 @@
 		"AggsenderPrivateKeyPath: " + c.AggsenderPrivateKey.Path + "\n" +
-		"AggsenderPrivateKeyPassword: " + c.AggsenderPrivateKey.Password + "\n" +
 		"URLRPCL2: " + c.URLRPCL2 + "\n" +
EOF
@@ -40,3 +40,2 @@
"AggsenderPrivateKeyPath: " + c.AggsenderPrivateKey.Path + "\n" +
"AggsenderPrivateKeyPassword: " + c.AggsenderPrivateKey.Password + "\n" +
"URLRPCL2: " + c.URLRPCL2 + "\n" +
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
a.log.Errorf("error updating certificate status in storage: %w", err)
continue
err = fmt.Errorf("error updating certificate %s status in storage: %w", certificateHeader.String(), err)
a.log.Error(err)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information

[Sensitive data returned by an access to Password](1) flows to a logging call.

Copilot Autofix AI 2 days ago

To fix the problem, we need to ensure that sensitive information such as passwords is not logged in clear text. Instead, we can either omit the sensitive information from the logs or obfuscate it. In this case, we will omit the AggsenderPrivateKey.Password from the log output.

  • In the aggsender/config.go file, we will modify the String method of the Config struct to exclude the AggsenderPrivateKey.Password from the returned string.
  • This change will ensure that the password is not logged when the Config struct is converted to a string.
Suggested changeset 1
aggsender/config.go
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/aggsender/config.go b/aggsender/config.go
--- a/aggsender/config.go
+++ b/aggsender/config.go
@@ -40,3 +40,3 @@
 		"AggsenderPrivateKeyPath: " + c.AggsenderPrivateKey.Path + "\n" +
-		"AggsenderPrivateKeyPassword: " + c.AggsenderPrivateKey.Password + "\n" +
+		// "AggsenderPrivateKeyPassword: " + c.AggsenderPrivateKey.Password + "\n" +
 		"URLRPCL2: " + c.URLRPCL2 + "\n" +
EOF
@@ -40,3 +40,3 @@
"AggsenderPrivateKeyPath: " + c.AggsenderPrivateKey.Path + "\n" +
"AggsenderPrivateKeyPassword: " + c.AggsenderPrivateKey.Password + "\n" +
// "AggsenderPrivateKeyPassword: " + c.AggsenderPrivateKey.Password + "\n" +
"URLRPCL2: " + c.URLRPCL2 + "\n" +
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
}
}
if slices.Contains(nonSettledStatuses, certificateHeader.Status) {
a.log.Infof("certificate %s is still pending, elapsed time:%s ",
certificateHeader.String(), elapsedTime)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information

[Sensitive data returned by an access to Password](1) flows to a logging call.

Copilot Autofix AI 2 days ago

To fix the problem, we need to ensure that sensitive information such as passwords is not logged in clear text. The best way to fix this is to modify the String method in aggsender/config.go to exclude the password from the output. This way, when the String method is called and its output is logged, the sensitive information will not be included.

Suggested changeset 1
aggsender/config.go
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/aggsender/config.go b/aggsender/config.go
--- a/aggsender/config.go
+++ b/aggsender/config.go
@@ -40,3 +40,3 @@
 		"AggsenderPrivateKeyPath: " + c.AggsenderPrivateKey.Path + "\n" +
-		"AggsenderPrivateKeyPassword: " + c.AggsenderPrivateKey.Password + "\n" +
+		// "AggsenderPrivateKeyPassword: " + c.AggsenderPrivateKey.Password + "\n" + // Removed for security reasons
 		"URLRPCL2: " + c.URLRPCL2 + "\n" +
EOF
@@ -40,3 +40,3 @@
"AggsenderPrivateKeyPath: " + c.AggsenderPrivateKey.Path + "\n" +
"AggsenderPrivateKeyPassword: " + c.AggsenderPrivateKey.Password + "\n" +
// "AggsenderPrivateKeyPassword: " + c.AggsenderPrivateKey.Password + "\n" + // Removed for security reasons
"URLRPCL2: " + c.URLRPCL2 + "\n" +
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
@zjg555543 zjg555543 changed the title Merge upstream form v0.4.0 beta10 feat: merge upstream form v0.4.0 beta10 Nov 18, 2024
@zjg555543 zjg555543 closed this Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants