Fixing various security issues and updating null_types to handle over… #407

safaci2000 · 2024-10-08T14:22:50Z

Adding gosec and lint scanning to project

safaci2000 · 2024-10-09T16:58:23Z

@go-jet do you have any insights on this? I'm not very familiar with circleCI, but it seems like all the tests are passing, the tests tab is all clear, the report has no failures I can find. Though it seems something is still breaking.

.circleci/config.yml

qrm/internal/null_types.go

qrm/internal/null_types_test.go

qrm/internal/null_types.go

internal/testutils/test_utils.go

go.mod

go-jet · 2024-10-10T18:46:29Z

@go-jet do you have any insights on this? I'm not very familiar with circleCI, but it seems like all the tests are passing, the tests tab is all clear, the report has no failures I can find. Though it seems something is still breaking.

Hmm, it is weird. Indeed, all tests pass. A wield guess would be that after all the tests have finished, TestMain' panics at some point. But only profile delay.Start().Stop()` is executed afterwards.

Maybe add some log lines and panic catcher in TestMain.
If that fails, then I guess start removing the changes one by one until we find out exactly which change is breaking the tests.

safaci2000 · 2024-10-10T20:13:58Z

@go-jet do you have any insights on this? I'm not very familiar with circleCI, but it seems like all the tests are passing, the tests tab is all clear, the report has no failures I can find. Though it seems something is still breaking.

Hmm, it is weird. Indeed, all tests pass. A wield guess would be that after all the tests have finished, TestMain' panics at some point. But only profile delay.Start().Stop()` is executed afterwards.

Maybe add some log lines and panic catcher in TestMain. If that fails, then I guess start removing the changes one by one until we find out exactly which change is breaking the tests.

The issue is with go-junit-report. I replaced with a very similar tool that I think is a bit easier to use. It also, actually gives you a break down as the tests are running without having to navigate to the tests tab.

Let me know if you have any objections to using this instead, otherwise I'll try to figure out how to make it go-junit-report work.

Also, how would you feel about moving the circle CI tests into a github action ?

go-jet · 2024-10-11T10:49:35Z

tests/mysql/main_test.go

@@ -33,7 +30,6 @@ func sourceIsMariaDB() bool {
 }

 func TestMain(m *testing.M) {
-	rand.Seed(time.Now().Unix())


It's deprecated and no longer needed, from go docs.

// Deprecated: As of Go 1.20 there is no reason to call Seed with // a random value. Programs that call Seed with a known value to get // a specific sequence of results should use New(NewSource(seed)) to // obtain a local random generator.

Aha, I see.

go-jet · 2024-10-11T11:09:02Z

@go-jet do you have any insights on this? I'm not very familiar with circleCI, but it seems like all the tests are passing, the tests tab is all clear, the report has no failures I can find. Though it seems something is still breaking.

Hmm, it is weird. Indeed, all tests pass. A wield guess would be that after all the tests have finished, TestMain' panics at some point. But only profile delay.Start().Stop()is executed afterwards. Maybe add some log lines and panic catcher inTestMain`. If that fails, then I guess start removing the changes one by one until we find out exactly which change is breaking the tests.

The issue is with go-junit-report. I replaced with a very similar tool that I think is a bit easier to use. It also, actually gives you a break down as the tests are running without having to navigate to the tests tab.

Let me know if you have any objections to using this instead, otherwise I'll try to figure out how to make it go-junit-report work.

Also, how would you feel about moving the circle CI tests into a github action ?

What was the issue with go-junit-report?

I'm fine with gotestsum, but I don't see codecoverage on codecov now.

I'm fine with github actions as well, if they provide the same as circle.ci.

safaci2000 · 2024-10-11T11:19:59Z

I'm fine with gotestsum, but I don't see codecoverage on codecov now.

I'm fine with github actions as well, if they provide the same as circle.ci.

Okay, let me see if I can get the junit report to work, or fix gotestsum. They should be compatible... let me see what I can do with that.

Not 100% sure of the issue, but it required an update to V2 and it still didn't work for me. I can't replicate it locally so need to use the CICD to test the behavior.

safaci2000 · 2024-10-11T11:21:21Z

I'm fine with github actions as well, if they provide the same as circle.ci.

Okay, I'll do another MR without any other changes being added for that later on if I can manage something that's comparable.

safaci2000 · 2024-10-11T12:11:13Z

Okay, done. Fixed the code coverage issue. I created a token for my own project and verified the upload. https://app.codecov.io/gh/safaci2000/jet/tree/feature%2Fscanners

go-jet · 2024-10-11T17:21:44Z

Okay, done. Fixed the code coverage issue. I created a token for my own project and verified the upload. https://app.codecov.io/gh/safaci2000/jet/tree/feature%2Fscanners

Something ain't right. Code coverage should be above 90%. Also the last two steps store_artifacts and store_test_results did not upload anything.

safaci2000 · 2024-10-14T13:24:19Z

Okay, done. Fixed the code coverage issue. I created a token for my own project and verified the upload. https://app.codecov.io/gh/safaci2000/jet/tree/feature%2Fscanners

Something ain't right. Code coverage should be above 90%. Also the last two steps store_artifacts and store_test_results did not upload anything.

Noted, I'll look at it when i have a chance.

ChangeLog: - Adding gosec linting - Adding static type to enum - fixing nulltype overflow - Trying out gotestsum as an alternative to go-junit-report.xml

safaci2000 · 2024-10-14T15:08:59Z

I had to bump the go version used for testing a bit further than go.mod due to a change in behavior. golang/go#65653.

Code coverage should be resolved

go-jet · 2024-10-15T09:54:52Z

I had to bump the go version used for testing a bit further than go.mod due to a change in behavior. golang/go#65653.

Interesting, now it makes sense.

.circleci/config.yml

.github/workflows/code_scanner.yml

go.mod

go-jet · 2024-10-16T12:37:03Z

Thanks. Looks good. 👍

safaci2000 force-pushed the feature/scanners branch 3 times, most recently from 087b6c6 to 396f988 Compare October 8, 2024 14:27

safaci2000 marked this pull request as draft October 8, 2024 15:16

safaci2000 force-pushed the feature/scanners branch from 8a4348e to 63cb493 Compare October 9, 2024 16:12