Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Protobuf reorder #281

Merged
merged 3 commits into from
Aug 1, 2023
Merged

Protobuf reorder #281

merged 3 commits into from
Aug 1, 2023

Conversation

sfc-gh-pchu
Copy link
Collaborator

@sfc-gh-pchu sfc-gh-pchu commented Jul 31, 2023

Current issue description:

In this PR: #271. We change the proto definition for PackageInfo which has impact on package list action.
Old definition

message PackageInfo {
  string name = 1;
  string version = 2;
  string repo = 3;
}

New definition

message PackageInfo {
  string name = 1;
  uint32 epoch = 2;
  string version = 3;
  string release = 4;
  string architecture = 5;
  string repo = 6;
}

When the sansshell server is still running with version older than v1.22.0, and sanssh client with version newer or equal to v1.22.0, the server is still returning the old message but the client is expecting the new one which makes following mapping

Server Client
name name
version epoch
repo version

Then in client, we will format the result and print here

for _, pkg := range r.Resp.Packages {

Current issue display

So when running the package list action $./sanssh --targets=localhost packages list, we will see the difference

// wrong result
Installed Packages
                           GeoIP.x86_64.           @base-
                     NessusAgent.x86_64. @sfc_external_software7-
                          PyYAML.x86_64.       installed-
                             acl.x86_64.           @base-
                             apr.x86_64.           @base-
                        apr-util.x86_64.           @base-
// expected result
Installed Packages
                             GeoIP.x86_64     1.5.0-14.el7                            @base
                      NessusAgent.x86_64       10.3.1-es7          @sfc_external_software7
                           PyYAML.x86_64      3.10-11.el7                        installed
                              acl.x86_64    2.2.51-15.el7                            @base
                              apr.x86_64      1.4.8-7.el7                            @base
                         apr-util.x86_64      1.5.2-6.el7                            @base

Resolution:

  • reorder the PackageInfo message definition to make them have right mapping
Server Client
name name
version version
repo repo
  • make adjustments to client side formatting to avoid trailing dot or dash
    With older version sansshell, the pkg.Architecture and pkg.Release will be empty, and we shouldn't print trailing . and -.

Output with above situation(newer sanssh + older sansshell)

./sanssh --targets=localhost packages list | tail -n 10
                              yum.noarch 3.4.3-168.el7.centos                        @anaconda
                    yum-langpacks.noarch      0.4.2-7.el7                        @anaconda
             yum-metadata-parser.aarch64     1.1.4-10.el7                        @anaconda
         yum-plugin-fastestmirror.noarch  1.1.31-54.el7_8                        @anaconda
                        yum-utils.noarch  1.1.31-54.el7_8                        @anaconda
                          zenity.aarch64   3.28.1-2.el7_9                         @updates
                             zip.aarch64       3.0-11.el7                        @anaconda
                            zlib.aarch64   1.2.7-21.el7_9                         @updates
                      zlib-devel.aarch64   1.2.7-21.el7_9                         @updates
                            zstd.aarch64      1.4.2-1.el7                            @epel

@sfc-gh-pchu sfc-gh-pchu marked this pull request as ready for review July 31, 2023 22:45
@sfc-gh-elinardi
Copy link
Collaborator

@sfc-gh-pchu could you add some more details of the root cause, and why the fix is effective?
Please also add test cases & results proving that this change actually works in all cases, e.g. new version of client against new version of server, new version of client against old version of server.

Copy link
Collaborator

@sfc-gh-elinardi sfc-gh-elinardi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but I'd love to get a review from @sfc-gh-srhodes who's more knowledgeable about protobuf.

@sfc-gh-pchu
Copy link
Collaborator Author

Yes, let me request @sfc-gh-srhodes to review.

Copy link
Collaborator

@sfc-gh-srhodes sfc-gh-srhodes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oof, I missed this mixup. We should get this in as soon as possible.

I highly recommend reading through https://protobuf.dev/programming-guides/dos-donts/ whenever you're figuring out evolution of a proto file.

@@ -646,6 +646,7 @@ func TestListInstalled(t *testing.T) {
testdataInputBad2 := "./testdata/yum-installed-bad2.out"
testdataInputBad3 := "./testdata/yum-installed-bad3.out"
testdataGolden := "./testdata/yum-installed.textproto"
testdataOldVersion := "./testdata/yum-installed-old-server-version.textproto"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Textprotos aren't very effective for this sort of test (https://protobuf.dev/programming-guides/dos-donts/#dont-use-text-format-messages-for-interchange). If you want to do a compatibility test, the testdata should be the binary proto encoding so that we're testing the on-the-wire format.

A better long-term option is adding a separate lint for protobuf changes, something like https://github.com/nilslice/protolock. We can do that as a separate PR.

@sfc-gh-pchu sfc-gh-pchu merged commit d298020 into main Aug 1, 2023
3 checks passed
@sfc-gh-pchu sfc-gh-pchu deleted the protobuf-reorder branch August 1, 2023 23:28
sfc-gh-srhodes added a commit that referenced this pull request Sep 26, 2023
We've had a couple recent cases (#327, #281) where we've accidentally made backwards incompatible changes to our protobufs. This seems preventable by adding in automated checks.
 
I'm using https://github.com/bufbuild/buf-breaking-action and checking against main. I considered adding buf linting as well, but the linting detects tons of preexisting stylistic choices so it didn't seem worth doing so.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants