API Versioning TRG

[tobzahn]

Note: This discussion relates to the pull request TRG API Versioning #526. The pull request has been updated after the first round of discussion in order to include the feedback.

Overview

The proposed TRG adds a rule that all (with some exceptions) APIs should be versioned by adding the version information of the API into the URL.
Furthermore, the TRG makes some recommendations on how to deal with versioning and related aspects.

Why

This TRG has been proposed in order to have alignment across Tractus-X, have good Developer experience for outside contributions and usage, easy / fast onboarding to have more traction.

As backward compatibility is required by the association in the Life Cycle Management 101 Whitepaper, therfore this overall guardrail needs to be translated to actual technical requirements.
Versioning APIs helps to transitions between major versions as you can continue to offer old API versions for a period of time. But even if the old API version is no longer supported, having versioned APIs helps to avoid endpoints beeing bothered with invalid requests from old clients and gives those client proper error messages in order to identify the issue.

If an API is not versioned, the clients might assume that the API is not going to include breaking changes and might therefore not be prepared. For this reason, it is more difficult to introduce a breaking change into an unversioned API. Therfore, it is industry best practice to version APIs from the very begining and this practice is followed by the TRG.

Why in the URL

The TRG requires the version information to be included into the URL. Alternatively, versioning could also be done using the HTTP header. However, the TRG has opted for URL-based versioning for the following reasons:

• the Version number can be clearly identified by everyone in the URL, while it is less obvious in the header
• some firewalls and proxy servers change the header, therefore header-based versioning might not work for all organizations
• some proprietary or legacy systems are not able to send customer headers, therefore header-based versioning might not work for all organizations
• for clients, URL-based versioning is easier to implement
• URL-based versioning is easier to test, e.g. in a browser, as setting customer headers is not required

Why not optional

In the previous discussion on the TRG it was discussed if the TRG could make recommendations instead of a rule.
I prefere a rule instead of a recommendation for the following reasons:

• backward-compatibilty is required by the association and API versioning contributes to that. I personaly do not see a possibility to follow the guardrail of the association while not versioning APIs, other than never introducing breaking changes - which is something I do not see as an realistic option
• even if no breaking changes are planned or anticipated for an API, that might change in the future, therefore we should be prepared
• some stakeholders (e.g. product owners) have asked for clear guidance, as that is more easy to follow
• having versioned and unversioned APIs would worsen the developer experience, would make it more difficult to understand and explain

That said, I understand that if we implement protocols defined elsewhere (e.g. DSP), the regulations might be conflicting. In the latest version of the TRG, I have therefore included a statement to explain what should and what shouldn't be covered by TRG. For example, I completly understand that the TRG should not apply to DSP and IATP, both need to come up with an own solution to tackle the issue of compatibility.

Why this discussion

This TRG has been discussed on the tractusx-dev mailinglist and in an office hour before. Since no agreement could be reached, it was decided to create this discussion in order to give more people the chance to participate in the discussion. So please feel free to share your thoughts.

[paullatzelsperger]

Versioning stuff (not just APIs) is a best practice, no-one will seriously debate that. And versioning is not limited to APIs. To be crystal-clear: versioning is good and everyone should do it. But while I agree on the merits of versioning per se, I strongly oppose making this a rule (and would likely -1 such a TRG-pull-request) for the following reasons.

1. how an API's URL is constructed is an implementation detail that is up to the engineering team and has no place in a TRG.
2. developers must read the API documentation anyway, and at that point its irrelevant what the specific base URL is. Dev experience is not influenced by the specific URL, but much more so by other things like REST-ful APIs, sensible endpoints, easily digestible DTOs, etc.
3. While the requirement of backwards compatibility is totally valid and understood, the way to achieve that must be left up to the engineering teams. It's a fundamental principle of requirements engineering to not prescribe implementation details.
4. Entities that require guidance are invited to stick to the recommendation.

Besides, arguments 4 and 5 of "Why in the URL" are invalid, as we are talking about machine-consumable APIs, which cannot be tested in a browser. Further, every developer will confirm that setting a header in code is no more difficult than setting a URL.

[tobzahn]

Hello Paul,
thanks for your reply and for participating in the discussion.
To 1.: I disagree that the URL is an implementation detail. At least not, if the API is offered for everybody to use. I can agree for internal APIs not to be used outside a specific component.
2. Having different APIs handle versioning in a similar manner adds to the developer experience from my point of view, as it makes different components on Tractus-X feel "similar". However, I am thinking about different Catena-X use cases here, so from the view of a developer who implements say DCM and PCF.
3. Please help me understand - would you say that the TRG therefore should just recommend to use URL versioning, but not enforce it? I think I could agree to that, if we still would recommend URL versioning and explain why (see my first post) - then developers could choose other ways after consideration.
4. Other people tell me that they struggle with the versioning and compatibility topic, as the rules and guidelines are not clear enough -while you explicitly say that versioning is good, others seem to not see it as important at all. Which is why I tried to give less room for (miss)interpretation. That was my intention, at least.

So, could you agree to a TRG that makes API versioning mandatory and only recommending URL versioning? Also the TRG would only apply for APIs to be used by others (that was always intended but maybe not clear enough).