Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

projects: add ORAS proposal #68

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ https://groups.google.com/a/opencontainers.org/forum/#!forum/tob (tob@opencontai
* [SELinux](proposals/selinux.md)
* [Tools](proposals/tools.md)
* [umoci](proposals/umoci.md)
* [ORAS](proposals/oras.md)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdolitsky if you could rebase on master to pick up the conflict in this list (umoci was merged) that would be good so this is mergeable when necessary.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@estesp done


## Voting

Expand Down
214 changes: 214 additions & 0 deletions proposals/oras.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,214 @@
# OCI ORAS Project Proposal #

## Abstract ##

In order to provide OCI end users a method to publish and retrieve any type of content to/from an OCI registry, as well as a reference implementation for the OCI Distribution Specification, the [ORAS project][oras-project] should be moved under the opencontainers GitHub org.

[oras-project]: https://github.com/deislabs/oras

### ORAS Details ###

jdolitsky marked this conversation as resolved.
Show resolved Hide resolved
ORAS is a CLI that can publish arbitrary content to an OCI registry, with special features for setting mediatypes on manifest configs and on content.
samuelkarp marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ORAS is a CLI that can publish arbitrary content to an OCI registry, with special features for setting mediatypes on manifest configs and on content.

ORAS is both a library for inclusion in other artifact CLIs and CLI that can be used directly to publish arbitrary content to an OCI registry. ORAS provides features for setting manifest.config.mediaType, to identify the artifact type, and layer.mediaType to identify the content.


Note: the manifest mediatype itself is always `application/vnd.oci.image.manifest.v1+json`.

Example - uploading rockets, a brand new type of package:

```
# Create a thing
printf '🚀' > rocket.txt

# Create a manifest config
printf '{"RocketVersion":"v0.1.0"}' > rocket-config.json

# Upload your thing with a custom mediatype
oras push localhost:5000/mystuff/myrocket:v0.1.0 rocket.txt:text/plain \
--manifest-config rocket-config.json:application/vnd.acme.rocket.config.v1+json
```

See manifest created:

```
$ curl -s -H 'Accept: application/vnd.oci.image.manifest.v1+json' \
http://localhost:5000/v2/mystuff/myrocket/manifests/v0.1.0 | jq
{
"schemaVersion": 2,
"config": {
"mediaType": "application/vnd.acme.rocket.config.v1+json",
"digest": "sha256:310175f34d2d4d5cba3418be06ddd1ef948147d729516d78318ec7f5c2d83d49",
"size": 26
},
"layers": [
{
"mediaType": "text/plain",
"digest": "sha256:ebbc0b2870eb323f2b6cffa5c493ceef81ae7eb36afc73d4e0367301631daec5",
"size": 4,
"annotations": {
"org.opencontainers.image.title": "rocket.txt"
samuelkarp marked this conversation as resolved.
Show resolved Hide resolved
}
}
]
}
```

Get that thing:

```
$ curl -s http://localhost:5000/v2/mystuff/myrocket/blobs/sha256:ebbc0b2870eb323f2b6cffa5c493ceef81ae7eb36afc73d4e0367301631daec5
🚀
```

jdolitsky marked this conversation as resolved.
Show resolved Hide resolved
#### Additional Usage ####

ORAS is built primarily on top of Go packages provided by [containerd][containerd-project], but it also imports packages from the [docker/cli][dockercli-project], which enables "docker-style" auth login:

```
oras login -u username -p password localhost:5000 -c rocket-creds.json
```

There are also public Go packages available to build on top of ORAS. The following is the equivalent of the rocket example with the CLI above, but in Go:

```go
package main

import (
"context"
"fmt"

"github.com/containerd/containerd/remotes/docker"
"github.com/deislabs/oras/pkg/content"
"github.com/deislabs/oras/pkg/oras"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)

func main() {
ctx := context.Background()
resolver := docker.NewResolver(docker.ResolverOptions{})
samuelkarp marked this conversation as resolved.
Show resolved Hide resolved
store := content.NewMemoryStore()

registryRootURL := "localhost:5000"
registryNamespace := "mystuff/myrocket"

rocketVersion := "v0.1.0"
rocketFileName := "rocket.txt"
rocketMediaType := "text/plain"
rocketContent := []byte("🚀")
rocketDescriptor := store.Add(rocketFileName, rocketMediaType, rocketContent)

rocketConfigMediaType := "application/vnd.acme.rocket.config.v1+json"
rocketConfigContent := []byte(fmt.Sprintf("{\"RocketVersion\":\"%s\"}", rocketVersion))
rocketConfigDescriptor := store.Add("", rocketConfigMediaType, rocketConfigContent)

ref := fmt.Sprintf("%s/%s:%s", registryRootURL, registryNamespace, rocketVersion)
_, err := oras.Push(ctx, resolver, ref, store, []ocispec.Descriptor{rocketDescriptor},
oras.WithConfig(rocketConfigDescriptor))
if err != nil {
panic(err)
}

fmt.Println("Pushed to", ref)
fmt.Printf("\nTry:\n\ncurl -s -H 'Accept: application/vnd.oci.image.manifest.v1+json' \\\n" +
" %s/v2/%s/manifests/%s | jq\n", registryRootURL, registryNamespace, rocketVersion)
}
```

You can see all features in the project [README][oras-readme].

[containerd-project]: https://github.com/containerd/containerd
[dockercli-project]: https://github.com/docker/cli
[oras-readme]: https://github.com/deislabs/oras/blob/master/README.md

## Proposal ##
Change the ownership of the existing ORAS project from deislabs:

https://github.com/deislabs/oras

And move it inside the `opencontainers` organization:

https://github.com/opencontainers/oras

The import paths will correspondingly be "github.com/opencontainers/oras" (oras does have some Go API users, but since the project will be renamed -- and GitHub will add a redirect -- there will be no significant downstream impact of the change).

### Initial Maintainers ###
Initial maintainers of the ORAS project would be:
SteveLasker marked this conversation as resolved.
Show resolved Hide resolved

* Josh Dolitsky <[email protected]> (@jdolitsky)
* Shiwei Zhang <[email protected]> (@shizhMSFT)
* Sajay Antony <[email protected]> (@sajayantony)
* Steve Lasker <[email protected]> (@stevelasker)
* Jimmy Zelinskie <[email protected]> (@jzelinskie)
* Vincent Batts <[email protected]> (@vbatts)

### Code of Conduct ###
This project would incorporate (by reference) the OCI [Code of Conduct][code-of-conduct].

[code-of-conduct]: https://github.com/opencontainers/org/blob/master/CODE_OF_CONDUCT.md

### Governance and Releases ###
This project would incorporate the Governance and Releases processes from the [OCI project template][oci-project-template].

It should be noted that since ORAS is not a specification, it is not bound by the ordinary quorum and voting rules for specification release.
As such, new versions will be released as regularly as needed without the need for a quorum vote.

Pull requests will require two (2) reviews (signified by "LGTM") from project maintainers.
Maintainers are not allowed to review a pull request which they authored.

[oci-project-template]: https://github.com/opencontainers/project-template

### Project Communications ###
The proposed project would continue to use existing channels in use by the OCI developer community for communication including:

* GitHub for issues and pull requests.
* The [`[email protected]`][oci-ml] email list.
* The weekly OCI developer community conference call.
* The `#opencontainers` IRC channel on Freenode.
* The [OCI Slack workspace][oci-slack].
* The [OCI Matrix Room][oci-matrix].

[oci-ml]: mailto:[email protected]
[oci-slack]: https://opencontainers.slack.com/
[oci-matrix]: https://matrix.to/#/#opencontainers:matrix.org

jdolitsky marked this conversation as resolved.
Show resolved Hide resolved
## Frequently Asked Questions (FAQ)
> *Does this proposal change the OCI Charter?*

This proposal does not in any way intend to amend the [OCI Charter][oci-charter].

[oci-charter]: https://github.com/opencontainers/tob/blob/master/CHARTER.md

> *Where does ORAS fit into the OCI suite of projects?*

ORAS is intended to be a *reference implementation of the OCI Distribution Specification*.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really hard for me to wrap my head around. As far as I can understand, containerd's dockerResolver seems to be the actual OCI distribution client that is being used as part of ORAS.

It seems to me that ORAS is more of a library and tool for enabling artifact-related workflows with an OCI distribution.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I sort of agree here.. I was going off of Aleksa's suggestion.

Any issue marking this as a library vs. reference implementation? Does marking it one way or another have any effect? @cyphar @SteveLasker

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there is a distinction. From my point of view, ORAS cannot be argued to be a library because it isn't small enough nor is it merely a tool to assist the implementation of OCI specifications. It is (ostensibly) a (reference) implementation of the distribution-spec that happens to have a bunch of other features. You could argue that the dockerResolver of containerd is also a (reference) implementation of the distribution-spec. Not to mention it has a CLI, but I don't want to give the impression that that is the sticking point.

You could argue we need another category for "things that aren't reference implementations" but I think that would dilute the mission of the OCI too far. If you both feel that ORAS could be argued as a library then you should comment on #76 (and eventually #86) to express your disagreement with the current text of both documents.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ORAS was created as a library to support additional artifacts with oci-distribution. Pushing pulling content, creating manifests, .tar-ing up contents isn't trivial for the average person wanting to focus on their unique artifact type and benefit from a registry.

Sure, it happens to implement the spec, but I don't know if I'd call it a reference implementation as mentioned, it does use other libraries as well. We didn't think it needed to duplicate what functionality already existed.

So, happy to see the text changed to be a library, intended for artifact usage.
I'd suggest we need libraries with focus on container runtimes, but we can also benefit from libraries that leverage the generic nature of registries to support all types of artifacts.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cyphar Can you help me understand more about how you see ORAS as an implementation of the distribution-spec? I can see how ORAS enables people to work more easily with distributions, but ORAS itself (at least in my understanding) doesn't implement the distribution protocol.


As ORAS was designed to handle any type of content, it will serve to exercise the spec
to make it more independent of details that may have leaked in from image-spec or Docker.

> *Why bless ORAS over other alternative tools?*

ORAS has already been used successfully in the wild as a building block for
publishing custom content to an OCI registry.

The following projects are already successfully using ORAS to work with custom artifacts:

- [Helm][helm-usage]
- [Conftest][conftest-usage]
- [Singularity][singularity-usage]

[helm-usage]: https://github.com/helm/helm/search?q=oras
[conftest-usage]: https://github.com/instrumenta/conftest/search?q=oras
[singularity-usage]: https://github.com/sylabs/singularity/search?q=oras

> *How do we avoid the runc issue with implementation-specific quirks becoming a de-facto standard?*

When developing new features in ORAS (which are within the scope of the OCI Distribution Specification),
a strong effort will be made to include those features in the upstream specification.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can ORAS commit to ensuring that non-specified features remain marked as strictly experimental and unsupported until such time as those features are included in the upstream specification?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@samuelkarp sure. Meaning from a readme/documentation perspective? I'm assuming this in reference to keying off of config.mediaType for determining artifact types

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and discouraging use. The concern I have here is the same as the concern about the annotations: I'm worried about non-standardized behavior of a reference implementation becoming a de-facto requirement.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would go further than just documentation. You should have to pass an actual --unsupported or --experimental or --i-understand-this-voids-my-warranty flag. That is what I intend to do for umoci, and given my experience with documentation not being read by any user ever (in the case of runc at least), I think putting text in documentation is mostly a CYA thing than actually having an impact on user behaviour.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marking experiments that aren't in a versioned/released spec as experimental makes sense. As we enable Notary v2, I expect a bunch of experiments that we'd likely enable.
That said, I don't understand the question on the config.mediaType. Can someone elaborate the concern here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Talking with Sam, the question was related to the implicit use of annotations, not config.mediatype and how the use of annotations could imply a standard, instead of something used for the bin prototype. And, the experimental references which we agree with as well.
We'll discuss removing the annotations, possibly placing in the config object. Tracked at oras-project/oras#167
We'd likely leave the helper capabilities for artifact authors to add annotations for their specific usage, but avoid always generating, implying a standard.


> *Who are the other target users of ORAS?*

Users seeking a common way to store different types of content (not just container runtime images),
using the OCI Distribution Spec as the baseline API.

> *How do you pronounce ORAS?*

The name ORAS is actually an acronym for "OCI Registry As Storage",
but when speaking of it, you can say "or-ahs".