Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hooks: add pre-resume blocking hook #1074

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions docs/hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ The table below provides an overview of all available hooks.
|----------------|-----------|------------------------------------------------------------------------|---------------------------------------------------------------------------------|---------------------|
| pre-create | Yes | before a new upload is created. | validation of meta data, user authentication, specification of custom upload ID | Yes |
| post-create | No | after a new upload is created. | registering the upload with the main application, logging of upload begin | Yes |
| pre-resume | Yes | before an existing upload is continued. | validation of user authentication | No |
| post-receive | No | regularly while data is being transmitted. | logging upload progress, stopping running uploads | No |
| pre-finish | Yes | after all upload data has been received but before a response is sent. | sending custom data when an upload is finished | Yes |
| post-finish | No | after all upload data has been received and after a response is sent. | post-processing of upload, logging of upload end | Yes |
Expand Down Expand Up @@ -135,9 +136,9 @@ Below you can find an annotated, JSON-ish encoded example of a hook response:
},

// RejectUpload will cause the upload to be rejected and not be created during
// POST request. This value is only respected for pre-create hooks. For other hooks,
// it is ignored. Use the HTTPResponse field to send details about the rejection
// to the client.
// POST/PATCH request. This value is only respected for pre-create and pre-resume hooks.
// For other hooks, it is ignored. Use the HTTPResponse field to send details
// the rejection to the client.
"RejectUpload": false,

// ChangeFileInfo can be set to change selected properties of an upload before
Expand Down Expand Up @@ -303,7 +304,7 @@ For example, assume that every upload must belong to a specific user project. Th

### Authenticating Users

User authentication can be achieved by two ways: Either, user tokens can be included in the upload meta data, as described in the above example. Alternatively, traditional header fields, such as `Authorization` or `Cookie` can be used to carry user-identifying information. These header values are also present for the hook requests and are accessible for the `pre-create` hook, where the authorization tokens or cookies can be validated to authenticate the user.
User authentication can be achieved by two ways: Either, user tokens can be included in the upload meta data, as described in the above example. Alternatively, traditional header fields, such as `Authorization` or `Cookie` can be used to carry user-identifying information. These header values are also present for the hook requests and are accessible for the `pre-create` and `pre-resume` hooks, where the authorization tokens or cookies can be validated to authenticate the user.

If the authentication is successful, the hook can return an empty hook response to indicate tusd that the upload should continue as normal. If the authentication fails, the hook can instruct tusd to reject the upload and return a custom error response to the client. For example, this is a possible hook response:

Expand All @@ -321,7 +322,9 @@ If the authentication is successful, the hook can return an empty hook response
}
```

Note that this handles authentication during the initial POST request when creating an upload. When tusd responds, it sends a random upload URL to the client, which is used to transmit the remaining data via PATCH and resume the upload via HEAD requests. Currently, there is no mechanism to ensure that the upload is resumed by the same user that created it. We plan on addressing this in the future. However, since the upload URL is randomly generated and only short-lived, it is hard to guess for uninvolved parties.
Note that listen `pre-create` hook only handles authentication during the initial POST request when creating an upload. When tusd responds, it sends a random upload URL to the client, which is used to transmit the remaining data via PATCH and resume the upload via HEAD requests. Since the upload URL is randomly generated and only short-lived, it is hard to guess for uninvolved parties.

To have full protection, consider adding `pre-resume` hook too.

### Interrupting Uploads

Expand Down
5 changes: 5 additions & 0 deletions pkg/handler/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,11 @@ type Config struct {
// that should be overwriten before the upload is create. See its type definition for
// more details on its behavior. If you do not want to make any changes, return an empty struct.
PreUploadCreateCallback func(hook HookEvent) (HTTPResponse, FileInfoChanges, error)
// PreUploadResumeCallback will be invoked before resuming an upload, if the
// property is supplied. If the callback returns no error, the upload will continue.
// If the error is non-nil, the upload will be rejected. This can be used to implement
// authorization.
PreUploadResumeCallback func(hook HookEvent) error
// PreFinishResponseCallback will be invoked after an upload is completed but before
// a response is returned to the client. This can be used to implement post-processing validation.
// If the callback returns no error, optional values from HTTPResponse will be contained in the HTTP response.
Expand Down
46 changes: 46 additions & 0 deletions pkg/handler/patch_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -706,6 +706,52 @@ func TestPatch(t *testing.T) {
a.False(more)
})

SubTest(t, "RejectResumeUpload", func(t *testing.T, store *MockFullDataStore, composer *StoreComposer) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
upload := NewMockFullUpload(ctrl)

gomock.InOrder(
store.EXPECT().GetUpload(gomock.Any(), "yes").Return(upload, nil),
upload.EXPECT().GetInfo(gomock.Any()).Return(FileInfo{
ID: "yes",
Offset: 0,
Size: 5,
}, nil),
)

handler, _ := NewHandler(Config{
StoreComposer: composer,
PreUploadResumeCallback: func(event HookEvent) error {
err := ErrUploadRejectedByServer
err.HTTPResponse = HTTPResponse{
StatusCode: http.StatusForbidden,
Body: "upload is stopped because authorization hook failed",
Header: HTTPHeader{
"X-Foo": "bar",
},
}
return err
},
})

(&httpTest{
Method: "PATCH",
URL: "yes",
ReqHeader: map[string]string{
"Tus-Resumable": "1.0.0",
"Upload-Offset": "0",
"Content-Type": "application/offset+octet-stream",
},
ReqBody: strings.NewReader("hello"),
Code: http.StatusForbidden,
ResHeader: map[string]string{
"X-Foo": "bar",
},
ResBody: "upload is stopped because authorization hook failed",
}).Run(handler, t)
})

SubTest(t, "BodyReadError", func(t *testing.T, store *MockFullDataStore, composer *StoreComposer) {
// This test ensure that error that occurr from reading the request body are not forwarded to the
// storage backend but are still causing an
Expand Down
8 changes: 8 additions & 0 deletions pkg/handler/unrouted_handler.go
Original file line number Diff line number Diff line change
Expand Up @@ -749,6 +749,14 @@ func (handler *UnroutedHandler) PatchFile(w http.ResponseWriter, r *http.Request
Header: make(HTTPHeader, 1), // Initialize map, so writeChunk can set the Upload-Offset header.
}

if handler.config.PreUploadResumeCallback != nil {
err := handler.config.PreUploadResumeCallback(newHookEvent(c, info))
if err != nil {
handler.sendError(c, err)
return
}
}

// Do not proxy the call to the data store if the upload is already completed
if !info.SizeIsDeferred && info.Offset == info.Size {
resp.Header["Upload-Offset"] = strconv.FormatInt(offset, 10)
Expand Down
34 changes: 31 additions & 3 deletions pkg/hooks/hooks.go
Original file line number Diff line number Diff line change
Expand Up @@ -90,11 +90,12 @@ const (
HookPostReceive HookType = "post-receive"
HookPostCreate HookType = "post-create"
HookPreCreate HookType = "pre-create"
HookPreResume HookType = "pre-resume"
HookPreFinish HookType = "pre-finish"
)

// AvailableHooks is a slice of all hooks that are implemented by tusd.
var AvailableHooks []HookType = []HookType{HookPreCreate, HookPostCreate, HookPostReceive, HookPostTerminate, HookPostFinish, HookPreFinish}
var AvailableHooks []HookType = []HookType{HookPreCreate, HookPostCreate, HookPreResume, HookPostReceive, HookPostTerminate, HookPostFinish, HookPreFinish}

func preCreateCallback(event handler.HookEvent, hookHandler HookHandler) (handler.HTTPResponse, handler.FileInfoChanges, error) {
ok, hookRes, err := invokeHookSync(HookPreCreate, event, hookHandler)
Expand All @@ -118,6 +119,25 @@ func preCreateCallback(event handler.HookEvent, hookHandler HookHandler) (handle
return httpRes, changes, nil
}

func preResumeCallback(event handler.HookEvent, hookHandler HookHandler) error {
ok, hookRes, err := invokeHookSync(HookPreResume, event, hookHandler)
if !ok || err != nil {
return err
}

httpRes := hookRes.HTTPResponse

// If the hook response includes the instruction to reject the upload, reuse the error code
// and message from ErrUploadRejectedByServer, but also include custom HTTP response values.
if hookRes.RejectUpload {
err := handler.ErrUploadRejectedByServer
err.HTTPResponse = err.HTTPResponse.MergeWith(httpRes)

return err
}
return nil
}

func preFinishCallback(event handler.HookEvent, hookHandler HookHandler) (handler.HTTPResponse, error) {
ok, hookRes, err := invokeHookSync(HookPreFinish, event, hookHandler)
if !ok || err != nil {
Expand Down Expand Up @@ -165,12 +185,14 @@ func SetupHookMetrics() {
MetricsHookErrorsTotal.WithLabelValues(string(HookPostReceive)).Add(0)
MetricsHookErrorsTotal.WithLabelValues(string(HookPostCreate)).Add(0)
MetricsHookErrorsTotal.WithLabelValues(string(HookPreCreate)).Add(0)
MetricsHookErrorsTotal.WithLabelValues(string(HookPreResume)).Add(0)
MetricsHookErrorsTotal.WithLabelValues(string(HookPreFinish)).Add(0)
MetricsHookInvocationsTotal.WithLabelValues(string(HookPostFinish)).Add(0)
MetricsHookInvocationsTotal.WithLabelValues(string(HookPostTerminate)).Add(0)
MetricsHookInvocationsTotal.WithLabelValues(string(HookPostReceive)).Add(0)
MetricsHookInvocationsTotal.WithLabelValues(string(HookPostCreate)).Add(0)
MetricsHookInvocationsTotal.WithLabelValues(string(HookPreCreate)).Add(0)
MetricsHookInvocationsTotal.WithLabelValues(string(HookPreResume)).Add(0)
MetricsHookInvocationsTotal.WithLabelValues(string(HookPreFinish)).Add(0)
}

Expand Down Expand Up @@ -218,8 +240,9 @@ func invokeHookSync(typ HookType, event handler.HookEvent, hookHandler HookHandl
//
// If you want to create an UnroutedHandler instead of the routed handler, you can first create a routed handler and then
// extract an unrouted one:
// routedHandler := hooks.NewHandlerWithHooks(...)
// unroutedHandler := routedHandler.UnroutedHandler
//
// routedHandler := hooks.NewHandlerWithHooks(...)
// unroutedHandler := routedHandler.UnroutedHandler
//
// Note: NewHandlerWithHooks sets up a goroutine to consume the notfication channels (CompleteUploads, TerminatedUploads,
// CreatedUploads, UploadProgress) on the created handler. These channels must not be consumed by the caller or otherwise
Expand All @@ -241,6 +264,11 @@ func NewHandlerWithHooks(config *handler.Config, hookHandler HookHandler, enable
return preCreateCallback(event, hookHandler)
}
}
if slices.Contains(enabledHooks, HookPreResume) {
config.PreUploadResumeCallback = func(event handler.HookEvent) error {
return preResumeCallback(event, hookHandler)
}
}
if slices.Contains(enabledHooks, HookPreFinish) {
config.PreFinishResponseCallback = func(event handler.HookEvent) (handler.HTTPResponse, error) {
return preFinishCallback(event, hookHandler)
Expand Down
35 changes: 34 additions & 1 deletion pkg/hooks/hooks_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,19 @@ func TestNewHandlerWithHooks(t *testing.T) {
HTTPResponse: response,
RejectUpload: true,
}, nil),
hookHandler.EXPECT().InvokeHook(HookRequest{
Type: HookPreResume,
Event: event,
}).Return(HookResponse{
HTTPResponse: response,
}, nil),
hookHandler.EXPECT().InvokeHook(HookRequest{
Type: HookPreResume,
Event: event,
}).Return(HookResponse{
HTTPResponse: response,
RejectUpload: true,
}, nil),
hookHandler.EXPECT().InvokeHook(HookRequest{
Type: HookPreFinish,
Event: event,
Expand Down Expand Up @@ -112,7 +125,7 @@ func TestNewHandlerWithHooks(t *testing.T) {
Event: event,
})

uploadHandler, err := NewHandlerWithHooks(&config, hookHandler, []HookType{HookPreCreate, HookPostCreate, HookPostReceive, HookPostTerminate, HookPostFinish, HookPreFinish})
uploadHandler, err := NewHandlerWithHooks(&config, hookHandler, []HookType{HookPreCreate, HookPostCreate, HookPreResume, HookPostReceive, HookPostTerminate, HookPostFinish, HookPreFinish})
a.NoError(err)

// Successful pre-create hook
Expand All @@ -138,6 +151,26 @@ func TestNewHandlerWithHooks(t *testing.T) {
a.Equal(handler.HTTPResponse{}, resp_got)
a.Equal(handler.FileInfoChanges{}, change_got)

// Successful pre-resume hook
err = config.PreUploadResumeCallback(event)
a.NoError(err)

// Pre-create hook with rejection
err = config.PreUploadResumeCallback(event)
a.Equal(handler.Error{
ErrorCode: handler.ErrUploadRejectedByServer.ErrorCode,
Message: handler.ErrUploadRejectedByServer.Message,
HTTPResponse: handler.HTTPResponse{
StatusCode: 200,
Body: "foobar",
Header: handler.HTTPHeader{
"X-Hello": "here",
"Content-Type": "text/plain; charset=utf-8",
},
},
}, err)
a.Equal(handler.HTTPResponse{}, resp_got)

// Succesful pre-finish hook
resp_got, err = config.PreFinishResponseCallback(event)
a.NoError(err)
Expand Down