Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

api: add HTTP-like object uploader #124

Merged
merged 2 commits into from
Jan 19, 2024
Merged

api: add HTTP-like object uploader #124

merged 2 commits into from
Jan 19, 2024

Conversation

tatiana-nspcc
Copy link
Contributor

Add POST /upload/{cid} API.

Close #111.

@tatiana-nspcc
Copy link
Contributor Author

I receive an error while trying to use custom multipart reader:

2023-12-22T16:06:27.370Z        error   handlers/api.go:206     could not receive multipart/form        {"error": "multipart: NextPart: EOF"}

It's line

return nil, fmt.Errorf("multipart: NextPart: %v", err)

@cthulhu-rider
Copy link
Contributor

cthulhu-rider commented Dec 25, 2023

I receive an error while trying to use custom multipart reader:

@tatiana-nspcc

u dont need this manual multipart decoder

contentType := params.HTTPRequest.Header.Get("Content-Type")
_, paramsCT, err := mime.ParseMediaType(contentType)
if err != nil {
resp := a.logAndGetErrorResponse("parse content type", err)
return errorResponse.WithPayload(resp)
}
boundary := paramsCT["boundary"]
if file, err = fetchMultipartFile(a.log, params.HTTPRequest.Body, boundary); err != nil {
resp := a.logAndGetErrorResponse("could not receive multipart/form", err)
return errorResponse.WithPayload(resp)
}

try this:

	if swagFile, ok := params.File.(*swag.File); ok {
		header = swagFile.Header
		file = swagFile.Data
	} else {
		const fileKey = "file"
		file, header, err = params.HTTPRequest.FormFile(fileKey)
		if err != nil {
			resp := a.logAndGetErrorResponse(fmt.Sprintf("get file %q from HTTP request", fileKey), err)
			return errorResponse.WithPayload(resp)
		}
	}

@tatiana-nspcc tatiana-nspcc force-pushed the 111-upload branch 2 times, most recently from 396c21d to 8e80e09 Compare December 27, 2023 11:23
@tatiana-nspcc
Copy link
Contributor Author

Thanks, @cthulhu-rider I've used that.

When I tried to run the test cases, I noticed that http-gw doesn't care about the naming of the field for upload. In the examples (https://github.com/nspcc-dev/neofs-http-gw/blob/0b22557622db8551bfbfc8d0ff0fde51fbdea9c3/README.md#L538), I see the file field, but the test cases use the upload_file field name and sometimes file. However, any arbitrary name can be used for the field. For the rest-gw, we should choose a fixed name for the file field, I think. Which one should we choose?

@cthulhu-rider
Copy link
Contributor

cthulhu-rider commented Dec 28, 2023

Which one should we choose?

definitely strict behavior. There may be several files in any request. API server must clearly distinguish them on the server and require strict compliance with the protocol for system stability.

However, any arbitrary name can be used for the field.

it's an incorrect behavior to me that will break if we add one more file into the request. Lucky we dont care bout HTTP GW anymore

For the rest-gw, we should choose a fixed name for the file field

payload (same as in the NeoFS native protocol). And we must double-check that server responds with 4XX (in practice 400 Bad Request, but most important that not 2XX) if we set any other name like file

Comment on lines 801 to 802
//swagFile *swag.File
//ok bool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like unused, it is better to remove

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, you're right, fixed. And also rebased to fresh master state.

@tatiana-nspcc tatiana-nspcc force-pushed the 111-upload branch 2 times, most recently from c002c0a to b8f60d3 Compare January 15, 2024 17:38
Copy link

codecov bot commented Jan 15, 2024

Codecov Report

Attention: 163 lines in your changes are missing coverage. Please review.

Comparison is base (8a638bd) 13.70% compared to head (404baff) 14.27%.
Report is 1 commits behind head on master.

Files Patch % Lines
handlers/objects.go 0.00% 156 Missing ⚠️
handlers/util.go 90.90% 3 Missing and 1 partial ⚠️
handlers/api.go 0.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #124      +/-   ##
==========================================
+ Coverage   13.70%   14.27%   +0.57%     
==========================================
  Files          10       10              
  Lines        2080     2262     +182     
==========================================
+ Hits          285      323      +38     
- Misses       1765     1908     +143     
- Partials       30       31       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -1200,3 +1232,17 @@ definitions:
- success
example:
success: true
AddressForUpload:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this redundant while having

neofs-rest-gw/spec/rest.yaml

Lines 994 to 1007 in eee7071

Address:
description: Address of the object in NeoFS.
type: object
properties:
containerId:
type: string
objectId:
type: string
required:
- containerId
- objectId
example:
objectId: 8N3o7Dtr6T1xteCt6eRwhpmJ7JhME58Hyu1dvaswuTDd
containerId: 5HZTn5qkRnmgSz9gSrw22CEdPPk6nQhkwf2Mgzyvkikv
?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, while we are keeping the rest-gw in a working state and want to pass tests for http-gw. This is because the tests require object_id and container_id, but we've had containerId and objectId.

handlers/objects.go Outdated Show resolved Hide resolved
handlers/util.go Outdated Show resolved Hide resolved
handlers/util.go Outdated Show resolved Hide resolved
handlers/util.go Outdated Show resolved Hide resolved
handlers/util.go Outdated Show resolved Hide resolved
handlers/util.go Outdated Show resolved Hide resolved
handlers/util.go Outdated Show resolved Hide resolved
handlers/util.go Outdated Show resolved Hide resolved
Add `POST /upload/{cid}` API.

Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
Fix filename and Content-type for GET and HEAD APIs.
Fix cookie prefix for bearer token.

Signed-off-by: Tatiana Nesterenko <tatiana@nspcc.io>
@roman-khimov roman-khimov merged commit 12dddb7 into master Jan 19, 2024
13 checks passed
@roman-khimov roman-khimov deleted the 111-upload branch January 19, 2024 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add uploader HTTP-alike API
4 participants