Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multipart/form-data uploads #7

Closed
helje5 opened this issue Jan 20, 2021 · 5 comments
Closed

Support multipart/form-data uploads #7

helje5 opened this issue Jan 20, 2021 · 5 comments
Assignees
Labels
enhancement New feature or request

Comments

@helje5
Copy link
Member

helje5 commented Jan 20, 2021

We need a body parser for multipart/form-data uploads, as per apple/swift-nio#1724 (comment)

I.e. to handle things like

<form action="/upload" method="POST" enctype="multipart/form-data">
     <input type="file" id="filename" name="file" onchange="onchanged()" multiple="multiple" />
     <input type="submit" value="Upload">
</form>

Need to check whether there is a more official Node API for that. Generally I'd think this should stream the files to a temporary location, keep the file handles open, but unlink them - or just regular tmpfiles. The body structure would then have a reference to the file handles.

@helje5 helje5 added the enhancement New feature or request label Jan 20, 2021
@helje5 helje5 self-assigned this Jan 20, 2021
@helje5
Copy link
Member Author

helje5 commented Jan 20, 2021

RFC: https://tools.ietf.org/html/rfc7578

Multer seems to be the default Node thing: https://www.npmjs.com/package/multer

var upload = multer({ dest: 'uploads/' })
app.post('/profile',               upload.single('avatar'),       function (req, res, next) {})
app.post('/photos/upload', upload.array('photos', 12), function (req, res, next) {})
app.post('/profile',               upload.none(),                     function (req, res, next) {})

// req.files is array of `photos` files
// req.body will contain the text fields, if there were any

File info: filename, originalname, encoding, mimetype, size, destination (DiskStorage), filename, path, buffer

Options:

  • dest (path) or storage (xyzStorage)
  • fileFilter
  • limits
  • preservePath

DiskStorage or MemoryStorage:

var storage = multer.diskStorage({
  destination: function (req, file, cb) {
    cb(null, '/tmp/my-uploads')
  },
  filename: function (req, file, cb) {
    cb(null, file.fieldname + '-' + Date.now())
  }
})
 
var upload = multer({ storage: storage })

var storage = multer.memoryStorage()
var upload = multer({ storage: storage })

@helje5
Copy link
Member Author

helje5 commented Jan 20, 2021

There is also Formidable: https://github.com/node-formidable/formidable

@helje5
Copy link
Member Author

helje5 commented Jan 26, 2021

The feature/multipart-formdata-1 now has an apparently working multipart/form-data parser, with quite a few tests. Only the MemoryStorage is supported so far, DiskStorage would require more async hops.

Performance doesn't seem to be too great yet, uploading a 3MB image (locally) takes 255ms in release (4s in debug), which seems a little slow for an M1 Mini. Need to check where the time gets lost.

@helje5
Copy link
Member Author

helje5 commented Jan 27, 2021

Uploading my 3MB test image on M1 now takes 2ms in release (12ms in Debug), which seems quite reasonable. 100 MB take about 50ms, which also sounds OK.

What we don't have yet is DiskStorage (though it's prepared), I put that into a separate Issue #8.

The multer module is available in 0.8.0.

@helje5 helje5 closed this as completed Jan 27, 2021
@Jinxiansen
Copy link

@helje5
Thank you very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Development

No branches or pull requests

2 participants