File not found (Android)

[jdwirrenga]

I am getting a File not found error running on Android that I am not getting in iOS when running a sim from xCode.

Here is the output:

V/Capacitor/Plugin: To native (Cordova plugin): callbackId: File1325299088, service: File, action: resolveLocalFileSystemURI, actionArgs: ["file:\/\/\/storage\/emulated\/0\/DCIM\/Camera\/VID_20220726_085410110.mp4"]
V/Capacitor/Plugin: To native (Cordova plugin): callbackId: File1325299091, service: File, action: requestAllFileSystems, actionArgs: []
V/Capacitor/Plugin: To native (Cordova plugin): callbackId: FileTransferBackground1325299094, service: FileTransferBackground, action: startUpload, actionArgs: [{"id":"1","filePath":"\/__cdvfile_sdcard__\/DCIM\/Camera\/VID_20220726_085410110.mp4","fileKey":"file","serverUrl":"myServerUrl","notificationTitle":"UploadVideoComplete","headers":{"Authorization":"bearer null","enctype":"multipart\/form-data"},"parameters":{}}]
D/FileTransferBackground: startUpload: Starting work via work manager
D/CordovaBackgroundUpload: eventLabel='Uploader starting upload' uploadId='1'

E/CordovaBackgroundUpload: doWork: File not found !
    java.io.FileNotFoundException: /__cdvfile_sdcard__/DCIM/Camera/VID_20220726_085410110.mp4: open failed: ENOENT (No such file or directory)
        at libcore.io.IoBridge.open(IoBridge.java:492)
        at java.io.FileInputStream.<init>(FileInputStream.java:160)
        at com.spoon.backgroundfileupload.UploadTask.createRequest(UploadTask.java:340)
        at com.spoon.backgroundfileupload.UploadTask.doWork(UploadTask.java:178)
        at androidx.work.Worker$1.run(Worker.java:86)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:923)
     Caused by: android.system.ErrnoException: open failed: ENOENT (No such file or directory)
        at libcore.io.Linux.open(Native Method)
        at libcore.io.ForwardingOs.open(ForwardingOs.java:166)
        at libcore.io.BlockGuardOs.open(BlockGuardOs.java:254)
        at libcore.io.ForwardingOs.open(ForwardingOs.java:166)
        at android.app.ActivityThread$AndroidOs.open(ActivityThread.java:7685)
        at libcore.io.IoBridge.open(IoBridge.java:478)
        at java.io.FileInputStream.<init>(FileInputStream.java:160) 
        at com.spoon.backgroundfileupload.UploadTask.createRequest(UploadTask.java:340) 
        at com.spoon.backgroundfileupload.UploadTask.doWork(UploadTask.java:178) 
        at androidx.work.Worker$1.run(Worker.java:86) 
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) 
        at java.lang.Thread.run(Thread.java:923) 

I am not sure what is converting "file:///storage/emulated/0" to "cdvfile_sdcard" in the filepath but that seems like it might be a problem? Additionally, maybe the file system on the computer via the ios sim is simply not a valid sim of what it will be on an actual iOS device and it will fail in there as well.

I have also tried files paths without the "file://" appended to the front, and with the capacitor localhost file thing Capacitor.convertFileSrc(uri) but these don't seem to work.

[zfir]

Hello @jdwirrenga,

Can you send us your package.json and your config.xml

Regards,
Zafir.

[jdwirrenga]

I am using latest version of ionic capacitor so there is no config.xml, so let me know what you're looking for and I can send that as well. I originally had the cordova-plugin-background-upload package instead of the @spoonconsulting one but I could not get it to build on android because of old versions of things.

package.txt

[zfir]

Hello @jdwirrenga,

As you are using capacitor, you can send us the capacitor.config.json.

Also are you using this package "@awesome-cordova-plugins/background-upload": "^6.0.0-alpha.4". If not, try to remove it in your package.json and run the app again.

Also let us know which version of XCode, iOS, Emulator Device, ... you are using.

Regards,
Zafir

[jdwirrenga]

@zafirskthelifehacker I am using the @awesome-corodova-plugins package. I tried removing it and importing from @spoonconsulting but I can't get ionic to recognize it as a module. I know for a fact I tried not using @spoonconsulting and nothing would even compile. I also tried following the readme to set up the uploader but couldn't get any of that to work.

All the xcode emulators and the iPad I ran it on seem to work. What failed was running it in Android on an actual device we use for testing that is running Android 11 right now.

My config:

import { CapacitorConfig } from '@capacitor/cli';

const config: CapacitorConfig = {
  appId: 'com.something.app',
  appName: 'something',
  webDir: 'www',
  bundledWebRuntime: false,
  plugins: {
    SplashScreen: {
      launchAutoHide: false
    }
  }
};

export default config;

Here is the relevant code where I am actually doing the upload

import { FileTransferManager, FTMOptions, FTMPayloadOptions, UploadEvent } from '@awesome-cordova-plugins/background-upload';

uploader: FileTransferManager

constructor() { 
  if(this.platform.is("mobile") && !this.platform.is("mobileweb")) {
	  var config: FTMOptions = {
		  callBack: this.handleVideoUpload
	  };
  
	  this.uploader = new FileTransferManager(config)
  }
}

uploadFile(params) {
  let payload: FTMPayloadOptions = {
	  id: id,
	  filePath: filePath,
	  fileKey: fileKey,
	  serverUrl: url,
	  notificationTitle: "UploadVideoComplete",
	  headers: fullHeaders,
  }

  return of(this.uploader.startUpload(payload))
}

[zfir]

Hello @jdwirrenga,

Based on the error, the path that is given from the capacitor is not recognized by the native side for Android.

To fix that you can use cordova.file.dataDirectory from cordova-plugin-file to rewrite the file path before sending it to the plugin.

Regards,
Zafir.

[jdwirrenga]

@zafirskthelifehacker I tried file.moveFile to file.dataDirectory but it still was unable to upload the file. Why does it always look in the /cdvfile_sdcard/ directory even when you pass in the file such as file:///data/user/0/com.name.app/files/VID_20220726_085410110.mp4? I noticed that if you get the fileEntry and do toInternalURL() or toURL() that they give you this same path effectively, Ex: http://localhost/__cdvfile_files__/VID_20220726_085410110.mp4

I tried using nativeURL as well with the same error.

let fileSplitIndex = file.lastIndexOf('/')
      let filePath = file.substring(0, fileSplitIndex)
      let fileName = file.substring(fileSplitIndex + 1)
      
      this.file.resolveDirectoryUrl(filePath).then((dirEntry: DirectoryEntry) => {
        this.file.getFile(dirEntry, fileName, {}).then((fileEntry: FileEntry) => {
          console.log("Get file stuff:")
          console.log(fileEntry.fullPath)
          console.log(fileEntry.nativeURL)
          console.log(fileEntry.toInternalURL())
          console.log(fileEntry.toURL())
          this.rest.backgroundUploadFile('/Profile/UploadVideo', videoId.toString(), fileEntry.toInternalURL(), "file").subscribe()
        })
      })

      this.file.moveFile(filePath, fileName, this.file.dataDirectory, fileName).then((output: Entry) => {
        console.log("Move file stuff:")
        console.log(output.fullPath)
        console.log(output.nativeURL)
        console.log(output.toInternalURL())
        console.log(output.toURL())
        this.rest.backgroundUploadFile('/Profile/UploadVideo', videoId.toString(), output.toInternalURL(), "file").subscribe()
        videoId++
      })   

Whose output looks like:

Msg: Get file stuff:
Msg: /DCIM/Camera/VID_20220726_085410110.mp4
Msg: file:///storage/emulated/0/DCIM/Camera/VID_20220726_085410110.mp4
Msg: http://localhost/__cdvfile_sdcard__/DCIM/Camera/VID_20220726_085410110.mp4
Msg: http://localhost/__cdvfile_sdcard__/DCIM/Camera/VID_20220726_085410110.mp4
Msg: RestService upload sending Upload to http://192.168.1.192:45461/Profile/UploadVideo with file http://localhost/__cdvfile_sdcard__/DCIM/Camera/VID_20220726_085410110.mp4

Msg: Move file stuff:
Msg: /VID_20220726_085410110.mp4
Msg: file:///data/user/0/com.appname.app/files/VID_20220726_085410110.mp4
Msg: http://localhost/__cdvfile_files__/VID_20220726_085410110.mp4
Msg: http://localhost/__cdvfile_files__/VID_20220726_085410110.mp4
Msg: RestService upload sending Upload to http://192.168.1.192:45461/Profile/UploadVideo with file http://localhost/__cdvfile_files__/VID_20220726_085410110.mp4

E/CordovaBackgroundUpload: doWork: File not found !
    java.io.FileNotFoundException: /__cdvfile_sdcard__/DCIM/Camera/VID_20220726_085410110.mp4: open failed: ENOENT (No such file or directory)
        at libcore.io.IoBridge.open(IoBridge.java:492)
        at java.io.FileInputStream.<init>(FileInputStream.java:160)
        at com.spoon.backgroundfileupload.UploadTask.createRequest(UploadTask.java:340)
        at com.spoon.backgroundfileupload.UploadTask.doWork(UploadTask.java:178)
        at androidx.work.Worker$1.run(Worker.java:86)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:923)

[zfir]

Hello @jdwirrenga,

As what I am seeing, you are passing fileEntry.toInternalURL() to Background Upload. You should use fileEntry.nativeURL instead as Background Upload take these kind of inputs(e.g. file:///storage/emulated/0/DCIM/Camera/VID_20220726_085410110.mp4)

this.rest.backgroundUploadFile('/Profile/UploadVideo', videoId.toString(), fileEntry.nativeURL, "file").subscribe()

Something like this.

Regards,
Zafir.

[jdwirrenga]

@zafirskthelifehacker I tried all of them including nativeURL, they all give the same error. Which is to say neither
file:///storage/emulated/0/DCIM/Camera/VID_20220726_085410110.mp4 nor file:///data/user/0/com.appname.app/files/VID_20220726_085410110.mp4 worked either.

[zfir]

Hello @jdwirrenga,

Have you request the permission READ_EXTERNAL_STORAGE in your app? If not, try to do that and continue to use nativeUrl.

You can try to use WRITE_EXTERNAL_STORAGE if the error persists.

Regards,
Zafir.

[jdwirrenga]

@zafirskthelifehacker this is what I have for permissions in the android app so far:

<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

As I look at my AndroidManifest.xml I do notice this

<provider
            android:name="androidx.core.content.FileProvider"
            android:authorities="${applicationId}.fileprovider"
            android:exported="false"
            android:grantUriPermissions="true">
            <meta-data
                android:name="android.support.FILE_PROVIDER_PATHS"
                android:resource="@xml/file_paths"></meta-data>
        </provider>

Is it possible I need something similar for the background upload with the android:grantUriPermissions="true" ?

[zfir]

Hello @jdwirrenga,

You can try the android:grantUriPermissions="true". Also try on different devices.

Have you tried an image file(jpg, png, ...)? If not, you can try this and let me know.

Regards,
Zafir.

[jdwirrenga]

@zafirskthelifehacker Tried on a Pixel 5 as well and got the same error. Tried a jpg and got the same error.

[zfir]

Hello @jdwirrenga,

Try to add android:requestLegacyExternalStorage="true" in the generated AndroidManifest.xml file of Android Studio.

Regards,
Zafir.

[jdwirrenga]

@zfir I added that line and two others to my manifest so that it looks like below, but am still getting the error.

<application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:roundIcon="@mipmap/ic_launcher_round"
        android:supportsRtl="true"
        android:theme="@style/AppTheme"
        android:usesCleartextTraffic="true"
        android:grantUriPermissions="true"
        android:requestLegacyExternalStorage="true"
        android:preserveLegacyExternalStorage="true">

[jdwirrenga]

@zfir I found this stackoverflow about using old storage while targeting version 30 or higher...I am targeting 31 which I believe I had to in order to even get this package to compile in Android:
https://stackoverflow.com/questions/63364476/requestlegacyexternalstorage-is-not-working-in-android-11-api-30

[zfir]

Hello @jdwirrenga,

Have you found the fix?

Based on this StackOverflow, Google suggested to use MANAGE_EXTERNAL_STORAGE permission(https://developer.android.com/training/data-storage/manage-all-files). You can try that also.

Regards,
Zafir.

[jdwirrenga]

@zfir No I haven't found a fix. I added the MANAGE_EXTERNAL_STORAGE permission but that did not help.

I suspect a workaround would be to find a version which allows me to bump android down to 29 but still new enough to prevent other problems.

I think maybe I will just take the time to make a repo you can checkout and test the error yourself. However, it might be a little while before I have time to do so.

[jdwirrenga]

@zfir I had a little time to set the repo up. I must be missing one piece though because it does everything except actually kick off the upload it seems. But if that one issue was fixed it should be an exact repro of my issue.

https://github.com/jdwirrenga/CordovaUploadAndroidBug

[zfir]

Hello @jdwirrenga,

I am getting this error when running on Android:

Regards,
Zafir.

[jdwirrenga]

@zfir where did you see this error at? I could not repro on my machine. I did however add the package to the repo but I'm still getting the same thing in the output where it gets to this line
D/CordovaBackgroundUpload: eventLabel='Uploader starting upload' uploadId='1'

Which should be followed immediately by the error but no error. Just in case I tried adding the core package to my actual project as well to see if that resolved the error and it did not.

[zfir]

Hello @jdwirrenga,

When doing ionic capacitor run android I am getting this error. How do you run the project?

Regards,
Zafir.

[jdwirrenga]

@zfir I typically run on an android device by first opening Android Studio with ion cap open android and then apply changes as needed with ion cap copy android and run it each time. ion cap sync android on occasion as well of course.

It really bothers me the repo I made doesn't hit the error like the actual project does. It seems like it doesn't actually "do work" or something like it should. Not sure what the different between the two is. I can't get the repo to hit an endpoint or anything.

[zfir]

Hello @jdwirrenga,

So this repository is not getting the same file path error as your actual project? This repository does not start the uploads based on your comments. I would recommend you to include Logs in the Background Upload Plugin and checks where it stops.

As for running your repository ion cap copy android, I am still getting the errors. Can you commit your ./www also.

Regards,
Zafir

[ckamil]

@jdwirrenga Did you manage to solve the problem? I have the same error

I also use capacitor but sdk version 32

[jdwirrenga]

@ckamil unfortunately I have not had more time to look into this error. We have been focusing on launching for iOS so this was put on the backburner. I still suspect that there is a magic version number between Android and the plugin that just needs to be found. I don't think it is a permissions problem anymore. I think the location of things has simply changed.

If you find the solution please let us know! Feel free to use that repo I made to test things out.

@zfir if is it determined that the file system has simply changed and it is not a permissions problem, what are the chances an update can be made to the plugin?

[zfir]

Hello @jdwirrenga,

We are already building a cordova app with SDK Build Tools 30 with no problems. When upgrading to SDK Build Tools 32 if we overcome any problem, we will surely fix the plugin.

If ever you are able to find any solution, pull requests are most welcome.

Regards,
Zafir.

[ckamil]

@jdwirrenga @zfir thanks for the reply, unfortunately I can't go lower with the SDK, so it will be nice if there is a fix in the near future

[ckamil]

@jdwirrenga @zfir the problem is here FileTransferManager.js#L54

when I replace
payload.filePath = new URL(entry.toURL()).pathname.replace(/^\/local-filesystem/, '')

to
payload.filePath = payload.filePath.replace("file://", "")

then the upload works properly

[jdwirrenga]

@ckamil great find! What file path were you sending? Is it the capacitor one or did you do something with FileEntry first?
@zfir any chance this can get put into a pull request and pushed out in the near future?

[ckamil]

@jdwirrenga url in the form file:///data/user/0/com.example/files/photo_0__1664542670860.png taken from the Filesystem.writeFile method

[zfir]

Hello @jdwirrenga @ckamil,

@ckamil Nice. I would recommend you to submit a PR, we will try to review and merge it as fast as we can. Thanks.

Regards,
Zafir.

[ckamil]

@zfir I do not know what exactly should be here, in my app I have so:

if(window.cordova.platformId === 'android') {
  payload.filePath = payload.filePath.replace("file://", "");
} else {
  payload.filePath = new URL(entry.toURL()).pathname.replace(/^\/local-filesystem/, '')
}

[zfir]

Hello @ckamil

I think for the android part, it should be something like:
payload.filePath = entry.toURL().replace('file://', '')

Try this and let me know.

Regards,
Zafir.

[ckamil]

@zfir this won't work because entry.toURL() returns the wrong url:

http://192.168.88.223:8100/__cdvfile_sdcard__/Documents/photo_0__1665145973561.png

[zfir]

@ckamil Create a PR with changes that you have made. I will test and let you know.

[jdwirrenga]

@dennismuench might you have a solution for this? I see you are the last commit for this line of code.

[gbrits]

Hi @zfir & @jdwirrenga - I just got my app uploading 🚀 (after 2 days of suffering haha)

Firstly, I just add file:/// to the front of my fileUri when sending it as a payload. Also, I had no success until I changed the permissions in my manifest, try matching yours to these & trying again?

<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.READ_MEDIA_IMAGES" />
<uses-permission android:name="android.permission.READ_MEDIA_VIDEO" />
<uses-permission android:name="android.permission.READ_MEDIA_AUDIO" />
<uses-permission android:maxSdkVersion="32" android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:maxSdkVersion="29" android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

My implementation is an Ionic web app, I'm using Capacitor, but I am using this Cordova plugin to talk to the backgrounding:

Here are my wrappers that talk to the plugin:

import { FileTransferManager, FTMOptions, FTMPayloadOptions, UploadEvent } from '@awesome-cordova-plugins/background-upload/ngx';
import { File } from '@awesome-cordova-plugins/file/ngx';
import { BackgroundUpload } from '@awesome-cordova-plugins/background-upload/ngx';

[MaximBelov]

changes for support android 11+ and cordova-plugin-file v7.0.0
#252