[BUG]: flashing and booting issues

[sskorol]

Describe the issue

• Followed the steps from this video: https://www.jetsonhacks.com/2021/08/25/native-boot-for-jetson-xaviers/
• When I ran flash_jetson_external_storage.sh it failed with the following stacktrace:

[   0.5738 ] tegrasign_v3.py --key None --list bct_list.xml --pubkeyhash pub_key.key --getmontgomeryvalues montgomery.bin
[   0.5775 ] /usr/bin/env: Traceback (most recent call last):
  File "./tegraflash.py", line 1296, in <module>
    tegraflash_run_commands()
  File "./tegraflash.py", line 1155, in tegraflash_run_commands
    interpreter.onecmd(command)
  File "/usr/lib/python3.8/cmd.py", line 217, in onecmd
    return func(arg)
  File "./tegraflash.py", line 667, in do_sign
    tegraflash_sign(exports)
  File "/home/skorol/Projects/bootFromExternalStorage/R32.6.1/Linux_for_Tegra/bootloader/tegraflash_internal.py", line 1078, in tegraflash_sign
    tegraflash_generate_bct()
  File "/home/skorol/Projects/bootFromExternalStorage/R32.6.1/Linux_for_Tegra/bootloader/tegraflash_internal.py", line 4004, in tegraflash_generate_bct
    tegraflash_generate_br_bct(True)
  File "/home/skorol/Projects/bootFromExternalStorage/R32.6.1/Linux_for_Tegra/bootloader/tegraflash_internal.py", line 3510, in tegraflash_generate_br_bct
    run_command(command)
  File "/home/skorol/Projects/bootFromExternalStorage/R32.6.1/Linux_for_Tegra/bootloader/tegraflash_internal.py", line 202, in run_command
    log = print_process(process, enable_print)
  File "/home/skorol/Projects/bootFromExternalStorage/R32.6.1/Linux_for_Tegra/bootloader/tegraflash_internal.py", line 164, in print_process
    outputchar = output.decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)
Error: /home/skorol/Projects/bootFromExternalStorage/R32.6.1/Linux_for_Tegra/bootloader/signed/flash.idx is not found
Error: failed to relocate images to /home/skorol/Projects/bootFromExternalStorage/R32.6.1/Linux_for_Tegra/tools/kernel_flash/images
Cleaning up...

Which version of Ubuntu on the Host machine
Ubuntu version: 20.04.2 LTS

Which Jetson
Jetson: Xavier NX

To Reproduce
Steps to reproduce the behavior: just run flash_jetson_external_storage.sh following the video guide

Expected behavior
Flashing should be completed.

[sskorol]

I added export LC_ALL=C.UTF-8. Now it's failing with a new error. Continue googling... :)

[   0.5848 ] Get Signed section of bct
[   0.5867 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x19 0 --listbct bct_list.xml
[   0.5875 ] 
[   0.5894 ] tegrasign_v3.py --key None --list bct_list.xml --pubkeyhash pub_key.key --getmontgomeryvalues montgomery.bin
[   0.5901 ] /usr/bin/env: ‘python’: No such file or directory
[   0.5907 ] 
Error: Return value 127
Command tegrasign_v3.py --key None --list bct_list.xml --pubkeyhash pub_key.key --getmontgomeryvalues montgomery.bin
Error: /home/skorol/Projects/bootFromExternalStorage/R32.6.1/Linux_for_Tegra/bootloader/signed/flash.idx is not found
Error: failed to relocate images to /home/skorol/Projects/bootFromExternalStorage/R32.6.1/Linux_for_Tegra/tools/kernel_flash/images
Cleaning up...

[sskorol]

The following package installation did the trick:

sudo apt-get install python-is-python3

Waiting for completion... Fingers crossed.

[sskorol]

Ok, seems like I'm good to continue:

[ 184]: l4t_flash_from_kernel: Successfully flash the qspi
[ 184]: l4t_flash_from_kernel: Flashing success

Note: The flash process might have added some nvidia-flash-* connections in
NetworkManager to set up USB ethernet through flashing port (L4t USB Device mode).
You might want to check your NetworkManager configuration if you have some special
configuration.
Reboot target
Run command: sync; { sleep 1; reboot; } >/dev/null & on root@fe80::1%usb0
bash: warning: setlocale: LC_ALL: cannot change locale (C.UTF-8)
Success
Cleaning up...
Log is saved to Linux_for_Tegra/initrdlog/flash_1-2.2_0_20210826-114426.log 

[sskorol]

Moved to Jetson and see the following:

EXT4-fs (nvme0n1): VFS: Can't find ext4 filesystem
You are in emergency mode. After logging in, type "journalctl -xb" to view system logs, "systemctl reboot" to reboot, "systemctl default" or "exit" to boot into default mode.
Press Enter for maintenance
(or Control-D to continue)

[sskorol]

In logs I see the following:

[sskorol]

I ran the process again. Reached the point where I need to switch to Xavier. I connected it via hdmi cable to see what's going on there. And there's a black screen with a blinking cursor. Key presses don't work. BTW, in logs of a flash_jetson_external_storage I noticed that it formats /dev/sda. Shouldn't it be a nvme0n1p1 selected by default? Or I misinterpreted the output?

[sskorol]

Ok, it seems like the size of a drive is correct, so it formatted a valid one. Finally, I noticed one important note in a README that I've previously missed: removing sd card. I removed it and repeated the process from scratch. Then manually rebooted the board after flashing and now I see the default oem screen! So it works! :)

[sskorol]

BTW, I also noticed some python warnings before flashing:

/home/skorol/Projects/bootFromExternalStorage/R32.6.1/Linux_for_Tegra/bootloader/tegraflash_internal.py:2064: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if sig_type is not "zerosbk":
/home/skorol/Projects/bootFromExternalStorage/R32.6.1/Linux_for_Tegra/bootloader/tegraflash_internal.py:2067: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if sig_type is "oem-rsa":
/home/skorol/Projects/bootFromExternalStorage/R32.6.1/Linux_for_Tegra/bootloader/tegraflash_internal.py:2560: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  while count is not 0 and not check_ismb2():
/home/skorol/Projects/bootFromExternalStorage/R32.6.1/Linux_for_Tegra/bootloader/tegraflash_internal.py:2569: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  while count is not 0:

They are self-explanatory and easy to fix. I did it and no more warnings.

[sskorol]

One more point. When I tried to get_jetson_files when they have been already present on my host, I faced different issues with permissions and cudatools. If I'm not mistaken, the first one I got w/ disconnected board. And in terms of a second, it seems like I ran a script with a connected board. Not sure what had happened. But when I removed the repo folder and cloned it from scratch, the process has finished w/o errors.

[sskorol]

@jetsonhacks to summarize, I'd made the following improvements to avoid any further confusion:

• add export LC_ALL=C.UTF-8
• add python-is-python3 package
• fix tegraflash_internal.sh python warnings
• add a step to the README which requires rebooting the Jetson board (if it should be done automatically, it didn't work for me)
• reword the warning about SD card, that it must be absent and may cause booting issues (or if it's not true, then would be nice to figure out why I couldn't boot for the first time)
• add a cleanup script/step for situations when users need to run the process multiple times (e.g. in case of any errors)

[anhmiuhv]

The SD card thing is because of the boot order of the bootloader. If Your sd card is present and not empty, the jetson xavier nx will priotize booting from the SD card.

[jetsonhacks]

Most of these script issues have been addressed now.

[NickBeers]

I have tried this step by step a dozen times. All the scripts seem to work, the flash ends in success. If I try to screen into it, it auto fails. If I try to putty from Windows, it asks for a localhost login and password. Any idea what that could be? I have tried from an Ubuntu 20.04 host, and then a fresh ubuntu 18.04 host. I'm at a loss

[jetsonhacks]

@NickBeers Are you using a Xavier NX or AGX Xavier? Did you try to set it up with a monitor, keyboard and mouse?

[NickBeers]

I'm have an Xavier NX, with a SK Hynix 512 MB Gen 3x4 drive, no sd card. It asks for the username/password from screen too, I just have to sudo screen to get to that part. I'm trying to set this up headless, so maybe that is the problem. After the flashing script exits, do I turn off the jetson? I may be making a silly mistake between the end of the script and trying to screen or putty into it. I'll try hooking it up to a monitor and keyboard this time, I've ran that script maybe 20 times.

Thanks for your awesome videos, I enjoy the style of them greatly!

[NickBeers]

@jetsonhacks Just tried with a monitor, it shows the Nvidia logo, some text for a second, then goes blank. Let me try another drive and see if maybe I have a defective drive.

[jetsonhacks]

I'm have an Xavier NX, with a SK Hynix 512 MB Gen 3x4 drive, no sd card. It asks for the username/password from screen too, I just have to sudo screen to get to that part. I'm trying to set this up headless, so maybe that is the problem. After the flashing script exits, do I turn off the jetson? I may be making a silly mistake between the end of the script and trying to screen or putty into it. I'll try hooking it up to a monitor and keyboard this time, I've ran that script maybe 20 times.

Thanks for your awesome videos, I enjoy the style of them greatly!

Thank you for the kind words. Some people have reported issues with oem-config not coming up correctly. anhmiuhv is one of the NVIDIA engineers who works on this, this is reported in another issue:

Flash from external storage should already flash to emmc in the process.

Oem-config is expected to run fine when booting from NVMe; but in case when flashing to NVMe and oem-config fails to start, you can run

sudo ./tools/l4t_create_default_user.sh -u ubuntu -p ubuntu # this command create username ubuntu and password ubuntu

to skip oem-config

You'll need to be in the Linux_for_Tegra directory when executing the above command. Note that you may have to expand your APP partition to fill up the rest of your disk. That's part of the config walk through process. I don't know yet if this is due to a change that they're making for adding this feature to the GUI SDK Manager, or some other issue.

I know that people of AGX Xavier are experiencing this issue.

[NickBeers]

@jetsonhacks The new drive did the same thing.

I ran that command you just suggested directly after the flashing was complete, without rebooting, and it made a user that allows me to log in. Thank you very much!!!

[NickBeers]

@jetsonhacks Just ran into an issue, my root partition only allocated 14GB. Any tips on extending it to fill the whole drive?

[jetsonhacks]

@NickBeers Open the Disks application, select the drive.
Go to Partition 1 (it's Filesystem, labeled APP)
Click the gear icon.
Select 'Resize..'

[jetsonhacks]

@NickBeers Did you flash from a PC running Ubuntu natively?

[NickBeers]

@jetsonhacks I did flash it from an Ubuntu 18.04 install. Do you know how to do it headless? I tried using gdisk, and couldn't figure out how to get the resize to stick. I selected the /dev/nvme0n1, then resized partition 1, which is my App partition. In gpart, it says it goes for 450GB, but when I quit and reboot it is still 14GB.

[anhmiuhv]

You also need to resize the file system itself in addition to resizing the partition.

sudo resize2fs /dev/nvme0n1p… ( the APP parition)

[anhmiuhv]

This step is done by oem config but because oem-config is skipped you have to do it yourself

[NickBeers]

@anhmiuhv That worked! Thank you very much!!