Keep running out of space trying to build timelapse

[kf6spf]

QUESTION

Is there a way to increase the allsky/tmp size post install??

Environment

• Camera: RPi HQ ArduCam_16_MP
• OS: Bullseye
• Allsky version: v2023.05.01_04

Model: Raspberry Pi 4 Model B Rev 1.5
# CPUs: 4 Cortex-A72 at 1800 MHz (BCM2835)
Memory: 3834296 kB
Kernel: Linux 6.1.21-v8+ aarch64
OS: Raspbian 11 (bullseye)

Bug Description

• My system was building Timelapse movies fine, but now it runs out of memory
• Reproducible:

$ scripts/timelapse.sh --mini 20240623
timelapse.sh
Not regenerating sequence because KEEP_SEQUENCE is enabled.

*** timelapse.sh: ERROR: ffmpeg failed.
Error log:
 [image2 @ 0xe2a960] Could find no file with path '/home/mwallis/work/allsky/tmp/sequence-timelapse.20240627/%04d.jpg' and index in the range 0-4
/home/mwallis/work/allsky/tmp/sequence-timelapse.20240627/%04d.jpg: No such file or directory
av_interleaved_write_frame(): No space left on device
Error writing trailer of /home/mwallis/work/allsky/tmp/mini-timelapse.mp4: No space left on device
av_interleaved_write_frame(): No space left on device
Error writing trailer of /home/mwallis/work/allsky/tmp/mini-timelapse.mp4: No space left on device'.
==============================================
Links in '/home/mwallis/work/allsky/tmp/sequence-timelapse.20240623' left for debugging.
Remove them when the problem is fixed.

• I had been trying to fix an issue with the Overlay, but that shouldn't have changed Timelapse
• See above for msg trace.

Log / configuration files

* ~/allsky/config/config.sh

mwallis_allsky_config.sh.txt

* ~/allsky/config/settings.json

mwallis_allsky_settings.json

[EricClaeys]

I converted this to a Discussion since it's not a Bug - it's a question. Issues are only for bugs.

[EricClaeys]

Yes, the size of allsky/tmp can be changed, but let's first address the underlying problems that are filling tmp. 75 MB for allsky/tmp is more than enough under normal circumstances.
First, please set KEEP_SEQUENCE="false" in config.sh. It should very rarely be enabled, and then only for certain debugging.
Then run:

cd ~/allsky/tmp
du -h -s *

and post the results. It will show how much space each item in the tmp directory is taking. Typically, nothing is more than a couple MB.
Now, remove the unused timelapse sequences:

rm -r sequence*

Next step will depend on what's in allsky/tmp.

Are you trying to create a daily timelapse, or a "mini" timelapse? A daily timelapse includes all the images in a day's directory (e.g., ~/allsky/images/20240623) whereas a mini timelapse only contains the last few dozen or so images, and is recreated every few images.

[kf6spf]

𝛑  27 Jun 2024 @ 21:16:55 mwallis@allsky
/home/mwallis/allsky/tmp 502
$ dush *
0	aborts
4.0K	allskydb.py
4.0K	capture_args.txt
4.0K	flowtimings
84K	image.jpg
4.0K	notification_log.txt
4.0K	__pycache__
𝛑  27 Jun 2024 @ 21:16:59 mwallis@allsky
/home/mwallis/allsky/tmp 503
$ rm -r sequence*
rm: cannot remove 'sequence*': No such file or directory
𝛑  27 Jun 2024 @ 21:17:42 mwallis@allsky
/home/mwallis/allsky/tmp 504
$ ll
total 100
drwxrwxr-x  5 mwallis   180 Jun 27 13:23 ./
drwxr-xr-x 15 mwallis  4096 Jun 27 11:02 ../
drwxrwxr-x  2 mwallis    40 Jun 27 12:14 aborts/
-rw-r--r--  1 mwallis    75 Jun 27 13:23 allskydb.py
-rw-r--r--  1 mwallis  1519 Jun 27 12:14 capture_args.txt
drwxr-xr-x  2 mwallis    60 Jun 27 12:28 flowtimings/
-rw-r--r--  1 mwallis 82897 Jun 27 13:23 image.jpg
-rw-r--r--  1 mwallis    33 Jun 27 12:14 notification_log.txt
drwxr-xr-x  2 mwallis    60 Jun 27 13:24 __pycache__/
𝛑  27 Jun 2024 @ 21:17:54 mwallis@allsky
/home/mwallis/allsky/tmp 505
$

I was creating regular all-day Timelapse's but started to get the No room error. I tried getting mini-time lapses, but got the same results.

[kf6spf]

After restarting (with KEEP_SEQUENCE="false"), I get:

Jun 27 21:22:44 allsky allsky[5950]: ==========
Jun 27 21:22:44 allsky allsky[5950]: === Starting nighttime capture ===
Jun 27 21:22:44 allsky allsky[5950]: ==========
Jun 27 21:22:44 allsky allsky[5950]: -----
Jun 27 21:22:44 allsky allsky[5950]: STARTING EXPOSURE at: 2024-06-27 21:22:44   @ 20.0 sec
Jun 27 21:22:44 allsky allsky[5950]:  > Running: libcamera-still --output '/home/mwallis/work/allsky/tmp/image-20240627212244.jpg' --timeout 1 --nopreview --width 4656 --height 3496 --shutter 20000000 --analoggain 8 --quality 90

Jun 27 21:23:07 allsky allsky[5950]:   > Got:    shutter_us: 20.0 sec, gain: 8.000, mean: 0.539, target mean: 0.200, diff (target - mean): -0.339
Jun 27 21:23:07 allsky allsky[5950]:   > New mean target: 0.539, mean_forecast: 0.878, mean_diff (newMean - target mean): 0.339, idx=0, idxN1=2
Jun 27 21:23:07 allsky allsky[5950]:   > fast forward ExposureChange now 244 (meanDiff=0.339 > 1.75*threshold=0.175)
Jun 27 21:23:07 allsky allsky[5950]:   > Next image:  exposure time: 59.3 sec, gain: 1.000
Jun 27 21:23:07 allsky allsky[5950]:                  mean: 0.539 (diff from target: 0.339), Exposure level: 212 (minus 262: -50)
Jun 27 21:23:07 allsky allsky[5950]:   >>>> Skipping this frame.  0 left to skip
Jun 27 21:23:07 allsky allsky[5950]: -----

The "--width 4656 --height 3496" is NOT the settings I have in the config file.

[EricClaeys]

@kf6spf, you don't have the width and height set for images in the config file. Only for timelapse. For RPi cameras, Allsky needs to specify the width and height, and those are the correct numbers for the Arducam 16 MP camera.