v0.8.2

What's Changed

• PRU now gets partially zeroed buffer-segments
• PRU had a race-condition with a loose mutex resulting in keeping old gpio-samples
• python warns on full gpio-buffer (as it can only hold ~16k entries in 100 ms)
• python warns if first or last timestamp of gpio-buffer is out of scope of outer buffer-period
• hw cape - add errata-list
• vsrc - add datatype to determine state-variables
• CalibrationPair - add units
• split pru0-shepherd-fw into hrv & emu -> kModule, shepherd-sheep, playbooks, workflows adapted
• pru-vsource - add feedback to harvester
• pru-harvester - add feedback & extrapolation for cv-harvester
• tested: pytest sheep, pytest herd, playbook dev_rebuild_sw.yml

Full Changelog: v0.8.1...v0.8.2

v0.8.1

What's Changed

• big bugfix release
• sheep
  • limit pru-warning count
  • fix forced stopping of ntp-service
  • improve debug-output
  • fix ingestion of ivcurves (windowsize was not propagated)
  • fix copy of emu-input into shared mem (off-by-one-error)
• herd
  • query for alive status of testbed (all hosts responding)
  • more robust unittesting
  • add automated benchmark in software\shepherd-herd\test_manual\
• python in general
  • add progress-bars to long processes
  • remove progress-bar after task finishes (most)
  • don't limit pandas to <v2 anymore
  • prepare for py313
• pru vsourve & harvester
  • fix residue feature
  • remove limiting-behavior of boost-regulator
  • fix calculation of window_size for individual usecases
  • ivcurve - cutout measurements during big step
  • improve code-quality (cleaner fetching of emu-input and special math-functions are easier to understand)
• harvesting ivcurves
  • fix max age of samples
  • improve initial interval_step to intake two whole ivcurves before reset
  • improve VOC-harvester
• python module "shepherd-pru" interfaces pru-c-code via ctypes
  • harvesting & emulation can be done
  • benchmarking revealed some bugs
• remove cython-playground (in favor of ctypes-implementation)
• ansible: remove py-packages before install
• extend ruff and fix ~ 200 linting-errors
• tested: pytest sheep, pytest herd, playbook dev_rebuild_sw.yml

Full Changelog: v0.8.0...v0.8.1

v0.8.0

What's Changed

• linux
  • optimize for real-time kernel, moved to branch
  • make phc2sys & ptp4l more reliable
  • test and optimize for debian 12.6 bookworm
    • harvest cpu-usage drops from 69% (py310) to 61% (py311)
• kernel-module
  • cleanup, optimize
  • more futureproof (use ktime_get_X() instead of getnstimeofday())
  • remove mutex (seems to have deadlocked sometimes)
  • fix hrtimer_forward()-usage (source for instability)
  • more const-correctness
  • can now change firmware of both PRUs
  • more pru-messages are handed to python (for logging)
  • warn/error if sync-config is wrong
• python
  • reduce load by 50 % (omit timestamps & change compression)
  • IV-Stream -> add meta-data for stored buffer-segment
    • meta: buffer-timestamp, sample-count, pru-util mean & max
    • this can reconstruct timestamp-stream after the measurement
  • refactored h5writer into smaller modules (monitor and recorder threads)
  • added monitors for uart, pru-usage, ptp-status
  • replaced setup.cfg by pyproject.toml with ini2toml
  • added heartbeat-messages during operation
  • fix launcher - was misbehaving with 100% cpu-usage
  • added watchdog-reset service - functionality was in launcher before
  • reduced ram-usage of services
  • removed scipy-dependency (less ram usage, faster startup)
  • disable NTP before starting a measurement
• herd
  • resync - give info about time-diff
  • improve interpretation of sheep-exit-codes
• ansible
  • major overhaul
  • more removed packages during cleanup
  • safer kernel-downgrading
  • faster code
  • safer firmware-removal
  • disable unwanted services
• toolchain: replace isort, black, flake8, pylint by ruff
• debug
  • add option to generate kernel gpio edges (trigger_loop_callback() in pru_sync_control.c)
  • allow analyzing recorded gpio sync traces (software/time_sync_analyzer)
• update floorplan of testbed
• add tooling to allow analyzing timesync-behavior (software/time_sync_analyzer)
• add current hardware design files
• major overhaul documentation
• tested (fully): pytest sheep, pytest herd, ansible install
• ready-to-use image will be created - look in getting-started guide

Full Changelog: v0.7.1...v0.8.0

v0.7.1

What's Changed

• python
  • speed improvements, linting, simplifications
  • optimize for py310..py312
  • safer sheep-shutdown
  • fully type-hinted
  • more error-catching
  • more responsive monitor-threads
  • identify and try to avoid deadlocks and infinite loops
• improve timesync reliability (phc2sys & ptp4l)
• forbid unsafe ssh cypher
• update deps

Full Changelog: v0.7.0...v0.7.1

v0.7.0

What's Changed

• pru-firmware:
  • improve blind-spots of gpio-sampling
  • fix compiler warning (cgt & gcc)
  • measure timings
  • bugfixes
• sheep
  • refactoring of monitors
  • repair ptp & dmesg / kernel logging
  • repair uart-monitor
  • allow to record stdout of sheep
  • set gpio-direction to input for now
  • redo logging-system
• herd
  • more reliable (needs to enter context now)
  • tests availability of nodes / sheep
  • redo logging-system
• doc
  • improve details
  • add info about testbed and subprojects
• ansible
  • tweaks, speedups
  • fixes
  • (timesync) services are now more reliable
  • update kernel version
  • lots of linting
• cal
  • lots of tweaks and fixes

Full Changelog: v0.4.5...v0.7.0

v0.4.5

What's Changed

• add option to build an inventory
• herd, sheep, cal-tool: fully integrate datalib
• sheep
  • improve exit behavior
  • various small improvements, refactorings
  • speed-improvements through tracing
• ansible
  • safer pipes
  • bugfixes
• pru-programmer: bugfixes, add direction pin, test with new target
• integrate other shepherd-projects as submodules
• ubuntu 22.04 now default distribution for sheep (py 3.10)
• tested: shepherd-sheep & -herd

Full Changelog: v0.4.4...v0.4.5

v0.4.4 - with new ready to use image

Changelog

• ansible
  • fix shell vs command usage
  • fix kernel-change-bug that bricked the image
  • fix bugs and improve code
  • allow deeper cleanup
  • increase compression for ready-to-use shp-image
• improve documentation

Tested

• sheep testbench
• herd testbench
• ansible bootstrap and deploy
• ansible acquire image

Full Changelog: v0.4.3...v0.4.4

v0.4.3

Changelog

• kernel module
  • fix race condition for use after free while unloading module -> made unittests impossible to run through
  • silence wrong errors
  • better error-msg
  • refactoring
  • fix potential mem-leaks
  • decouple sub-services
• programmer
  • add support for direction pins on higher levels (TODO: handler in each programmer)
  • add support for changing pru-firmware in python-package, kmodule, ansible
• calTool
  • improvements and bugfixes
  • unify cli.interface with reference to shepherd-package
  • add profiler for analog frontend
• HerdTool
  • add support for programmer
  • unify cli.interface with reference to shepherd-package
  • add unittests
  • modularize & transform into lib
  • parallelize important routines
  • improvements and bugfixes
• ansible playbooks
  • linter proposed changes
  • improve large installs (~10 nodes) - fix various timeouts & pitfalls
• cython-vsource
  • fix code to allow compiling
  • code more modular and explicit
• py-package
  • allow installation on x86 (with warnings)
• sheep
  • dhcp registration - switch from machine-id to MAC for reliable IP
  • add playbook & support for nfs datastorage from ZIH
• github actions: tests use python 3.9 - 3.11 now (instead of 3.8+, due to dependencies)
• update dependencies
• doc: add simplified installation guide with video
• remove travis
• currently open student projects
  • extend unittests for new virtual source
  • port kernel module to 5.10 (from 4.19)
  • reimplement memory-interface for higher performing buffer-exchanges
  • evaluate UIs for viewing shepherd-data

Full Changelog: v0.4.2...v0.4.3

Tested on

• shepherd-sheep testbench
• shepherd-herd testbench

add pru-programmer and preliminary gcc-support

Changes:

• merge programmer-branch (thanks to @geissdoerfer)
  • currently alpha, as shepherd is not able to switch between pru-firmwares
• add gcc-support (thanks to previous work of @fedy0)
  • make switches to gcc as soon as PRU_GCC is defined
  • currently beta! as pru0-firmware does not fit into IMEM
• shepherd_herd
  • fix and modernize shepherd-herd
  • add testbench
  • refactor code
  • extend functionality: distribute files, start / stop / check measurements
• change interface for CLI
  • enable_io (was set_target_io_lvl_conv or similar)
  • io_target and pwr_target (was io_target_a and b, or io_sel_target_a, same for pwr
• sheep detects missing kernel-module and missing sudo and fails early
• added doc for herd
• added doc for shepherd-v2 improvements
• ansible
  • refactor role "dev-host" into sheep
  • add installer for GCC-Toolchain
  • remove duplicate code (dev_rebuild can now be done by sheep-role)
  • modernize some loops
  • fix reboot-hang
• github action
  • now builds pru-firmwares (including programmer)
  • now also upload shepherd_herd to pypi

tested on:

• shepherd-sheep testbench
• shepherd-herd testbench
• ansible install on 11 nodes

trouble:

• PRUs seem more unstable in the big herd of 11 (sync is oscillating, state is rarely stuck in reset)
• switching to programmer-firmware is not implemented yet

compile-sizes (CGT vs GCC)

• pru0-fw 85020 byte, 91660 byte (IMEM overflow)
• pru1-fw 63820 byte, 42644 byte
• prog-swd 80108 byte, 68176 byte
• prog-sbw 83620 byte, 72900 byte

intermediate release

Changelog:

• fix bugs in virtual source
• fixed potential bugs in c-code (mostly kModule)
• fixed buffer overflow and added canaries to important structs
• add initial cythonized virtual-source (pru-code) (thanks @WiresharkIO)
• improved logging-performance and cleaned up code
• fixed doc-builder

maintenance:

• primary branch is now "main"
• added code of conduct
• ansible-scripts: linter, reformat, cleanup, restructure
• add pre-commit and github-actions, for
  • building documentation
  • test and build python-packages
  • linter, format most source-code
  • spellcheck