Add caproto Zebra IOC #2

mrakitin · 2024-02-14T17:18:56Z

…t in CI

…tuptools to an older version; more verbose test output

act -W .github/workflows/ci.yml -j checks --matrix python-version:3.11 NOTE: Python 3.12 build fails on a datetime/TZ issue. | OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k | ImportError while loading conftest '/Users/mrakitin/src/NSLS-II/SRX/srx-caproto-iocs/tests/conftest.py'. | tests/conftest.py:13: in <module> | from srx_caproto_iocs.base import OphydDeviceWithCaprotoIOC | /opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/srx_caproto_iocs/base.py:11: in <module> | from caproto import ChannelType | /opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/caproto/__init__.py:2: in <module> | from ._utils import * | /opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/caproto/_utils.py:29: in <module> | from ._dbr import SubscriptionType | /opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/caproto/_dbr.py:17: in <module> | from ._constants import (EPICS2UNIX_EPOCH, EPICS_EPOCH, MAX_ENUM_STATES, | /opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/caproto/_constants.py:11: in <module> | EPICS_EPOCH = datetime.datetime.utcfromtimestamp(EPICS2UNIX_EPOCH) | E DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).

mrakitin · 2024-02-15T15:53:43Z

I've opened caproto/caproto#838 to report the issue we are observing with Python 3.12.

add `suid` to filename formatter

…nt_dataset` method

hyperrealist

Overall looks good.
I have some general suggestions in the comments.

Also, it seems we have from __future__ import annotations at the beginning of every file, but we are not using type hints much, if not at all.

hyperrealist · 2024-02-20T16:33:34Z

.github/workflows/ci.yml

@@ -57,13 +63,34 @@ jobs:
          python-version: ${{ matrix.python-version }}
          allow-prereleases: true

+      - name: Set env vars
+        run: |
+


Should we add a set -x here?

It would not hurt. I copied this piece from other workflows where we don't use set for the step, but it's OK to add.

hyperrealist · 2024-02-20T17:08:37Z

scripts/test-file-saving.sh

+
+data_dir="/tmp/test/$(date +%Y/%m/%d)"
+
+if [ ! -d "${data_dir}" ]; then


if block is probably not needed since you are using mkdir -p

Right, it can be eliminated.

hyperrealist · 2024-02-21T16:34:48Z

src/srx_caproto_iocs/utils.py

+            frame_num = dataset.shape[0]
+
+        # https://docs.h5py.org/en/stable/swmr.html
+        h5file_desc.swmr_mode = True


Pretty cool feature, didn't know it existed! If it may help someone I'd add a comment here to mention that older HDF5 libraries do not support reading those files

OK, please add the comment.

hyperrealist · 2024-02-21T17:10:09Z

tests/test_base_ophyd.py

+import shutil
+import time as ttime
+import uuid
+from pathlib import Path


This import is not being used for testing anything

It's used in the test_base_ophyd_templates test a number of times.

hyperrealist · 2024-02-21T21:42:05Z

tests/test_base_ophyd.py

+    date = now(as_object=True)
+    write_dir_root = Path(tmpdirname)
+    dir_template = f"{write_dir_root}/{date_template}"
+    write_dir = Path(date.strftime(dir_template))


I am not sure if we need to "manually" strftime here. Shouldn't that happen when we stage base_ophyd_device?

It's to make the directory structure on the next line. What we pass to the ophyd object and then to the caproto IOC is dir_template.

mrakitin · 2024-02-22T00:24:27Z

Also, it seems we have from __future__ import annotations at the beginning of every file, but we are not using type hints much, if not at all.

That import is automatically added by pre-commit. I didn't want to fight with it, so just left it as is.

…t`, `cainfo`, etc.)

…ant test

* replaces unsupport characters with underscores _ * github artifact upload does not support the special chars: - Double quote " - Colon : - Less than < - Greater than > - Vertical bar | - Asterisk * - Question mark ? - Carriage return \r - Line feed \n * note that \r, \n are included in whitespace regex symbol \s

String/Char caproto IOC with demo tests

…esky side should do the logic and call the saver when done

Dev/tests for Zebra Saver IOC

mrakitin · 2024-05-17T19:19:15Z

We are going to merge it to test at SRX.

mrakitin added 19 commits February 13, 2024 15:33

Add dependencies and update pre-commit config

d4eb5cd

Create base classes and utility functions

517213c

CI: run ci workflow on all pushes

bd46a1a

CI: Exclude pypy tests

f393c63

TST: add ophyd-based test for Zebra Caproto IOC

cc80d49

Working ophyd tests

e320f28

More fixes for the tests

a61223f

TST: add a caproto IOC fixture to be able to start and test against i…

ca99a7e

…t in CI

CI: disable Win/OSX builds; more python versions to test with; pin se…

d2d25e2

…tuptools to an older version; more verbose test output

TST: add broadcast address a smarter way

e151089

CI: Do not run tests twice on a PR from the same repo's branch

7a9944a

TST: print caproto stdout and stderr

d7ec78c

Use base IOC/ophyd in CI tests

09a08ca

Add saving capabilities to the base IOC

eb36115

Fixed saving capabilities

d29d442

Make it work on MacOS

42f073b

Working saving example of multiple frames with ophyd

f18d187

Add scikit-image to the dependencies

fbf17aa

mrakitin and others added 10 commits February 15, 2024 14:43

Add missing skimage dependency to pull the dataset

384c001

PKG: apply setuptools limitation better

7dbbaca

CI: configure PyPI uploading

8248509

Add convenience script to run act locally

69d4306

add suid to filename formatter

15bb697

fix bug where uid not generated if only suid is used in file_name

a4e70d4

fix typo, make more readable

1d3aae3

Merge pull request #3 from hyperrealist/add-zebra-ioc

f63e243

add `suid` to filename formatter

CNL: remove irrelevant comment and add a docstring to the `_get_curre…

618cbb3

…nt_dataset` method

TST: add input arg for a number of frames to the testing script

1ac8b6f

hyperrealist added 2 commits February 21, 2024 12:20

add test parameters to test hidden spaces in write dir

e75be6c

fix lint errors from e75be6c

a33c735

hyperrealist requested changes Feb 21, 2024

View reviewed changes

mrakitin and others added 13 commits February 22, 2024 03:28

WIP: test with string IOC

9724d19

STY: fix linting errors

dc98043

Parametrize the script to start different caproto IOCs

cfe47e2

TST: cainfo and caput tests for string caproto IOC

9f1f5c6

TST: fix the blocking issue in the tests

2d05e76

CI: install epics-base package from conda-forge (for caget, `capu…

9c9f539

…t`, `cainfo`, etc.)

Minor tweaks and addressing review comments from PR#2

590e554

CI: use micromamba for the python env

ab77f7f

Change write_dir and file_name PV types to CHAR and fix the relev…

def5ded

…ant test

Minor improvements

ccca4d3

update test_base_ophyd.py to use the correct path sanitizer

571539e

Merge pull request #4 from NSLS-II-SRX/more-caproto-ioc-flavors

9abfa20

String/Char caproto IOC with demo tests

mrakitin requested a review from hyperrealist May 3, 2024 15:03

mrakitin and others added 8 commits May 14, 2024 18:46

Dev updates for the Zebra IOC

af0f986

WIP: can save enc1 1-D data

0afd370

Vendor acquire/saver methods from the base IOC in the Zebra one

6524861

Prototyped IOC to accept the a dict of arrays to save - the ophyd/blu…

adb96a3

…esky side should do the logic and call the saver when done

Add support for SIS scaler exporting

d7ea796

Fix enum default value for the dev_type

afdb488

Clean up irrelevant code

246d885

Merge pull request #11 from NSLS-II-SRX/add-zebra-ioc-dev

9dc73ff

Dev/tests for Zebra Saver IOC

mrakitin marked this pull request as ready for review May 17, 2024 19:19

mrakitin merged commit d5ed90d into main May 17, 2024
1 of 3 checks passed

mrakitin deleted the add-zebra-ioc branch May 17, 2024 19:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add caproto Zebra IOC #2

Add caproto Zebra IOC #2

mrakitin commented Feb 14, 2024 •

edited

Loading

mrakitin commented Feb 15, 2024

hyperrealist left a comment

hyperrealist Feb 20, 2024

mrakitin Feb 22, 2024

hyperrealist Feb 20, 2024

mrakitin Feb 22, 2024

hyperrealist Feb 21, 2024

mrakitin Feb 22, 2024

hyperrealist Feb 21, 2024

mrakitin Feb 22, 2024

hyperrealist Feb 21, 2024

mrakitin Feb 22, 2024

mrakitin commented Feb 22, 2024

mrakitin commented May 17, 2024


		data_dir="/tmp/test/$(date +%Y/%m/%d)"

		if [ ! -d "${data_dir}" ]; then

Add caproto Zebra IOC #2

Add caproto Zebra IOC #2

Conversation

mrakitin commented Feb 14, 2024 • edited Loading

mrakitin commented Feb 15, 2024

hyperrealist left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrakitin commented Feb 22, 2024

mrakitin commented May 17, 2024

mrakitin commented Feb 14, 2024 •

edited

Loading