Looking for an OCB (or whatever filetype you think best) format description

[seeigecannon]

I am working on reverse engineering my Hitachi 7000 HPLC from the ground up (ripping original electronics and putting my own in). Can you guys link me to something describing how to format the data I collect into a file for OpenChrom to use? Thanks.

[eselmeister]

Please provide some Hitachi 7000 HPLC raw files for testing purposes.

[seeigecannon]

00000001 - Copy.DAD.CSV (Note: remove the CSV file extension)
Thank you for looking into this. Here is a file from the original HPLC hardware and interface. Later on tonight I can make a breakdown of it. However, I am gutting all of the original electronics out of this machine and putting in custom hardware, so I can customize the output data to whatever you want. For example, I am already able to get the full spectrum out of the DAD at 40Hz when the original hardware only supported 10Hz for half the spectra. The original software was also written for Windows 3.1 and had a hard file size cap of 8MB. I will be controlling everything via Python, so will have no such limitations. So if you sent me some kind of descriptor for how an ideal HPLC file works I should have no problem conforming to that.

If you were able to add support for the original file though I am sure that would be appreciated as there are still a few of these out in the world, and the software is horrible due to the hardware limitations at the time.
Here are a few things I remember off the top of my head:
7 bytes after the word "Boundary" is where the DAD data starts. Each pixel is 3 bytes, but is interpreted as a 32-bit unsigned (I believe) int, but the first byte is assumed as a 0x00.
PressPointsA+0x00060000007404, the 0x7404 is a 16-bit unsigned int, little endian that indicates the pressure profile is 1140 samples long. After PressureProfileA+7 bytes is the pump pressure chart, each pair of bytes after that is a 16-bit int in bar. The temperature profile (TempPoints, TempProfile) is much the same, but each byte pair has an implied decimal (example, 349 is 34.9C).

I will need to go back and figure out a bunch of other data points for you later tonight.

[seeigecannon]

Ok, looking through file now:
I can't find the injection volume as they are all showing 0s in the file. The injection was 10uL though.
Bandwidth+0x00060000000A00, the 0A indicates a DAD bandwidth of 1 (hex 10 divided by 10).
MonWV+0x0006000000FA00, the 0xFA is the wavelength that is selected to be displayed, 250nm.
WV_RangeMin+0x0006000000C800, lowest wavelength recorded, C800 = 200nm
WV_RangeMax+0x00060000009001, highest wavelength recorded, 0x9001 = 400nm
Samp_Period+0x00060000001000, time between points on the DAD recording, 0x1000 = 16 = 1600ms.

I am not seeing anything that indicates how many bytes are in a vertical column, but I suspect it would be ((WV_RangeMax-WV_RangeMin)/Bandwidth)*3.

I can't make heads or tails of the pump profile at the moment.

[eselmeister]

Lovely, I'm already able to extract information from the given raw file:

[eselmeister]

Give me some time to play with it. I love to see that you tinker with the Hitachi 7000 instrument and get more out of it as initially possible. I will give you a feedback approx. in January how to best save and export the data from the instrument with your specific setup.

[seeigecannon]

Fantastic. Thank you.

I have not yet decided how I will be going forward on the detector side. At the moment I am using an arduino to pull the 16-bit parallel output of the ADC, but if I am feeling adventurous I might implement my own ADC with a better noise floor and maybe run a 24-bit. There is also a LOT of noise in the +/- 12V lines going to the op-amps that sit right next to the photo detector chip, so I am thinking of the best way of dealing with that. Considering this is a 30yo instrument it could just be dry caps. If I design a PCB for a new ADC I will also use linear power supplies for the +/-12V rails.

This has been a fun project, and I am learning a lot.

[eselmeister]

You are a genius!

[eselmeister]

I will additionally create a specific import converter for data that is exported by your system. You are then able to adjust it to your needs.

[LouisPM]

Love your tinkering project and just wanted to chime in.

If you are looking for an affordable high precision ADC, I suggest to have a look at the ARD-LTC1867 or ARD-LTC2499 by Iowa Scaled Engineering. These 16 and 24 bit ADCs are shield extension for the Arduino board. With a low noise external power supply that provides a reference voltage of ~ 5V, one can obtain a resolution in the (sub-) microvolt range. However, that might not be necessary, as these boards come with decent on-board voltage regulators.
I have been using ARD-LTC2499 boards myself for the real-time data collection from several old analog analytical devices.
The code for these boards is available on GitHub.

Best of luck!

[seeigecannon]

Oh nice. Thank you for this. I have done a bit of looking around for ADCs, but have never tried to spec one out before, so I pretty quickly got lost. I will certainly take a closer look at this, or at least use this as a starting point. The ADC the original DAD is using is a ADS7805, and one nice thing about this is the detector board sends a pulse when a pixel should be read which feeds straight into pin 24 of the ADC. If I take over the clock generation though I am sure I would be able to just set a delay or a digital gate of some sort for when to actually take readings. That chip you linked is so fast that I should also be able to average multiple readings over a single pixel to help further reduce noise.

I was planning on releasing everything open source when I finally had something worth showing, but I think I will go ahead and publish what I have over Christmas. My code will make your eyes want to bleed, but in my defense, I am a hardware guy lol.

[seeigecannon]

That would be great because I also have a bid on a HPLC Refractive Index detector. I have no idea what use it could have for me, but it looks like I will get it for only $10, so I might as well.

…

On Wed, Dec 18, 2024, 1:18 AM Philip Wenig ***@***.***> wrote: I will additionally create a specific import converter for data that is exported by your system. You are then able to adjust it to your needs. — Reply to this email directly, view it on GitHub <#578 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOR364A2NCSEWTBI7526WD2GEVWXAVCNFSM6AAAAABTT4PDECVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNRQGIZTONY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[eselmeister]

@seeigecannon My colleague @Mailaender mentioned, that the simplest way for you to write and import data in OpenChrom is to use the *.arw format.

[eselmeister]

Here is the above displayed example file:
HPLC-DAD.zip

[eselmeister]

Details of the converter can be viewed here:
https://github.com/eclipse-chemclipse/chemclipse/blob/develop/chemclipse/plugins/org.eclipse.chemclipse.wsd.converter.supplier.arw/src/org/eclipse/chemclipse/wsd/converter/supplier/arw/io/ChromatogramReader.java

[seeigecannon]

This is exactly what I was looking for. Thank you. Now that I know what the converters look like I can look at the others to get a better idea of how the system works internally as I would like to also record things like temperature, pressure, solvent ratio, etc.

[eselmeister]

Perfect. Temperature, pressure, solvent ratio ... are currently now displayed in the chromatogram editor. But I could extend it to also display additional data.

[seeigecannon]

I am talking on the import side. The /arw/io/ChromatogramReader.java file has functions for importing wavelengths, signal, and time, but nothing else.
What I need to do though is just spend a day looking through the code as well as learning the basics of Java. I have a week off for Christmas where I will be able to work on this.

[eselmeister]

IMHO, the *.arw format is quite simple and offers no option to store additional detector data, like temperature. You could store the temperatures in a separate files, e.g.:

time milliseconds, temperature °C

[seeigecannon]

Ah. This is good to know. If I could get my hands on a supported file from Agilent or something I could try emulating that.
Does the OCB format support multiple data streams? I haven't found the thing that reads that yet to check for myself. I did find some OCB demo files though, and if I knew what is and is not important in the file that doesn't look too bad to write.

If there is a preferred native format for OpenChrom though I could just emulate that.

[eselmeister]

OCB supports to store n-chromatograms. You can open a chromatogram in the editor and add additional chromatograms as references. Then save it in *.ocb format. It's a ZIP container with subfolders.

[seeigecannon]

Oh, that goes a long way to explain why I was having a hard time with figuring out OCB files in a hex editor. Chromatogram4-1501.ocb looks like it could have been made by a DAD, but it seems to be calling it a WSD. What does WSD stand for in this context? The only thing I could really find was it could be "window selection diagrams", but that doesn't sound right. There is a TIC file in there too, so I am wondering if this was made with a LC/MS. I will have a lot of time to play with this next week though, so I should be able to get a lot of this figured out without bothering you so much. I do appreciate your patients with me though.

[Mailaender]

WSD stands for wavelength selective detector. It stems from MSD being the Agilent trade name for mass selective detector, but they were so popular that you colloquially say MSD when you mean any MS detector. So OpenChrom adopted this shorthand and invented CSD (current selective detector) for GC-FID and WSD for wavelength selective detector to match. There is also VSD now for vibrational spectroscopy detector which are less common.

OCB was designed with MS in mind and everything else as an afterthought. It is also not really made for adoption by thirdparties but really only as a backwards-compatible archival file format that matches the OpenChrom feature set.

We have limited support for mzML which has DAD support (again as an afterthought) but it is meant to be vendor independent and for raw data exchange so checkout org.eclipse.chemclipse.wsd.converter.supplier.mzml as it may be more suitable.