Converting physiological recordings

[tjhendrickson]

Summary

I cannot seem to figure out how to convert physiological recordings present within the DICOM folder from a neuroimaging dataset. The physiological recordings are organized as one DICOM file within its own folder and with the same naming scheme as the BOLD it relates to with "PhysioLog" suffixed. The heuristic script below failed to find this and even when I run convertall.py as the heuristic the physiological recordings are not found.

Any ideas?

• heuristic

import pdb

def create_key(template, outtype=('nii.gz','dicom'), annotation_classes=None): #), annotation_classes=None):
    if template is None or not template:
        raise ValueError('Template must be a valid format string')
    return (template, outtype, annotation_classes)


def infotodict(seqinfo):
    """Heuristic evaluator for determining which runs belong where

    allowed template fields - follow python string module:

    item: index within category
    subject: participant id
    seqitem: run number during scanning
    subindex: sub index within group
    """ 
    t1 = create_key('sub-{subject}/{session}/anat/sub-{subject}_{session}_rec-{rec}_run-{item:02d}_T1w')
    t2 = create_key('sub-{subject}/{session}/anat/sub-{subject}_{session}_rec-{rec}_run-{item:02d}_T2w')

    rest_ten_minute = create_key('sub-{subject}/{session}/func/sub-{subject}_{session}_task-rest_acq-10minute_run-{item:02d}_part-{part}_bold')
    rest_ten_minute_sbref = create_key('sub-{subject}/{session}/func/sub-{subject}_{session}_task-rest_acq-10minute_run-{item:02d}_sbref')
    rest_ten_minute_physio = create_key('sub-{subject}/{session}/func/sub-{subject}_{session}_task-rest_acq-10minute_run-{item:02d}_physio')

    rest_sixteen_minute = create_key('sub-{subject}/{session}/func/sub-{subject}_{session}_task-rest_acq-16minute_run-{item:02d}_part-{part}_bold')
    rest_sixteen_minute_sbref = create_key('sub-{subject}/{session}/func/sub-{subject}_{session}_task-rest_acq-16minute_run-{item:02d}_sbref')
    rest_sixteen_minute_physio = create_key('sub-{subject}/{session}/func/sub-{subject}_{session}_task-rest_acq-16minute_run-{item:02d}_physio')
    
    spinecho_fieldmap_ME_bold = create_key('sub-{subject}/{session}/fmap/sub-{subject}_{session}_acq-SpinEchoME_dir-{dir}_run-{item:02d}_epi')
    spinecho_fieldmap_ME_physio = create_key('sub-{subject}/{session}/fmap/sub-{subject}_{session}_acq-SpinEchoME_dir-{dir}_run-{item:02d}_physio')
    
    gradientecho_fieldmap_ME_bold_mag = create_key('sub-{subject}/{session}/fmap/sub-{subject}_{session}_acq-GradientEchoME_dir-{dir}_run-{item:02d}_magnitude')
    gradientecho_fieldmap_ME_bold_phase = create_key('sub-{subject}/{session}/fmap/sub-{subject}_{session}_acq-GradientEchoME_dir-{dir}_run-{item:02d}_phase')
    gradientecho_fieldmap_ME_bold_sbref = create_key('sub-{subject}/{session}/fmap/sub-{subject}_{session}_acq-GradientEchoME_dir-{dir}_run-{item:02d}_sbref')
    gradientecho_fieldmap_ME_bold_physio = create_key('sub-{subject}/{session}/fmap/sub-{subject}_{session}_acq-GradientEchoME_dir-{dir}_run-{item:02d}_physio')
    
    
    info = {t1: [], t2: [], 
            rest_ten_minute: [], rest_ten_minute_sbref: [], rest_ten_minute_physio: [],
            rest_sixteen_minute: [], rest_sixteen_minute_sbref: [], rest_sixteen_minute_physio: [],
            spinecho_fieldmap_ME_bold: [], spinecho_fieldmap_ME_physio: [],
            gradientecho_fieldmap_ME_bold_mag: [], gradientecho_fieldmap_ME_bold_phase: [], gradientecho_fieldmap_ME_bold_sbref: [], gradientecho_fieldmap_ME_bold_physio: []}

    for idx, s in enumerate(seqinfo):
        # retreive previous element in seqinfo
        if idx > 0:
            s_previous = seqinfo[idx-1]
        if idx - 1 > 0:
            s_previous_two = seqinfo[idx-2]
        # retreive next element in seqinfo
        if idx + 1 < len(seqinfo):
            s_next = seqinfo[idx+1]
        # retreive next next element in seqinfo
        if idx + 2 < len(seqinfo):
            s_next_two = seqinfo[idx+2]

        # find pre scan normalized anatomicals
        if (s.dim3 == 176) and ('NORM' in s.image_type):
            if 'ABCD_T1' in s.dcm_dir_name:
                rec = 'normalized'
                info[t1].append({'item': s.series_id, 'rec': rec})
            elif 'ABCD_T2' in s.dcm_dir_name:
                rec = 'normalized'
                info[t2].append({'item': s.series_id, 'rec': rec})
        # find resting state scans. Differentiate by mag or phase
        elif (s.dim4 > 5) and ('rest' in s.series_description):
            if '10MIN' in s.protocol_name:
                if s.image_type[2] == 'M':
                    if (s_next.dim4 > 5) and ('rest' in s_next.series_description) and (s_next.image_type[2] == 'P'):
                        part = 'mag'
                        info[rest_ten_minute].append({'item': s.series_id,'part': part})
                elif s.image_type[2] == 'P':
                    if (s_previous.dim4 > 5) and ('rest' in s_previous.series_description) and (s_previous.image_type[2] == 'M'):
                        part = 'phase'
                        info[rest_ten_minute].append({'item': s.series_id,'part': part})
            elif '16MIN' in s.protocol_name:
                if s.image_type[2] == 'M':
                    if (s_next.dim4 > 5) and ('rest' in s_next.series_description) and (s_next.image_type[2] == 'P'):
                        part = 'mag'
                        info[rest_sixteen_minute].append({'item': s.series_id,'part': part})
                elif s.image_type[2] == 'P':
                    if (s_previous.dim4 > 5) and ('rest' in s_previous.series_description) and (s_previous.image_type[2] == 'M'):
                        part = 'phase'
                        info[rest_sixteen_minute].append({'item': s.series_id,'part': part})
        # retreive field maps
        elif 'FieldMap' in s.series_description:
            if 'GEFieldMap' in s.series_description:  
                if 'AP' in s.series_description:
                    if s.image_type[2] == 'M':
                        if (s_next.dim4 >= 15) and ('GEFieldMap' in s_next.series_description) and (s_next.image_type[2] == 'P'):
                            info[gradientecho_fieldmap_ME_bold_mag].append({'item': s.series_id, 'dir': 'AP'})
                    elif s.image_type[2] == 'P':
                        if (s_previous.dim4 >= 15) and ('GEFieldMap' in s_previous.series_description) and (s_previous.image_type[2] == 'P'):
                            info[gradientecho_fieldmap_ME_bold_phase].append({'item': s.series_id, 'dir': 'AP'})
                elif 'PA' in s.series_description:
                    if s.image_type[2] == 'M':
                        if (s_next.dim4 >= 15) and ('GEFieldMap' in s_next.series_description) and (s_next.image_type[2] == 'P'):
                            info[gradientecho_fieldmap_ME_bold_mag].append({'item': s.series_id, 'dir': 'PA'})
                    elif s.image_type[2] == 'P':
                        if (s_previous.dim4 >= 15) and ('GEFieldMap' in s_previous.series_description) and (s_previous.image_type[2] == 'P'):
                            info[gradientecho_fieldmap_ME_bold_phase].append({'item': s.series_id, 'dir': 'PA'})
            elif 'SpinEchoFieldMap' in s.series_description:
                if s.dim4 == 15: 
                    if 'AP' in s.series_description:
                        info[spinecho_fieldmap_ME_bold].append({'item': s.series_id, 'dir': 'AP'})
                    elif 'PA' in s.series_description:
                        info[spinecho_fieldmap_ME_bold].append({'item': s.series_id, 'dir': 'PA'})
                    
        # retreive sbref images
        elif 'SBRef' in s.series_description:
            if 'rest' in s.series_description:
                if '10MIN' in s.protocol_name:
                    if (s_next.dim4 > 5) and ('10MIN' in s_next.series_description) \
                    and (s_next.image_type[2] == 'M') and (s_next_two.dim4 > 5) \
                    and ('10MIN' in s_next_two.series_description) \
                    and (s_next_two.image_type[2] == 'P'):
                        info[rest_ten_minute_sbref].append({'item': s.series_id})
                elif '16MIN' in s.protocol_name:
                    if (s_next.dim4 > 5) and ('16MIN' in s_next.series_description) \
                    and (s_next.image_type[2] == 'M') and (s_next_two.dim4 > 5) \
                    and ('16MIN' in s_next_two.series_description) \
                    and (s_next_two.image_type[2] == 'P'):
                        info[rest_sixteen_minute_sbref].append({'item': s.series_id})
            elif 'GEFieldMap' in s.series_description:
                pdb.set_trace()
                if 'AP' in s.series_description:
                    if s_next.image_type[2] == 'M' and (s_next.dim4 >= 15) and ('GEFieldMap' in s_next.series_description) and (s_next_two.dim4 >= 15) and ('GEFieldMap' in s_next_two.series_description) and (s_next_two.image_type[2] == 'P'):
                        info[gradientecho_fieldmap_ME_bold_sbref].append({'item': s.series_id, 'dir': 'AP'})
                elif 'PA' in s.series_description:
                    if s_next.image_type[2] == 'M' and (s_next.dim4 >= 15) and ('GEFieldMap' in s_next.series_description) and (s_next_two.dim4 >= 15) and ('GEFieldMap' in s_next_two.series_description) and (s_next_two.image_type[2] == 'P'):
                        info[gradientecho_fieldmap_ME_bold_sbref].append({'item': s.series_id, 'dir': 'PA'})
        # retreive physiological recordings
        elif 'PhysioLog' in s.dcm_dir_name:
            if 'rest' in s.series_description:
                if '10MIN' in s.protocol_name:
                    if (s_previous_two.dim4 > 5) and ('10MIN' in s_previous_two.series_description) \
                    and (s_previous_two.image_type[2] == 'M') and (s_previous.dim4 > 5) \
                    and ('10MIN' in s_previous.series_description) \
                    and (s_previous.image_type[2] == 'P'):
                        info[rest_ten_minute_physio].append({'item': s.series_id})
                elif '16MIN' in s.protocol_name:
                    if (s_previous_two.dim4 > 5) and ('16MIN' in s_previous_two.series_description) \
                    and (s_previous_two.image_type[2] == 'M') and (s_previous.dim4 > 5) \
                    and ('16MIN' in s_previous.series_description) \
                    and (s_previous.image_type[2] == 'P'):
                        info[rest_sixteen_minute_physio].append({'item': s.series_id})
            elif 'GEFieldMap' in s.series_description:
                if 'AP' in s.series_description:
                    if s_previous_two.image_type[2] == 'M' and (s_previous_two.dim4 >= 15) and ('GEFieldMap' in s_previous_two.series_description) and (s_previous.dim4 >= 15) and ('GEFieldMap' in s_previous.series_description) and (s_previous.image_type[2] == 'P'):
                        info[gradientecho_fieldmap_ME_bold_physio].append({'item': s.series_id, 'dir': 'AP'})
                elif 'PA' in s.series_description:
                    if s_previous_two.image_type[2] == 'M' and (s_previous_two.dim4 >= 15) and ('GEFieldMap' in s_previous_two.series_description) and (s_previous.dim4 >= 15) and ('GEFieldMap' in s_previous.series_description) and (s_previous.image_type[2] == 'P'):
                        info[gradientecho_fieldmap_ME_bold_physio].append({'item': s.series_id, 'dir': 'PA'})
                               
    return info

Platform details:

Choose one:

• [X ] Local environment

python = 3.9.5
OS Ubuntu Xenial-20210429

• Container

• Heudiconv version:

heudiconv = 0.9.0

[pvelasco]

@tjhendrickson , could you show an example of the DICOM folder organization? Is it something like this?:

my_DICOMs \
          |- run_1 \
          |        |- run_1_volume_1.dcm
          |        |- run_1_volume_2.dcm
          |        | ...
          |- run_2 \
          |        |- run_2_volume_1.dcm
          |        |- run_2_volume_2.dcm
          |        | ...
          | ...
          |- run_1_PhysioLog \
          |        |- run_1_PhysioLog.dcm
          |- run_2_PhysioLog \
          |        |- run_2_PhysioLog.dcm
          | ...

Also, can you post the contents of the .heudiconv/<subID>/info/dicominfo.tsv and .heudiconv/<subID>/info/<subID>.auto.txt files (at the root of the output folder)?

[tjhendrickson]

@pvelasco Thanks for getting back to me. Yes, the DICOMs are organized in the way you suspected.
Here is an example:
`20211001-ST001-SUB1000101ses1/MR-SE030-rest_ep2d_bold_ME_16MIN_PhysioLog:
MR-ST001-SE030-0001.dcm

20211001-ST001-SUB1000101ses1/MR-SE031-rest_ep2d_bold_ME_16MIN2_SBRef:
MR-ST001-SE031-0001.dcm MR-ST001-SE031-EC3-0001.dcm MR-ST001-SE031-EC5-0001.dcm
MR-ST001-SE031-EC2-0001.dcm MR-ST001-SE031-EC4-0001.dcm
`
Attached are the files you asked for.

1000101_ses-1.auto.txt
dicominfo_ses-1.txt

[pvelasco]

Actually, the physio data extraction has not been added yet to the master branch. PR #446 addresses it, but it is still under review. You can test that PR, to see if it works with your data (still, your heuristic.py file will need some changes, like adding an extra argument (outtype = ('physio',)) to all the create_key calls for physio files.

[pvelasco]

Looking at the dicominfo file, it didn't "see" series number 30 (the one with the PhysioLog DICOM). I'm not sure why.
Do you mind posting the full heudiconv call?
I assume these DICOMs with the physio in them come from the CMRR SMS sequence. Is that correct?

[sajjadtorabian]

I'm checking in to see if there has been any update on this. I guess since #446 is still open, the best way to convert physio is directly through bidsphysio, right? I'm thinking to run heudiconv on everything except physio, and then on top of it run bidsphysio