hires frames

doc/dev/hiresframes.rst

@@ -0,0 +1,128 @@
+.. include:: ../include/links.rst
+
+.. _hires_frames:
+
+Automated typing of HIRES frames
+================================
+
+Version History
+---------------
+
+
+=========   ================   =========== ===========
+*Version*   *Author*           *Date*      ``PypeIt``
+=========   ================   =========== ===========
+1.0         Debora Pelliccia   17 Jun 2024 1.15.1.dev
+=========   ================   =========== ===========
+
+----
+
+Basics
+------
+
+The general procedure used to assign frames a given type is described
+here: :ref:`frame_types`.
+
+HIRES frame typing
+-----------------
+
+The primary typing of HIRES frames is performed by
+:func:`pypeit.spectrographs.keck_hires.KECKHIRESSpectrograph.check_frame_type`.
+This function checks the values of various header keywords against a
+set of criteria used to classify the frame type.
+The header cards required for the frame-typing and their associated keyword in the
+:class:`~pypeit.metadata.PypeItMetaData` object are:
+
+===============     ======================================================
+``fitstbl`` key     Header Key
+===============     ======================================================
+``exptime``         ``ELAPTIME``
+``hatch``           ``HATOPEN``
+``lampstat01``      See below
+No key              ``XCOVOPEN``
+No key              ``AUTOSHUT``
+===============     ======================================================
+
+``lampstat01`` is defined using a combination of header keywords, which include
+``LAMPCAT1``, ``LAMPCAT2``, ``LAMPQTZ2``, ``LAMPNAME``. If ``LAMPCAT1 = True`` or
+``LAMPCAT2 = True``, ``lampstat01`` will be equal to ``'ThAr1'`` or ``'ThAr2'``, respectively.
+If ``LAMPQTZ2 = True`` or ``LAMPNAME = 'quartz1'``, ``lampstat01`` will be equal to ``'on'``.
+
+
+The criteria used to select each frame type are as follows:
+
+====================   ============   ============   ============   ======================================   ======================================================
+Frame                  ``hatch``      ``AUTOSHUT``   ``XCOVOPEN``   ``lampstat01``                           ``exptime``
+====================   ============   ============   ============   ======================================   ======================================================
+``science``            ``True``       ``True``       ``True``       ``'off'``                                ``>601s``
+``standard``           ``'open'``     ``True``       ``True``       ``'off'``                                ``>1s`` & ``<600s``
+``bias``               ``False``      ``False``      ``True``       ``'off'``                                ``<0.001s``
+``dark``               ``False``      ``True``       ``True``       ``'off'``                                Not used
+``slitless_pixflat``   ``False``      ``True``       ``False``      ``'off'``                                ``<60s``
+``pixelflat``          ``False``      ``True``       ``True``       ``'on'``                                 ``<60s``
+``trace``              ``False``      ``True``       ``True``       ``'on'``                                 ``<60s``
+``illumflat``          ``False``      ``True``       ``True``       ``'on'``                                 ``<60s``
+``arc``                ``False``      ``True``       ``True``       ``'ThAr1'`` or ``'ThAr2'``               Not used
+``tilt``               ``False``      ``True``       ``True``       ``'ThAr1'`` or ``'ThAr2'``               Not used
+====================   ============   ============   ============   ======================================   ======================================================
+
+Note that PypeIt employs commonly used value of ``exptime`` to distinguish frame type;
+however, if needed, the user can specify a different value by
+using the ``exprng`` parameter in the :ref:`pypeit_file`; see also :ref:`frame_types`.
+
+The ``science`` and ``standard`` frames have identical selection criteria, except for the
+``exptime`` value. In order to better distinguish between the two types, the ``RA`` and ``DEC`` header
+keywords are also used to assign the ``standard`` type to frames with ``RA`` and ``DEC`` values that are
+within 10 arcmin of one of the standard stars available in PypeIt (see :ref:`standards`).
+
+The criteria used to select ``arc`` and ``tilt`` frames are identical; the same is true for
+``pixelflat``, ``trace``, and ``illumflat`` frames. Note that if both ``pixelflat`` and
+``slitless_pixflat`` frames are identified, the ``pixelflat`` assignment will be removed
+so that the ``slitless_pixflat`` frames will be used for the flat fielding.
+
+Finally, note that a HIRES frame is never given a ``pinhole`` type.
+
+
+Testing
+-------
+
+To test that PypeIt can successfully identify HIRES framt types
+among a set of files, we have added the
+``test_hires()`` test to ``${PYPEIT_DEV}/unit_tests/test_frametype.py``.
+
+Here is an example of how to run the test:
+
+.. code-block:: bash
+
+    cd ${PYPEIT_DEV}/unit_tests
+    pytest test_frametype.py::test_hires -W ignore
+
+The tests requires that you have downloaded the PypeIt
+:ref:`dev-suite` and defined the ``PYPEIT_DEV`` environmental
+variable that points to the relevant directory. The algorithm for
+all these tests is the same and is as follows:
+
+    1. Find the directories in the :ref:`dev-suite` with Keck
+       HIRES data.
+
+    2. For each directory (i.e., instrument setup):
+
+        a. Make sure there is a "by-hand" version of the pypeit file
+           for this setup where a human (one of the pypeit
+           developers) has ensured the frame types are correct.
+
+        b. Effectively run :ref:`pypeit_setup` on each of the
+           instrument setups to construct a new pypeit file with the
+           automatically generated frame types.
+
+        c. Read both the by-hand and automatically generated frame
+           types from these two pypeit files and check that they are
+           identical. This check is *only* performed for the
+           calibration frames, not any ``science`` or ``standard``
+           frames.
+
+Because this test is now included in the ``PypeIt``
+:ref:`unit-tests`, this frame-typing check is performed by the
+developers for every new version of the code.
+
+

doc/dev/lrisframes.rst

@@ -63,7 +63,7 @@ Frame                  ``hatch``      ``lampstat01``                           `
 ``pixelflat``          ``'closed'``   ``'Halogen' or '2H'``                    ``!= 'GOH_LRIS'``   ``<60s`` (LRIS RED) or ``<300s`` (LRIS BLUE)
 ``pixelflat``          ``'open'``     ``'on'``                                 ``!= 'GOH_LRIS'``   ``<60s`` (LRIS RED) or ``<300s`` (LRIS BLUE)
 ``trace``              ``'closed'``   ``'Halogen'`` or ``'2H'``                ``!= 'GOH_LRIS'``   ``<60s`` (LRIS RED) or ``<300s`` (LRIS BLUE)
-``trace``              ``'closed'``   ``'Halogen'`` or ``'2H'``                ``!= 'GOH_LRIS'``   ``<60s`` (LRIS RED) or ``<300s`` (LRIS BLUE)
+``trace``              ``'open'``     ``'on'``                                 ``!= 'GOH_LRIS'``   ``<60s`` (LRIS RED) or ``<300s`` (LRIS BLUE)
 ``illumflat``          ``'closed'``   ``'Halogen'`` or ``'2H'``                ``!= 'GOH_LRIS'``   ``<60s`` (LRIS RED) or ``<300s`` (LRIS BLUE)
 ``illumflat``          ``'open'``     ``'on'``                                 ``!= 'GOH_LRIS'``   ``<60s`` (LRIS RED) or ``<300s`` (LRIS BLUE)
 ``arc``                ``'closed'``   ``!= 'Halogen', '2H', 'on', 'off'``      ``!= 'GOH_LRIS'``   Not used

doc/spectrographs/deimos.rst

@@ -15,9 +15,9 @@ settings that are related to the Keck/DEIMOS spectrograph.
 .. warning::
 
     PypeIt currently *cannot* reduce images produced by reading
-    the DEIMOS CCDs with the A amplifier or those taken in imaging
+    the DEIMOS CCDs with the A+B amplifier or those taken in imaging
     mode. All image-handling assumes DEIMOS images have been read
-    with the B amplifier in the "Spectral" observing mode. PypeIt
+    with the A or B amplifier in the "Spectral" observing mode. PypeIt
     handles files that do not meet these criteria in two ways:
 
         - When running :ref:`pypeit_setup`, any frames not in

doc/spectrographs/keck_hires.rst

@@ -6,13 +6,62 @@ Overview
 ========
 
 This file summarizes several instrument specific settings that are related to the Keck/HIRES spectrograph.
+Currently, PypeIt only supports the reduction of the post detector upgrade HIRES data (around August 2004).
+Information on the frame typing and instrument configuration can be found in ref:`hiresframes` and :ref:`hiresconfig`.
+
+Default Settings
+++++++++++++++++
+
+See :ref:`instr_par-keck_hires` for a list of modifications to the default settings.
+*You do not have to add these changes to your PypeIt reduction file!*  This is just a list of
+how the parameters used for HIRES differ from the defaults listed in the preceding tables on that page.
+Moreover, additional modifications may have been made for specific setups, e.g, for different binning,
+etc. You can see a list of all the used parameters in the ``keck_hires_XXX.par``
+file generated by PypeIt at the beginning of the reduction.
+
+MOSAIC
+======
+
+PypeIt, by default, uses a mosaic approach for the reduction. It basically constructs a mosaic
+of the blue, green, and red detector data and reduces it, instead of processing the detector data individually.
+The mosaic reduction is switched on by setting the parameter ``detnum`` in :ref:`reduxpar` to be a
+tuple of the detector indices that are mosaiced together. For HIRES, it looks like:
+
+.. code-block:: ini
+
+    [rdx]
+        spectrograph = keck_hires
+        detnum = (1,2,3)
+
+This is already the default for HIRES, but the user can modify it in the :ref:`pypeit_file` to
+turn off the mosaic reduction.
+
+
+Calibrations
+============
+
+Flat Fielding
+-------------
+
 
 
 Wavelengths
-===========
+-----------
 
 See :ref:`wvcalib-echelle` for details on the wavelength calibration.
 
-We also note that several Orders from 40-45 are 
-frequently flagged as bad in the wavelength solution.  
-This is due, in part, to very bright ThAr line contamination. 
+
+Additional Reading
+==================
+
+Here are additional docs related to Keck/HIRES.  Note all of them are related
+to the development of PypeIt for use with HIRES data:
+
+.. TODO: Generally useful information in these dev docs should be moved into
+.. user-level doc pages, even if that means repeating information.
+
+.. toctree::
+   :maxdepth: 1
+
+   ../dev/hiresframes
+   ../dev/hiresconfig

doc/spectrographs/lris.rst

@@ -227,7 +227,7 @@ And then set the following in your :ref:`pypeit_file`:
 If, instead, you have taken slitless pixelflat frames, PypeIt will be able to identify them
 (see :ref:`lris_frames_report`) and process them during the reduction. The slitless pixelflat
 is generated by first scaling each slitless flat frame to have the same mean counts, then
-combining them using the weighted mean. A slitless pixelflat file will be generated
+median combining them. A slitless pixelflat file will be generated
 and stored in the reduction folder. The file will also be saved to the PypeIt cache
 directory ``data/static_calibs/{spectrograph_name}`` and will be automatically used during
 the reduction process. To use this file in future reductions, the user can add the

pypeit/spectrographs/keck_hires.py

@@ -66,7 +66,7 @@ class KECKHIRESSpectrograph(spectrograph.Spectrograph):
     # 2. Test on several different setups - DONE
     # 3. Implement PCA extrapolation into the blue
 
-    comment = 'Post detector upgrade (~ August 2004)'
+    comment = 'Post detector upgrade (~ August 2004). See :doc:`keck_hires`'
 
 
     # TODO: Place holder parameter set taken from X-shooter VIS for now.