diff --git a/.github/workflows/run-tests-action.yaml b/.github/workflows/run-tests-action.yaml
index 29ddb6cdc..4c5bebb1f 100755
--- a/.github/workflows/run-tests-action.yaml
+++ b/.github/workflows/run-tests-action.yaml
@@ -10,7 +10,7 @@ jobs:
strategy:
fail-fast: false
matrix:
- python-version: ["3.7","3.8","3.9", "3.10", "3.11"]
+ python-version: ["3.9", "3.10", "3.11", "3.12"]
os: ["ubuntu-latest"]
steps:
- name: Check out repository code
diff --git a/README.rst b/README.rst
index 203432ab9..8f58e3b8f 100755
--- a/README.rst
+++ b/README.rst
@@ -5,7 +5,7 @@
:target: https://coveralls.io/github/stonerlab/Stoner-PythonCode?branch=master
.. image:: https://app.codacy.com/project/badge/Grade/a9069a1567114a22b25d63fd4c50b228
- :target: https://www.codacy.com/gh/stonerlab/Stoner-PythonCode/dashboard?utm_source=github.com&utm_medium=referral&utm_content=stonerlab/Stoner-PythonCode&utm_campaign=Badge_Grade
+ :target: https://app.codacy.com/gh/stonerlab/Stoner-PythonCode/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade
.. image:: https://badge.fury.io/py/Stoner.svg
:target: https://badge.fury.io/py/Stoner
@@ -63,14 +63,18 @@ After installing the current Anaconda version, open a terminal (Mac/Linux) or An
.. code-block:: sh
- conda install -c phygbu Stoner
+ conda install -c phygbu -c conda-forge Stoner
-If you are not using Anaconda python, then pip should also work:
+If (and only if) you are not using Anaconda python, then pip should also work:
.. code-block:: sh
pip install Stoner
+.. warning::
+ The conda packages are generally much better tested than the pip wheels, so we would recommend using
+ conda where possible.
+
This will install the Stoner package and any missing dependencies into your current Python environment. Since the package is under fairly
constant updates, you might want to follow the development with git. The source code, along with example scripts
and some sample data files can be obtained from the github repository: https://github.com/stonerlab/Stoner-PythonCode
@@ -78,11 +82,11 @@ and some sample data files can be obtained from the github repository: https://g
Overview
========
-The main part of the **Stoner** package provides four basic top-level classes that describe:
- - an individual file of experimental data (**Stoner.Data**),
+The main part of the **Stoner** package provides four top-level classes that describe:
+ - an individual file of experimental data (**Stoner.Data**) - somewhat similar to a DataFrame,
- an individual experimental image (**Stoner.ImageFile**),
- - a list (such as a directory tree on disc) of many experimental files (**Stoner.DataFolder**)
- - a list (such as a directory tree on disc) of many image files (**Stoner.ImageFolder**).
+ - a nested list (such as a directory tree on disc) of many experimental files (**Stoner.DataFolder**)
+ - a nested list (such as a directory tree on disc) of many image files (**Stoner.ImageFolder**).
For our research, a typical single experimental data file is essentially a single 2D table of floating point
numbers with associated metadata, usually saved in some ASCII text format. This seems to cover most experiments
@@ -99,6 +103,15 @@ operations to be chained together in one line.
This is a *data-centric* approach - we have some data and we do various operations on it to get to our result. In
contrasr, traditional functional programming thinks in terms of various functions into which you pass data.
+.. note::
+ This is rather similar to pandas DataFrames and the package provides methods to easily convert to and from
+ DataFrames. Unlike a DataFrame, a **Stoner.Data** object maintains a dictionary of additional metadata
+ attached to the dataset (e.g. of instrument settings, experimental ort environmental; conditions
+ when thedata was taken). To assist with exporting to pandas DataFrames, the package will add a custom
+ attrobute handler to pandas DataFrames **DataFrame.metadata** to hold this additional data.
+
+ Unlike Pandas, the **Stoner** package's default is to operate in-place and also to return the object
+ from method calls to facilitate "chaining" of data methods into short single line pipelines.
Data and Friends
----------------
@@ -188,11 +201,11 @@ At the moment the development version is maily broen....
Build Status
~~~~~~~~~~~~
-Version 0.7 onwards are tested using the Travis-CI services with unit test coverage assessed by Coveralls.
+Version 0.7-0.9 were tested using the Travis-CI services with unit test coverage assessed by Coveralls.
-Version 0.9 is tested with Python 2.7, 3.5, 3.6 using the standard unittest module.
+Version 0.9 was tested with Python 2.7, 3.5, 3.6 using the standard unittest module.
-Version 0.10 is tested using **pytest** with Python 3.6-3.9 using a github action.
+Version 0.10 is tested using **pytest** with Python 3.7-3.11 using a github action.
Citing the Stoner Package
@@ -207,6 +220,7 @@ Stable Versions
New Features in 0.10 include:
+ * Support for Python 3.10 and 3.11
* Refactor Stoner.Core.DataFile to move functionality to mixin classes
* Start implementing PEP484 Type hinting
* Support pathlib for paths
@@ -252,12 +266,11 @@ The ancient stable version is 0.7.2. Features of 0.7.2 include
* DataFolder has an options to skip iterating over empty Data files
* Further improvements to :py:attr:`Stoner.Core.DataFile.setas` handline.
-No further relases will be made to 0.7.x.
+No further relases will be made to 0.7.x - 0.9.x
-0.6, 0.7 should work on Python 2.7 and 3.5
-0.8 is also tested on Python 3.6
+Versions 0.6.x and earlier are now pre-historic!
-.. _online documentation: http://stoner-pythoncode.readthedocs.io/en/latest/
+.. _online documentation: http://stoner-pythoncode.readthedocs.io/en/stable/
.. _github repository: http://www.github.com/stonerlab/Stoner-PythonCode/
.. _Dr Gavin Burnell: http://www.stoner.leeds.ac.uk/people/gb
.. _User_Guide: http://stoner-pythoncode.readthedocs.io/en/latest/UserGuide/ugindex.html
diff --git a/Stoner/Analysis.py b/Stoner/Analysis.py
index 512a04406..c22ea08de 100755
--- a/Stoner/Analysis.py
+++ b/Stoner/Analysis.py
@@ -17,7 +17,6 @@
class AnalysisMixin:
-
"""A mixin calss designed to work with :py:class:`Stoner.Core.DataFile` to provide additional analysis methods."""
def __dir__(self):
diff --git a/Stoner/Core.py b/Stoner/Core.py
index 99f54fc6c..a75ba7392 100755
--- a/Stoner/Core.py
+++ b/Stoner/Core.py
@@ -1,4 +1,5 @@
"""Stoner.Core provides the core classes for the Stoner package."""
+
__all__ = [
"StonerLoadError",
"StonerSetasError",
@@ -64,7 +65,6 @@ class DataFile(
metadataObject,
MutableSequence,
):
-
"""Base class object that represents a matrix of data, associated metadata and column headers.
Attributes:
diff --git a/Stoner/Folders.py b/Stoner/Folders.py
index 8d3773b11..055499b25 100755
--- a/Stoner/Folders.py
+++ b/Stoner/Folders.py
@@ -2,6 +2,7 @@
The core classes provides a means to access them as an ordered collection or as a mapping.
"""
+
__all__ = ["DataFolder", "PlotFolder"]
from Stoner.tools import make_Data
@@ -11,7 +12,6 @@
class DataFolder(DataMethodsMixin, DiskBasedFolderMixin, baseFolder):
-
"""Provide an interface to manipulating lots of data files stored within a directory structure on disc.
By default, the members of the DataFolder are instances of :class:`Stoner.Data`. The DataFolder emplys a lazy
@@ -28,5 +28,4 @@ def __init__(self, *args, **kargs):
class PlotFolder(PlotMethodsMixin, DataFolder):
-
"""A :py:class:`Stoner.folders.baseFolder` that knows how to ploth its underlying data files."""
diff --git a/Stoner/HDF5.py b/Stoner/HDF5.py
index d7fa25628..196c91ef2 100755
--- a/Stoner/HDF5.py
+++ b/Stoner/HDF5.py
@@ -9,6 +9,7 @@
to :py:class:`Stoner.Core.Data`.
"""
+
__all__ = ["HDF5File", "HDF5Folder", "HGXFile", "SLS_STXMFile", "STXMImage", "HDFFileManager"]
import importlib
import os.path as path
@@ -37,7 +38,7 @@ def get_hdf_loader(f, default_loader=lambda *args, **kargs: None):
Callable function that can produce an object of an appropriate class.
"""
if "type" not in f.attrs:
- StonerLoadError("HDF5 Group does not specify the type attribute used to check we can load it.")
+ raise StonerLoadError("HDF5 Group does not specify the type attribute used to check we can load it.")
typ = bytes2str(f.attrs.get("type", ""))
if (typ not in globals() or not isinstance(globals()[typ], type)) and "module" not in f.attrs:
raise StonerLoadError(
@@ -52,7 +53,6 @@ def get_hdf_loader(f, default_loader=lambda *args, **kargs: None):
class HDFFileManager:
-
"""Context manager for HDF5 files."""
def __init__(self, filename, mode="r"):
@@ -84,8 +84,8 @@ def __init__(self, filename, mode="r"):
if not mode.startswith("w"):
with h5py.File(filename, "r"):
pass
- except (IOError, OSError):
- raise StonerLoadError(f"{filename} not at HDF5 File")
+ except (IOError, OSError) as err:
+ raise StonerLoadError(f"{filename} not at HDF5 File") from err
self.filename = filename
def __enter__(self):
@@ -108,14 +108,13 @@ def __enter__(self):
raise StonerLoadError("Note a resource that can be handled with HDF")
return self.handle
- def __exit__(self, type, value, traceback):
+ def __exit__(self, _type, _value, _traceback):
"""Ensure we close the hdf file no matter what."""
if self.file is not None and self.close:
self.file.close()
class HDF5File(DataFile):
-
"""A sub class of DataFile that sores itself in a HDF5File or group.
Args:
@@ -287,7 +286,6 @@ def to_hdf(self, filename=None, **kargs): # pylint: disable=unused-argument
class HGXFile(DataFile):
-
"""A subclass of DataFile for reading GenX HDF Files.
These files typically have an extension .hgx. This class has been based on a limited sample
@@ -372,7 +370,6 @@ def main_data(self, data_grp):
class HDF5FolderMixin:
-
"""Provides a method to load and save data from a single HDF5 file with groups.
See :py:class:`Stoner.Folders.DataFolder` for documentation on constructor.
@@ -568,7 +565,6 @@ def save(self, root=None):
class HDF5Folder(HDF5FolderMixin, DataFolder):
-
"""Just enforces the loader attriobute to be an HDF5File."""
def __init__(self, *args, **kargs):
@@ -578,7 +574,6 @@ def __init__(self, *args, **kargs):
class SLS_STXMFile(DataFile):
-
"""Load images from the Swiss Light Source Pollux beamline."""
priority = 16
@@ -671,7 +666,6 @@ def scan_meta(self, group):
class STXMImage(ImageFile):
-
"""An instance of KerrArray that will load itself from a Swiss Light Source STXM image."""
# pylint: disable=no-member
diff --git a/Stoner/Image/attrs.py b/Stoner/Image/attrs.py
index d6d4e864b..7c5b0b086 100755
--- a/Stoner/Image/attrs.py
+++ b/Stoner/Image/attrs.py
@@ -47,7 +47,6 @@ def _proxy(self, *args, **kargs):
@class_modifier(draw, adaptor=_draw_apaptor, RTD_restrictions=False, no_long_names=True)
class DrawProxy:
-
"""Provides a wrapper around :py:mod:`skimage.draw` to allow easy drawing of objects onto images.
This class allows access the user to draw simply shapes on an image (or its mask) by specifying the desired shape
@@ -205,7 +204,6 @@ def square(self, r, c, w, angle=0.0, shape=None, value=1.0):
class MaskProxy:
-
"""Provides a wrapper to support manipulating the image mask easily.
The actual mask of a :py:class:`Stonmer.ImageFile` is held by the mask attribute of the underlying
diff --git a/Stoner/Image/core.py b/Stoner/Image/core.py
index 87c90209c..7ea9267fa 100755
--- a/Stoner/Image/core.py
+++ b/Stoner/Image/core.py
@@ -151,7 +151,6 @@ def copy_into(source: "ImageFile", dest: "ImageFile") -> "ImageFile":
@class_modifier([ndi], transpose=True)
@class_modifier(imagefuncs, overload=True)
class ImageArray(np.ma.MaskedArray, metadataObject):
-
"""A numpy array like class with a metadata parameter and pass through to skimage methods.
ImageArray is for manipulating images stored as a 2d numpy array.
@@ -251,7 +250,6 @@ def __new__(cls, *args, **kargs):
We're using __new__ rather than __init__ to imitate a numpy array as
close as possible.
"""
-
array_arg_keys = ["dtype", "copy", "order", "subok", "ndmin", "mask"] # kwargs for array setup
array_args = {k: kargs.pop(k) for k in array_arg_keys if k in kargs.keys()}
user_metadata = kargs.get("metadata", {})
@@ -671,6 +669,10 @@ def __delitem__(self, index):
else:
super().__delitem__(index)
+ def save(self, filename=None, **kargs):
+ """Stub method for a save function."""
+ raise NotImplementedError(f"Save is not implemented in {self.__class__}")
+
@class_modifier(
[
@@ -698,7 +700,6 @@ def __delitem__(self, index):
@class_modifier(imagefuncs, overload=True, adaptor=image_file_adaptor)
@class_wrapper(target=ImageArray, exclude_below=metadataObject)
class ImageFile(metadataObject):
-
"""An Image file type that is analogous to :py:class:`Stoner.Data`.
This contains metadata and an image attribute which
diff --git a/Stoner/Image/folders.py b/Stoner/Image/folders.py
index 32ba64acc..c25524d43 100755
--- a/Stoner/Image/folders.py
+++ b/Stoner/Image/folders.py
@@ -8,7 +8,7 @@
from copy import deepcopy, copy
import numpy as np
-from matplotlib.pyplot import figure, Figure, subplot, tight_layout
+from matplotlib.pyplot import figure, Figure, subplot
from PIL.TiffImagePlugin import ImageFileDirectory_v2
from PIL import Image
@@ -19,7 +19,6 @@
class ImageFolderMixin:
-
"""Mixin to provide a folder object for images.
ImageFolderMixin is designed to behave pretty much like DataFolder but with
@@ -249,7 +248,7 @@ def from_tiff(cls, filename, **kargs):
except (TypeError, ValueError, IOError):
metadata = []
else:
- raise TypeError(f"Cannot load as an ImageFolder due to lack of description tag")
+ raise TypeError("Cannot load as an ImageFolder due to lack of description tag")
imglist = []
for ix, md in enumerate(metadata):
img.seek(ix)
@@ -314,8 +313,6 @@ def montage(self, *args, **kargs):
Passed to matplotlib figure call.
plots_per_page(int):
maximum number of plots per figure.
- tight_layout(dict or False):
- If not False, arguments to pass to a call of :py:func:`matplotlib.pyplot.tight_layout`. Defaults to {}
Returns:
A list of :py:class:`matplotlib.pyplot.Axes` instances.
@@ -332,7 +329,6 @@ def montage(self, *args, **kargs):
plts = min(plts, len(self))
extra = kargs.pop("extra", lambda i, j, d: None)
- tight = kargs.pop("tight_layout", {})
fig_num = kargs.pop("figure", getattr(self, "_figure", None))
if isinstance(fig_num, Figure):
@@ -344,7 +340,7 @@ def montage(self, *args, **kargs):
fig_num = fig_num.number
fig_args = getattr(self, "_fig_args", [])
- fig_kargs = getattr(self, "_fig_kargs", {})
+ fig_kargs = getattr(self, "_fig_kargs", {"layout": "constrained"})
for arg in ("figsize", "dpi", "facecolor", "edgecolor", "frameon", "FigureClass"):
if arg in kargs:
fig_kargs[arg] = kargs.pop(arg)
@@ -363,8 +359,6 @@ def montage(self, *args, **kargs):
for i, d in enumerate(self):
plt_kargs = copy(kargs)
if i % plts == 0 and i != 0:
- if isinstance(tight, dict):
- tight_layout(**tight)
fig = figure(*fig_args, **fig_kargs)
fignum = fig.number
j = 1
@@ -381,8 +375,6 @@ def montage(self, *args, **kargs):
plt_kargs["title"] = kargs["title"](d)
ret.append(d.imshow(*args, **plt_kargs))
extra(i, j, d)
- if isinstance(tight, dict):
- tight_layout(**tight)
return ret
def stddev(self, weights=None, _box=False, _metadata="first"):
@@ -481,7 +473,6 @@ def to_tiff(self, filename):
class ImageFolder(ImageFolderMixin, DiskBasedFolderMixin, baseFolder):
-
"""Folder object for images.
ImageFolder is designed to behave pretty much like DataFolder but with
diff --git a/Stoner/Image/kerr.py b/Stoner/Image/kerr.py
index 2df2aacc8..833686ccc 100755
--- a/Stoner/Image/kerr.py
+++ b/Stoner/Image/kerr.py
@@ -33,7 +33,6 @@
@class_modifier(kerrfuncs)
class KerrArray(ImageArray):
-
"""A subclass for Kerr microscopy specific image functions."""
# useful_keys are metadata keys that we'd usually like to keep from a
@@ -83,10 +82,13 @@ def tesseractable(self):
"""Do a test call to tesseract to see if it is there and cache the result."""
return _tesseractable
+ def save(self, filename=None, **kargs):
+ """Stub method for a save function."""
+ raise NotImplementedError(f"Save is not implemented in {self.__class__}")
+
@class_modifier(kerrfuncs, adaptor=image_file_adaptor)
class KerrImageFile(ImageFile):
-
"""Subclass of ImageFile that keeps the data as a KerrArray so that extra functions are available."""
priority = 16
@@ -99,7 +101,7 @@ def __init__(self, *args, **kargs):
self._image = self.image.view(KerrArray)
@ImageFile.image.getter
- def image(self):
+ def image(self): # pylint disable=invalid-overridden-method
"""Access the image data."""
return self._image.view(KerrArray)
@@ -123,7 +125,6 @@ def image(self, v): # pylint: disable=function-redefined
class KerrStackMixin:
-
"""A mixin for :py:class:`ImageStack` that adds some functionality particular to Kerr images.
Attributes:
@@ -327,7 +328,6 @@ def average_Hcmap(self, weights=None, ignore_zeros=False):
class MaskStackMixin:
-
"""A Mixin for :py:class:`Stoner.Image.ImageStack` but made for stacks of boolean or binary images."""
def __init__(self, *args, **kargs):
@@ -394,10 +394,8 @@ def switch_index(self, saturation_end=True, saturation_value=True):
class KerrStack(KerrStackMixin, ImageStack):
-
"""Represent a stack of Kerr images."""
class MaskStack(MaskStackMixin, KerrStackMixin, ImageStack):
-
"""Represent a set of masks for Kerr images."""
diff --git a/Stoner/Image/stack.py b/Stoner/Image/stack.py
index 8a54381e8..a9236d6a6 100755
--- a/Stoner/Image/stack.py
+++ b/Stoner/Image/stack.py
@@ -25,7 +25,6 @@ def _load_ImageArray(f, **kargs):
class ImageStackMixin:
-
"""Implement an interface for a baseFolder to store images in a 3D numpy array for faster access."""
_defaults = {"type": ImageFile}
@@ -481,7 +480,6 @@ def show(self):
class StackAnalysisMixin:
-
"""Add some analysis capability to ImageStack.
These functions may override :py:class:`Stoner,Image.ImageFile` functions but do them efficiently for a numpy
@@ -520,5 +518,4 @@ def subtract(self, background):
class ImageStack(StackAnalysisMixin, ImageStackMixin, ImageFolderMixin, DiskBasedFolderMixin, baseFolder):
-
"""An alternative implementation of an image stack based on baseFolder."""
diff --git a/Stoner/Image/widgets.py b/Stoner/Image/widgets.py
index 53c150d40..ab625e7cb 100755
--- a/Stoner/Image/widgets.py
+++ b/Stoner/Image/widgets.py
@@ -45,7 +45,6 @@ def _rotated_ellipse(p, data):
class LineSelect:
-
"""Show an Image and slow the user to draw a line on it using cursors."""
def __init__(self):
@@ -171,7 +170,6 @@ def draw_line(self, event):
class RegionSelect:
-
"""Show an Image and slow the user to select a rectangular section."""
def __init__(self):
@@ -257,7 +255,6 @@ def finish(self, key_event):
class ShapeSelect:
-
"""Show an Image and slow the user to draw a line on it using cursors."""
def __init__(self):
diff --git a/Stoner/Zip.py b/Stoner/Zip.py
index 82181e025..96e7f63c6 100755
--- a/Stoner/Zip.py
+++ b/Stoner/Zip.py
@@ -54,7 +54,6 @@ def test_is_zip(filename, member=""):
class ZippedFile(DataFile):
-
"""A sub class of DataFile that sores itself in a zip file.
If the first non-keyword argument is not an :py:class:`zipfile:ZipFile` then
@@ -228,7 +227,6 @@ def save(self, filename=None, **kargs):
class ZipFolderMixin:
-
"""Provides methods to load and save data from a single Zip file.
See :py:class:`Stoner.Folders.DataFolder` for documentation on constructor.
@@ -511,7 +509,6 @@ def _save(self, f, trail):
class ZipFolder(ZipFolderMixin, DiskBasedFolderMixin, baseFolder):
-
"""A sub class of DataFile that sores itself in a zip file.
If the first non-keyword argument is not an :py:class:`zipfile:ZipFile` then
diff --git a/Stoner/__init__.py b/Stoner/__init__.py
index 6719317cb..6f9d75fa7 100755
--- a/Stoner/__init__.py
+++ b/Stoner/__init__.py
@@ -3,6 +3,7 @@
It has been developed by members of the `Condensed Matter Group` at the
`University of Leeds`.
"""
+
# pylint: disable=import-error
__all__ = [
"core",
@@ -35,7 +36,7 @@
Options = _Options()
-__version_info__ = ("0", "10", "9")
+__version_info__ = ("0", "10", "10")
__version__ = ".".join(__version_info__)
__homepath__ = pathlib.Path(__file__).parent.resolve()
diff --git a/Stoner/analysis/__init__.py b/Stoner/analysis/__init__.py
index 5f12a9780..2e1a8f639 100755
--- a/Stoner/analysis/__init__.py
+++ b/Stoner/analysis/__init__.py
@@ -1,4 +1,5 @@
"""Subpaclage to support the data analysis functions."""
+
from . import fitting, utils, columns
__all__ = ["fitting", "utils", "columns", "filtering", "features"]
diff --git a/Stoner/analysis/columns.py b/Stoner/analysis/columns.py
index 2204e998c..dcd53e116 100755
--- a/Stoner/analysis/columns.py
+++ b/Stoner/analysis/columns.py
@@ -11,7 +11,6 @@
class ColumnOpsMixin:
-
"""A mixin calss designed to work with :py:class:`Stoner.Core.DataFile` to provide additional stats methods."""
def _do_error_calc(self, col_a, col_b, error_type="relative"):
diff --git a/Stoner/analysis/features.py b/Stoner/analysis/features.py
index 05bef6437..92e1c0ffd 100755
--- a/Stoner/analysis/features.py
+++ b/Stoner/analysis/features.py
@@ -14,7 +14,6 @@
class FeatureOpsMixin:
-
"""Mixin to provide additional functions to support finding features in a dataset."""
def peaks(self, **kargs):
diff --git a/Stoner/analysis/filtering.py b/Stoner/analysis/filtering.py
index bcee26010..2f114165d 100755
--- a/Stoner/analysis/filtering.py
+++ b/Stoner/analysis/filtering.py
@@ -19,7 +19,6 @@
class FilteringOpsMixin:
-
"""Provide additional filtering sndsmoothing methods to :py:class:`Stoner.Data`."""
def SG_Filter(
@@ -85,7 +84,7 @@ def SG_Filter(
ddata = savgol_filter(data, window_length=points, polyorder=poly, deriv=order, mode="interp")
if isinstance(pad, bool) and pad:
- offset = int(np.ceil(points * order**2 / 8))
+ offset = int(np.ceil(points * (order + 1) ** 2 / 8))
padv = np.mean(ddata[:, offset:-offset], axis=1)
pad = np.ones((ddata.shape[0], offset))
for ix, v in enumerate(padv):
@@ -478,6 +477,7 @@ def make_bins(self, xcol, bins, mode="lin", **kargs):
if mode.lower().startswith("lin"):
bin_centres = (bin_start + bin_stop) / 2.0
elif mode.lower().startswith("log"):
+ bin_start = np.where(bin_start <= 0, 1e-9, bin_start)
bin_centres = np.exp(np.log(bin_start) + np.log(bin_stop) / 2.0)
else:
raise ValueError(f"mode should be either lin(ear) or log(arthimitc) not {mode}")
diff --git a/Stoner/analysis/fitting/__init__.py b/Stoner/analysis/fitting/__init__.py
index 1ad0dc5ad..58a1bbce3 100755
--- a/Stoner/analysis/fitting/__init__.py
+++ b/Stoner/analysis/fitting/__init__.py
@@ -1,4 +1,5 @@
"""Provides additional functionality for doing curve fitting to data."""
+
__all__ = ["odr_Model", "FittingMixin", "models"]
from .mixins import odr_Model, FittingMixin
from . import models
diff --git a/Stoner/analysis/fitting/mixins.py b/Stoner/analysis/fitting/mixins.py
index bf445fbd3..8ab16f6a4 100755
--- a/Stoner/analysis/fitting/mixins.py
+++ b/Stoner/analysis/fitting/mixins.py
@@ -24,7 +24,6 @@
class odr_Model(odrModel):
-
"""A wrapper for converting lmfit models to odr models."""
def __init__(self, *args, **kargs):
@@ -109,7 +108,6 @@ def param_names(self):
class MimizerAdaptor:
-
"""Work with an lmfit.Model or generic callable to use with scipy.optimize global minimization functions.
The :pymod:`scipy.optimize` module's minimizers generally expect functions which take an array like parameter
@@ -171,8 +169,9 @@ def wrapper(beta, x, y, sigma, *args):
class _curve_fit_result:
-
- """Represent a result from fitting using :py:func:`scipy.optimize.curve_fit` as a class to make handling easier."""
+ """Represent a result from fitting using :py:func:`scipy.optimize.curve_fit`
+ as a class to make handling easier.
+ """
def __init__(self, popt, pcov, infodict, mesg, ier):
"""Store the results of the curve fit full_output fit.
@@ -469,7 +468,6 @@ def _prep_lmfit_p0(model, ydata, xdata, p0, kargs):
class FittingMixin:
-
"""A mixin calss for :py:class:`Stoner.Core.DataFile` to provide additional curve_fiotting methods."""
def annotate_fit(self, model, x=None, y=None, z=None, text_only=False, **kargs):
@@ -1140,7 +1138,7 @@ def _func(x, *beta):
raise TypeError(
"".join(
[
- f"curve_fit parameter 1 must be either a Model class from",
+ "curve_fit parameter 1 must be either a Model class from",
f" lmfit or scipy.odr, or a callable, not a {type(func)}",
]
)
diff --git a/Stoner/analysis/fitting/models/__init__.py b/Stoner/analysis/fitting/models/__init__.py
index 137303171..d4561851b 100755
--- a/Stoner/analysis/fitting/models/__init__.py
+++ b/Stoner/analysis/fitting/models/__init__.py
@@ -1,4 +1,5 @@
"""Sub package of various built-in models for the Stoner package."""
+
__all__ = [
"generic",
"thermal",
diff --git a/Stoner/analysis/fitting/models/e_transport.py b/Stoner/analysis/fitting/models/e_transport.py
index 8493ce923..d44d9e1e9 100755
--- a/Stoner/analysis/fitting/models/e_transport.py
+++ b/Stoner/analysis/fitting/models/e_transport.py
@@ -20,7 +20,7 @@
int64 = _dummy()
-@jit(float64(float64, int64))
+@jit(float64(float64, int64), nopython=True)
def _bgintegrand(x, n):
"""Calculate the integrand for the Bloch Grueneisen model."""
return x**n / ((np.exp(x) - 1) * (1 - np.exp(-x)))
@@ -94,7 +94,8 @@ def fluchsSondheimer(t, l, p, sigma_0):
"""
k = t / l
- kernel = lambda x, k: (x - x**3) * np.exp(-k * x) / (1 - np.exp(-k * x))
+ def kernel(x, k):
+ return (x - x**3) * np.exp(-k * x) / (1 - np.exp(-k * x))
result = np.zeros(k.shape)
for i, v in enumerate(k):
@@ -130,7 +131,6 @@ def blochGrueneisen(T, thetaD, rho0, A, n):
class WLfit(Model):
-
"""Weak localisation model class.
Args:
@@ -174,7 +174,6 @@ def guess(self, data, x=None, **kwargs):
class FluchsSondheimer(Model):
-
"""Evaluate a Fluchs-Sondheumer model function for conductivity.
Args:
@@ -209,7 +208,6 @@ def guess(self, data, t=None, **kwargs): # pylint: disable=unused-argument
class BlochGrueneisen(Model):
-
"""BlochGrueneiseen Function for fitting R(T).
Args:
diff --git a/Stoner/analysis/fitting/models/generic.py b/Stoner/analysis/fitting/models/generic.py
index 45528d6bd..16e561b06 100755
--- a/Stoner/analysis/fitting/models/generic.py
+++ b/Stoner/analysis/fitting/models/generic.py
@@ -112,7 +112,6 @@ def lorentzian_diff(x, A, sigma, mu):
class Linear(_Linear):
-
"""Simple linear fit class."""
diff --git a/Stoner/analysis/fitting/models/magnetism.py b/Stoner/analysis/fitting/models/magnetism.py
index d4bf183f5..cc497ab4a 100755
--- a/Stoner/analysis/fitting/models/magnetism.py
+++ b/Stoner/analysis/fitting/models/magnetism.py
@@ -125,11 +125,7 @@ def inverse_kittel(f, g, M_s, H_k):
\gamma^{2} \mu_{0}^{2} + 16 \pi^{2} f^{2}}`
"""
gamma = g * cnst.e / (2 * cnst.m_e)
- return (
- -H_k
- - M_s / 2
- + np.sqrt(M_s**2 * gamma**2 * cnst.mu_0**2 + 16 * np.pi**2 * f**2) / (2 * gamma * cnst.mu_0)
- )
+ return -H_k - M_s / 2 + np.sqrt(M_s**2 * gamma**2 * cnst.mu_0**2 + 16 * np.pi**2 * f**2) / (2 * gamma * cnst.mu_0)
def fmr_power(H, H_res, Delta_H, K_1, K_2):
@@ -267,7 +263,7 @@ class BlochLawThin(Model):
"""
def __init__(self, *args, **kwargs):
- """Setup the model."""
+ """Initialise the model."""
super().__init__(self.blochs_law_thinfilm, *args, **kwargs)
self.set_param_hint("g", vary=False, value=2.0)
self.set_param_hint("A", vary=True, min=0)
@@ -275,7 +271,7 @@ def __init__(self, *args, **kwargs):
self.prefactor = gamma(1.5) * zeta(1.5) / (4 * np.pi**2)
def blochs_law_thinfilm(self, T, D, Bz, S, v_ws, a, nz):
- """Thin film version of Blopch's Law.
+ r"""Thin film version of Blopch's Law.
Parameters:
T (array):
@@ -310,7 +306,6 @@ def blochs_law_thinfilm(self, T, D, Bz, S, v_ws, a, nz):
ln\left[1-\exp\left( -\frac{1}{k_B T}\left(g\mu_B B_z+D\left(\frac{m \pi}{a(n_z-1)}\right)^2
\right)\right) \right]`
"""
-
kz_sum = sum(
np.log(1.0 - np.exp(-(D * (m * np.pi / ((nz - 1) * a)) ** 2 + Bz) / (k * T))) for m in range(0, nz - 1)
)
@@ -346,7 +341,8 @@ def guess(self, data, x=None, **kwargs):
r"""Guess some starting values.
M_s is taken as half the difference of the range of thew M data,
- we can find m/T from the susceptibility :math:`chi= M_s \mu_o m / kT`,"""
+ we can find m/T from the susceptibility :math:`chi= M_s \mu_o m / kT`
+ """
M_s = (np.max(data) - np.min(data)) / 2.0
if x is not None:
d = np.sort(np.row_stack((x, data)))
diff --git a/Stoner/analysis/fitting/models/superconductivity.py b/Stoner/analysis/fitting/models/superconductivity.py
index 28d96f355..e56921416 100755
--- a/Stoner/analysis/fitting/models/superconductivity.py
+++ b/Stoner/analysis/fitting/models/superconductivity.py
@@ -78,9 +78,7 @@ def _strijkers_core(V, omega, delta, P, Z):
Au2 = (((np.abs(E) / (np.sqrt((E**2) - (delta**2)))) ** 2) - 1) / (
((np.abs(E) / (np.sqrt((E**2) - (delta**2)))) + (1 + 2 * (Z**2))) ** 2
)
- Bu2 = (4 * (Z**2) * (1 + (Z**2))) / (
- ((np.abs(E) / (np.sqrt((E**2) - (delta**2)))) + (1 + 2 * (Z**2))) ** 2
- )
+ Bu2 = (4 * (Z**2) * (1 + (Z**2))) / (((np.abs(E) / (np.sqrt((E**2) - (delta**2)))) + (1 + 2 * (Z**2))) ** 2)
Bp2 = Bu2 / (1 - Au2)
unpolarised_prefactor = (1 - P) * (1 + (Z**2))
@@ -312,7 +310,6 @@ def ic_RN_Dirty(d_f, IcRn0, E_x, v_f, d_0, tau, delta, T):
class Strijkers(Model):
-
"""strijkers Model for point-contact Andreev Reflection Spectroscopy.
Args:
diff --git a/Stoner/analysis/fitting/models/tunnelling.py b/Stoner/analysis/fitting/models/tunnelling.py
index 346925e33..a882cb9da 100755
--- a/Stoner/analysis/fitting/models/tunnelling.py
+++ b/Stoner/analysis/fitting/models/tunnelling.py
@@ -125,7 +125,6 @@ def tersoffHammann(V, A):
class Simmons(Model):
-
"""Simmons model of electron tunnelling.
Args:
@@ -160,7 +159,6 @@ def guess(self, data, V=None, **kwargs): # pylint: disable=unused-argument
class BDR(Model):
-
"""BDR model tunnelling.
Args:
@@ -195,7 +193,6 @@ def guess(self, data, V=None, **kwargs): # pylint: disable=unused-argument
class FowlerNordheim(Model):
-
"""Fowler Nordhiem Model of electron tunnelling.
Args:
@@ -224,7 +221,6 @@ def guess(self, data, V=None, **kwargs): # pylint: disable=unused-argument
class TersoffHammann(Model):
-
"""Tersoff-Hamman model for tunnelling through STM tip.
Args:
diff --git a/Stoner/compat.py b/Stoner/compat.py
index ddeb1b526..c0c8dacc0 100755
--- a/Stoner/compat.py
+++ b/Stoner/compat.py
@@ -4,7 +4,6 @@
"str2bytes",
"bytes2str",
"get_filedialog",
- "getargspec",
"string_types",
"path_types",
"int_types",
@@ -22,14 +21,15 @@
"_dummy",
]
-from sys import version_info as __vi__
+from sys import version_info as __vi__, modules
from os import walk, makedirs
from os.path import join, commonpath
import fnmatch
-from inspect import signature, getfullargspec
from shutil import which
from pathlib import PurePath
from packaging.version import parse as version_parse
+from inspect import signature
+import re
import numpy as np
import scipy as sp
@@ -40,6 +40,13 @@
sp_version = version_parse(sp.__version__)
mpl_version = version_parse(matplotlib.__version__)
+try: # This only works in PY 3.11 onwards
+ modules["sre_parse"] = re._parser
+ modules["sre_constants"] = re._constants
+ modules["sre_compile"] = re._compiler
+except AttributeError:
+ pass
+
try:
import hyperspy as hs # Workaround an issue in hs 1.5.2 conda packages
@@ -68,11 +75,6 @@
from re import Pattern as _pattern_type # pylint: disable = E0611
-def getargspec(*args, **kargs):
- """Wrap for getargspec for Python V3."""
- return getfullargspec(*args, **kargs)[:4]
-
-
def get_func_params(func):
"""Get the parameters for a function."""
sig = signature(func)
@@ -149,7 +151,6 @@ def listdir_recursive(dirname, glob=None):
class ClassPropertyDescriptor:
-
"""Supports adding class properties."""
def __init__(self, fget, fset=None):
@@ -178,7 +179,6 @@ def _jit(func, *_, **__):
class _dummy:
-
"""A class that does nothing so that float64 can be an instance of it safely."""
def jit(self, func, *_, **__): # pylint: disable=no-self-use
diff --git a/Stoner/core/base.py b/Stoner/core/base.py
index 17a212081..dee453246 100755
--- a/Stoner/core/base.py
+++ b/Stoner/core/base.py
@@ -119,12 +119,10 @@ def string_to_type(value: String_Types) -> Any:
class _evaluatable:
-
"""Placeholder to indicate that special action needed to convert a string representation to valid Python type."""
class regexpDict(sorteddict):
-
"""An ordered dictionary that permits looks up by regular expression."""
allowed_keys: Tuple = (object,)
@@ -260,7 +258,6 @@ def has_key(self, name: Any) -> bool:
class typeHintedDict(regexpDict):
-
"""Extends a :py:class:`blist.sorteddict` to include type hints of what each key contains.
The CM Physics Group at Leeds makes use of a standard file format that closely matches
@@ -670,7 +667,6 @@ def import_key(self, line: str) -> None:
class metadataObject(MutableMapping):
-
"""Represent some sort of object that has metadata stored in a :py:class:`Stoner.Core.typeHintedDict` object.
Attributes:
@@ -779,11 +775,10 @@ def _load(self, filename: Filename, *args: Any, **kargs: Any) -> "metadataObject
raise NotImplementedError("Save is not implemented in the base class.")
-if pd is not None:
+if pd is not None and not hasattr(pd.DataFrame, "metadata"): # Don;t double add metadata
@pd.api.extensions.register_dataframe_accessor("metadata")
class PandasMetadata(typeHintedDict):
-
"""Add a typehintedDict to PandasDataFrames."""
def __init__(self, pandas_obj):
diff --git a/Stoner/core/data.py b/Stoner/core/data.py
index 2ac088ab7..46050e45e 100755
--- a/Stoner/core/data.py
+++ b/Stoner/core/data.py
@@ -20,7 +20,6 @@
class Data(AnalysisMixin, FittingMixin, ColumnOpsMixin, FilteringOpsMixin, FeatureOpsMixin, PlotMixin, DataFile):
-
"""The principal class for representing a data file.
This merges:
diff --git a/Stoner/core/exceptions.py b/Stoner/core/exceptions.py
index 47d72dab6..586547bdc 100755
--- a/Stoner/core/exceptions.py
+++ b/Stoner/core/exceptions.py
@@ -4,7 +4,6 @@
class StonerLoadError(Exception):
-
"""An exception thrown by the file loading routines in the Stoner Package.
This special exception is thrown when one of the subclasses of :py:class:`Stoner.Core.DataFile`
@@ -15,7 +14,6 @@ class StonerLoadError(Exception):
class StonerUnrecognisedFormat(IOError):
-
"""An exception thrown by the file loading routines in the Stoner Package.
This special exception is thrown when none of the subclasses was able to load the specified file.
@@ -23,12 +21,10 @@ class StonerUnrecognisedFormat(IOError):
class StonerSetasError(AttributeError):
-
"""An exception tjrown when we try to access a column in data without setas being set."""
class StonerAssertionError(RuntimeError):
-
"""An exception raised when the library thinks an assertion has failed."""
diff --git a/Stoner/core/interfaces.py b/Stoner/core/interfaces.py
index 52e3299d4..121b3ad28 100755
--- a/Stoner/core/interfaces.py
+++ b/Stoner/core/interfaces.py
@@ -10,7 +10,6 @@
class DataFileInterfacesMixin:
-
"""Implement the required methods for a sequence and mapping type object."""
def __contains__(self, item):
diff --git a/Stoner/core/methods.py b/Stoner/core/methods.py
index a575bd4e9..b69488618 100755
--- a/Stoner/core/methods.py
+++ b/Stoner/core/methods.py
@@ -13,7 +13,6 @@
class DataFileSearchMixin:
-
"""Mixin class that provides the search, selecting and sorting methods for a DataFile."""
def _search_index(self, xcol=None, value=None, accuracy=0.0, invert=False):
diff --git a/Stoner/core/operators.py b/Stoner/core/operators.py
index 215800282..d8dbdf191 100755
--- a/Stoner/core/operators.py
+++ b/Stoner/core/operators.py
@@ -11,7 +11,6 @@
class DataFileOperatorsMixin:
-
"""Provides the operator mixins for DataFile like objects."""
def __add__(self, other):
diff --git a/Stoner/core/property.py b/Stoner/core/property.py
index a11f99134..c29a92809 100755
--- a/Stoner/core/property.py
+++ b/Stoner/core/property.py
@@ -27,7 +27,6 @@
class DataFilePropertyMixin:
-
"""Provide the properties for DataFile Like Objects."""
_subclasses = None
diff --git a/Stoner/core/setas.py b/Stoner/core/setas.py
index aef7acaea..acb9b485f 100755
--- a/Stoner/core/setas.py
+++ b/Stoner/core/setas.py
@@ -14,7 +14,6 @@
class setas(MutableMapping):
-
"""A Class that provides a mechanism for managing the column assignments in a DataFile like object.
Implements a MutableMapping bsed on the column_headers as the keys (with a few tweaks!).
diff --git a/Stoner/core/utils.py b/Stoner/core/utils.py
index cf7408f9b..f1c61032c 100755
--- a/Stoner/core/utils.py
+++ b/Stoner/core/utils.py
@@ -200,7 +200,6 @@ def sub_core(other: Union[Int_Types, slice, Callable], newdata: "DataFile") -> "
class tab_delimited(csv.Dialect):
-
"""A customised csv dialect class for reading tab delimited text files."""
delimiter = "\t"
diff --git a/Stoner/folders/core.py b/Stoner/folders/core.py
index 3e6253cbe..99529a169 100755
--- a/Stoner/folders/core.py
+++ b/Stoner/folders/core.py
@@ -178,7 +178,6 @@ def _build_select_function(kargs, arg):
class baseFolder(MutableSequence):
-
"""A base class for objectFolders that supports both a sequence of objects and a mapping of instances of itself.
Attributes:
@@ -346,7 +345,8 @@ def depth(self):
def each(self):
"""Return a :py:class:`Stoner.folders.each.item` proxy object.
- This is for calling attributes of the member type of the folder."""
+ This is for calling attributes of the member type of the folder.
+ """
return EachItem(self)
@property
@@ -441,7 +441,8 @@ def lsgrp(self):
def metadata(self):
"""Return a :py:class:`Stoner.folders.metadata.MetadataProxy` object.
- This allows for operations on combined metadata."""
+ This allows for operations on combined metadata.
+ """
return MetadataProxy(self)
@property
@@ -948,15 +949,15 @@ def __repr__(self):
cls = type(self).__name__
pth = self.key
pattern = getattr(self, "pattern", "")
- s = f"{cls}({pth}) with pattern {pattern} has {len(self)} files and {len(self.groups)} groups\n"
+ string = f"{cls}({pth}) with pattern {pattern} has {len(self)} files and {len(self.groups)} groups\n"
if not short:
- for r in self.ls:
- s += "\t" + r + "\n"
+ for row in self.ls:
+ string += "\t" + row + "\n"
for g in self.groups: # iterate over groups
r = self.groups[g].__repr__()
- for l in r.split("\n"): # indent each line by one tab
- s += "\t" + l + "\n"
- return s.strip()
+ for line in r.split("\n"): # indent each line by one tab
+ string += "\t" + line + "\n"
+ return string.strip()
def __reversed__(self):
"""Create an iterator function that runs backwards through the stored objects."""
@@ -1348,8 +1349,8 @@ def group(self, key):
Args:
key (string or callable or list):
- Either a simple string or callable function or a list. If a string then it is interpreted as an item of
- metadata in each file. If a callable function then takes a single argument x which should be an
+ Either a simple string or callable function or a list. If a string then it is interpreted as an item
+ of metadata in each file. If a callable function then takes a single argument x which should be an
instance of a metadataObject and returns some vale. If key is a list then the grouping is
done recursely for each element in key.
@@ -1473,7 +1474,7 @@ def make_name(self, value=None):
name = f"Untitled-{self._last_name}"
return name
- def pop(self, name=-1, default=None): # pylint: disable=arguments-differ
+ def pop(self, name=-1, default=None): # pylint: disable=arguments-differ,arguments-renamed
"""Return and remove either a subgroup or named object from this folder."""
try:
ret = self[name]
diff --git a/Stoner/folders/each.py b/Stoner/folders/each.py
index a7c158262..36cf7d691 100755
--- a/Stoner/folders/each.py
+++ b/Stoner/folders/each.py
@@ -1,4 +1,5 @@
"""Classes and support functions for the :py:attr:`Stoner.DataFolder.each`.magic attribute."""
+
__all__ = ["Item"]
from collections.abc import MutableSequence
from functools import wraps, partial
@@ -35,7 +36,6 @@ def _worker(d, **kwargs):
class SetasWrapper(MutableSequence):
-
"""Manages wrapping each member of the folder's setas attribute."""
def __init__(self, parent):
@@ -76,6 +76,7 @@ def __setitem__(self, index, value):
"""
if len(value) < len(self._folder):
value = value + value[-1] * (len(self._folder) - len(value))
+ v = "."
for v, data in zip(value, self._folder):
data.setas[index] = v
setas = self._folder._object_attrs.get("setas", self.collapse())
@@ -107,7 +108,6 @@ def collapse(self):
class Item:
-
"""Provides a proxy object for accessing methods on the inividual members of a Folder.
Notes:
diff --git a/Stoner/folders/groups.py b/Stoner/folders/groups.py
index 8bec57a24..586a26d36 100755
--- a/Stoner/folders/groups.py
+++ b/Stoner/folders/groups.py
@@ -11,7 +11,6 @@
class GroupsDict(regexpDict):
-
"""A typeHinted dictionary to manages collections of :py:class:`Stoner.folders.core.baseFolder` objects."""
def __init__(self, *args, **kargs):
diff --git a/Stoner/folders/metadata.py b/Stoner/folders/metadata.py
index 8834a4e03..0a5681cab 100755
--- a/Stoner/folders/metadata.py
+++ b/Stoner/folders/metadata.py
@@ -108,7 +108,6 @@ def _slice_keys(args, possible=None):
class MetadataProxy(MutableMapping):
-
"""Provide methods to interact with a whole collection of metadataObjects' metadata."""
def __init__(self, folder):
diff --git a/Stoner/folders/mixins.py b/Stoner/folders/mixins.py
index 66f8b3600..eb0fd1a4d 100755
--- a/Stoner/folders/mixins.py
+++ b/Stoner/folders/mixins.py
@@ -13,7 +13,7 @@
from numpy import mean, std, array, append, any as np_any, floor, sqrt, ceil
from numpy.ma import masked_invalid
-from matplotlib.pyplot import figure, tight_layout, subplots
+from matplotlib.pyplot import figure, subplots
from Stoner.tools import isiterable, make_Data
from ..compat import string_types, get_filedialog, _pattern_type, makedirs, path_types
@@ -57,7 +57,6 @@ def _loader(name, loader=None, typ=None, directory=None):
class DiskBasedFolderMixin:
-
"""A Mixin class that implements reading metadataObjects from disc.
Attributes:
@@ -469,7 +468,6 @@ def unload(self, name=None):
class DataMethodsMixin:
-
"""Methods for working with :py:class:`Stner.Data` in py:class:`Stoner.DataFolder`s."""
def concatenate(self, sort=None, reverse=False):
@@ -639,7 +637,6 @@ def _gatherer(group, _, xcol=None, ycol=None, xerr=None, yerr=None, **kargs):
class PlotMethodsMixin:
-
"""A Mixin for :py:class:`Stoner.folders.core.baseFolder` with extra methods for plotting lots of files.
Example:
@@ -683,8 +680,6 @@ def plot(self, *args, **kargs):
Passed to matplotlib figure call.
plots_per_page(int):
maximum number of plots per figure.
- tight_layout(dict or False):
- If not False, arguments to pass to a call of :py:func:`matplotlib.pyplot.tight_layout`. Defaults to {}
Returns:
A list of :py:class:`matplotlib.pyplot.Axes` instances.
@@ -705,7 +700,6 @@ def plot(self, *args, **kargs):
self[i] = make_Data(d)
extra = kargs.pop("extra", lambda i, j, d: None)
- tight = kargs.pop("tight_layout", {})
fig_args = getattr(self, "_fig_args", [])
fig_kargs = getattr(self, "_fig_kargs", {})
@@ -721,26 +715,22 @@ def plot(self, *args, **kargs):
fignum = fig.number
for i, d in enumerate(self):
if i % plts == 0 and i != 0:
- if isinstance(tight, dict):
- tight_layout(**tight)
fig, axs = subplots(plt_x, plt_y, *fig_args, **fig_kargs)
fignum = fig.number
j = 0
else:
j += 1
fig = figure(fignum)
- kargs["fig"] = fig
+ kargs["figure"] = fig
kargs["ax"] = axs.ravel()[j]
ret.append(d.plot(*args, **kargs))
extra(i, j, d)
for n in range(j + 1, plt_x * plt_y):
axs.ravel()[n].remove()
- tight_layout()
return ret
class DataFolder(DataMethodsMixin, DiskBasedFolderMixin, baseFolder):
-
"""Provide an interface to manipulating lots of data files stored within a directory structure on disc.
By default, the members of the DataFolder are instances of :class:`Stoner.Data`. The DataFolder emplys a lazy
@@ -756,5 +746,4 @@ def __init__(self, *args, **kargs):
class PlotFolder(PlotMethodsMixin, DataFolder):
-
"""A :py:class:`Stoner.folders.baseFolder` that knows how to ploth its underlying data files."""
diff --git a/Stoner/formats/__init__.py b/Stoner/formats/__init__.py
index c23eacaea..bdbc1481c 100755
--- a/Stoner/formats/__init__.py
+++ b/Stoner/formats/__init__.py
@@ -17,5 +17,6 @@
provide a :py:attr:`Stoner.Core.DataFile.mime_type` attribute which gives a list of mime types that this class might
be able to open. This helps identify classes that could be use to load particular file types.
"""
+
__all__ = ["instruments", "generic", "rigs", "facilities", "simulations", "attocube", "maximus"]
from . import instruments, generic, rigs, facilities, simulations, attocube, maximus
diff --git a/Stoner/formats/attocube.py b/Stoner/formats/attocube.py
index d2078b560..26f81d038 100755
--- a/Stoner/formats/attocube.py
+++ b/Stoner/formats/attocube.py
@@ -48,7 +48,6 @@ def _raise_error(openfile, message=""):
class AttocubeScanMixin:
-
"""Provides the specialist methods for dealing with Attocube SPM scan files.
See :py:class:`AttocubeScan` for details."""
@@ -468,7 +467,6 @@ def read_hdf5(cls, filename, *args, **kargs):
class AttocubeScan(AttocubeScanMixin, ImageStack):
-
"""An ImageStack subclass that can load scans from the AttocubeScan SPM System.
AttocubeScan represents a scan from an Attocube SPM system as a 3D stack of scan data with
diff --git a/Stoner/formats/data/__init__.py b/Stoner/formats/data/__init__.py
new file mode 100755
index 000000000..68a51d388
--- /dev/null
+++ b/Stoner/formats/data/__init__.py
@@ -0,0 +1,8 @@
+"""Sub-package of routines to load Data objects in different formats.
+
+This sub-package is lazy-loaded but then pulls in all of the modules in order to register the routines.
+"""
+
+__all__ = ["generic", "hdf5", "instruments", "facilities", "rigs", "simulations", "attocube", "maximus", "zip"]
+
+from . import generic, hdf5, instruments, facilities, rigs, simulations, attocube, maximus, zip
diff --git a/Stoner/formats/data/attocube.py b/Stoner/formats/data/attocube.py
new file mode 100755
index 000000000..00dbd8509
--- /dev/null
+++ b/Stoner/formats/data/attocube.py
@@ -0,0 +1,52 @@
+# -*- coding: utf-8 -*-
+"""Module to work with scan files from an AttocubeSPM running Daisy."""
+import re
+
+from ...tools.file import FileManager
+from ...core.exceptions import StonerLoadError
+from ..decorators import register_loader
+
+PARAM_RE = re.compile(r"^([\d\\.eE\+\-]+)\s*([\%A-Za-z]\S*)?$")
+SCAN_NO = re.compile(r"SC_(\d+)")
+
+
+def parabola(X, cx, cy, a, b, c):
+ """Parabola in the X-Y plane for levelling an image."""
+ x, y = X
+ return a * (x - cx) ** 2 + b * (y - cy) ** 2 + c
+
+
+def plane(X, a, b, c):
+ """Plane equation for levelling an image."""
+ x, y = X
+ return a * x + b * y + c
+
+
+@register_loader(patterns=(".txt", 32), mime_types=("text/plain", 32), name="AttocubeScanParametersFile", what="Data")
+def load_attocube_parameters(new_data, filename, *args, **kargs):
+ """Load the scan parameters text file as the metadata for a Data File.
+
+ Args:
+ root_name (str):
+ The scan prefix e.g. SC_###
+
+ Returns:
+ new_data:
+ The modififed scan stack.
+ """
+ new_data.filename = filename
+ with FileManager(filename, "r") as parameters:
+ if not parameters.readline().startswith("Daisy Parameter Snapshot"):
+ raise StonerLoadError("Parameters file exists but does not have correct header")
+ for line in parameters:
+ if not line.strip():
+ continue
+ parts = [x.strip() for x in line.strip().split(":")]
+ key = parts[0]
+ value = ":".join(parts[1:])
+ units = PARAM_RE.match(value)
+ if units and units.groups()[1]:
+ key += f" [{units.groups()[1]}]"
+ value = units.groups()[0]
+ new_data[key] = value
+ return new_data
diff --git a/Stoner/formats/data/facilities.py b/Stoner/formats/data/facilities.py
new file mode 100755
index 000000000..164761832
--- /dev/null
+++ b/Stoner/formats/data/facilities.py
@@ -0,0 +1,306 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+"""Implements DataFile like classes for various large scale facilities."""
+
+# Standard Library imports
+import linecache
+import re
+
+import numpy as np
+
+from ...compat import str2bytes
+from ...core.base import string_to_type
+from ...tools.file import FileManager
+from ...core.exceptions import StonerLoadError
+from ..decorators import register_loader
+
+try:
+ import fabio
+except ImportError:
+ fabio = None
+
+
+def _bnl_find_lines(new_data):
+ """Return an array of ints [header_line,data_line,scan_line,date_line,motor_line]."""
+ with FileManager(new_data.filename, "r", errors="ignore", encoding="utf-8") as fp:
+ new_data.line_numbers = [0, 0, 0, 0, 0]
+ counter = 0
+ for line in fp:
+ counter += 1
+ if counter == 1 and line[0] != "#":
+ raise StonerLoadError("Not a BNL File ?")
+ if len(line) < 2:
+ continue # if there's nothing written on the line go to the next
+ if line[0:2] == "#L":
+ new_data.line_numbers[0] = counter
+ elif line[0:2] == "#S":
+ new_data.line_numbers[2] = counter
+ elif line[0:2] == "#D":
+ new_data.line_numbers[3] = counter
+ elif line[0:2] == "#P":
+ new_data.line_numbers[4] = counter
+ elif line[0] in ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"]:
+ new_data.line_numbers[1] = counter
+ break
+
+
+def _bnl_get_metadata(new_data):
+ """Load metadta from file.
+
+ Metadata found is scan number 'Snumber', scan type and parameters 'Stype',
+ scan date/time 'Sdatetime' and z motor position 'Smotor'.
+ """
+ scanLine = linecache.getline(new_data.filename, new_data.line_numbers[2])
+ dateLine = linecache.getline(new_data.filename, new_data.line_numbers[3])
+ motorLine = linecache.getline(new_data.filename, new_data.line_numbers[4])
+ new_data.__setitem__("Snumber", scanLine.split()[1])
+ tmp = "".join(scanLine.split()[2:])
+ new_data.__setitem__("Stype", "".join(tmp.split(","))) # get rid of commas
+ new_data.__setitem__("Sdatetime", dateLine[3:-1]) # don't want \n at end of line so use -1
+ new_data.__setitem__("Smotor", motorLine.split()[3])
+
+
+def _parse_bnl_data(new_data):
+ """Parse BNL data.
+
+ The meta data is labelled by #L type tags
+ so easy to find but #L must be excluded from the result.
+ """
+ _bnl_find_lines(new_data)
+ # creates a list, line_numbers, formatted [header_line,data_line,scan_line,date_line,motor_line]
+ header_string = linecache.getline(new_data.filename, new_data.line_numbers[0])
+ header_string = re.sub(r'["\n]', "", header_string) # get rid of new line character
+ header_string = re.sub(r"#L", "", header_string) # get rid of line indicator character
+ column_headers = map(lambda x: x.strip(), header_string.split())
+ _bnl_get_metadata(new_data)
+ try:
+ new_data.data = np.genfromtxt(new_data.filename, skip_header=new_data.line_numbers[1] - 1)
+ except IOError:
+ new_data.data = np.array([0])
+ print(f"Did not import any data for {new_data.filename}")
+ new_data.column_headers = column_headers
+
+
+@register_loader(patterns=(".txt", 64), mime_types=("text/plain", 64), name="BNLFile", what="Data")
+def load_bnl(new_data, filename, *args, **kargs): # pylint: disable=unused-argument
+ """Load the file from disc.
+
+ Args:
+ filename (string or bool):
+ File to load. If None then the existing filename is used, if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+
+ Notes:
+ Overwrites load method in Core.DataFile class, no header positions and data
+ positions are needed because of the hash title structure used in BNL files.
+
+ Normally its good to use _parse_plain_data method from Core.DataFile class
+ to load data but unfortunately Brookhaven data isn't very plain so there's
+ a new method below.
+ """
+ new_data.filename = filename
+ try:
+ _parse_bnl_data(new_data) # call an internal function rather than put it in load function
+ except (IndexError, TypeError, ValueError, StonerLoadError) as err:
+ raise StonerLoadError("Not parseable as a NFLS file!") from err
+ linecache.clearcache()
+ return new_data
+
+
+def _read_mdaascii_header(data, new_data, i):
+ """Read the header block."""
+ for i[0], line in enumerate(data):
+ line.strip()
+ if "=" in line:
+ parts = line[2:].split("=")
+ new_data[parts[0].strip()] = string_to_type("".join(parts[1:]).strip())
+ elif line.startswith("# Extra PV:"):
+ # Onto the next metadata bit
+ break
+
+
+def _read_mdaascii_metadata(data, new_data, i):
+ """Read the metadata block."""
+ pvpat = re.compile(r"^#\s+Extra\s+PV\s\d+\:(.*)")
+ for i[1], line in enumerate(data):
+ if line.strip() == "":
+ continue
+ if line.startswith("# Extra PV"):
+ res = pvpat.match(line)
+ bits = [b.strip().strip(r'"') for b in res.group(1).split(",")]
+ if bits[1] == "":
+ key = bits[0]
+ else:
+ key = bits[1]
+ if len(bits) > 3:
+ key = f"{key} ({bits[3]})"
+ new_data[key] = string_to_type(bits[2])
+ else:
+ break # End of Extra PV stuff
+ else:
+ raise StonerLoadError("Overran Extra PV Block")
+ for i[2], line in enumerate(data):
+ line.strip()
+ if line.strip() == "":
+ continue
+ elif line.startswith("# Column Descriptions:"):
+ break # Start of column headers now
+ elif "=" in line:
+ parts = line[2:].split("=")
+ new_data[parts[0].strip()] = string_to_type("".join(parts[1:]).strip())
+ else:
+ raise StonerLoadError("Overran end of scan header before column descriptions")
+
+
+def _read_mdaascii_columns(data, new_data, i):
+ """Reads the column header block."""
+ colpat = re.compile(r"#\s+\d+\s+\[([^\]]*)\](.*)")
+ column_headers = []
+ for i[3], line in enumerate(data):
+ res = colpat.match(line)
+ line.strip()
+ if line.strip() == "":
+ continue
+ elif line.startswith("# 1-D Scan Values"):
+ break # Start of data
+ elif res is not None:
+ if "," in res.group(2):
+ bits = [b.strip() for b in res.group(2).split(",")]
+ if bits[-2] == "":
+ colname = bits[0]
+ else:
+ colname = bits[-2]
+ if bits[-1] != "":
+ colname += f"({bits[-1]})"
+ if colname in column_headers:
+ colname = f"{bits[0]}:{colname}"
+ else:
+ colname = res.group(1).strip()
+ column_headers.append(colname)
+ else:
+ raise StonerLoadError("Overand the end of file without reading data")
+ new_data.column_headers = column_headers
+
+
+@register_loader(patterns=(".txt", 32), mime_types=("text/plain", 32), name="MDAASCIIFile", what="Data")
+def load_mdaasci(new_data, filename, *args, **kargs): # pylint: disable=unused-argument
+ """Load function. File format has space delimited columns from row 3 onwards."""
+ new_data.filename = filename
+ i = [0, 0, 0, 0]
+ with FileManager(new_data.filename, "r", errors="ignore", encoding="utf-8") as data: # Slightly ugly text handling
+ if next(data).strip() != "## mda2ascii 1.2 generated output":
+ raise StonerLoadError("Not a file mda2ascii")
+ _read_mdaascii_header(data, new_data, i)
+ _read_mdaascii_metadata(data, new_data, i)
+ _read_mdaascii_columns(data, new_data, i)
+ new_data.data = np.genfromtxt(data) # so that's ok then !
+ return new_data
+
+
+@register_loader(patterns=(".dat", 16), mime_types=("text/plain", 16), name="OpenGDAFile", what="Data")
+def load_gda(new_data, filename=None, *args, **kargs):
+ """Load an OpenGDA file.
+
+ Args:
+ filename (string or bool): File to load. If None then the existing filename is used,
+ if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+ """
+ new_data.filename = filename
+ i = 0
+ with FileManager(new_data.filename, "r", errors="ignore", encoding="utf-8") as f:
+ for i, line in enumerate(f):
+ line = line.strip()
+ if i == 0 and line != "&SRS":
+ raise StonerLoadError(f"Not a GDA File from Rasor ?\n{line}")
+ if "&END" in line:
+ break
+ parts = line.split("=")
+ if len(parts) != 2:
+ continue
+ key = parts[0]
+ value = parts[1].strip()
+ new_data.metadata[key] = string_to_type(value)
+ column_headers = f.readline().strip().split("\t")
+ new_data.data = np.genfromtxt([str2bytes(l) for l in f], dtype="float", invalid_raise=False)
+ new_data.column_headers = column_headers
+ return new_data
+
+
+@register_loader(patterns=(".dat", 16), mime_types=("text/plain", 16), name="SNSFile", what="Data")
+def load_sns(new_data, filename=None, *args, **kargs):
+ """Load function. File format has space delimited columns from row 3 onwards."""
+ new_data.filename = filename
+ with FileManager(new_data.filename, "r", errors="ignore", encoding="utf-8") as data: # Slightly ugly text handling
+ line = data.readline()
+ if not line.strip().startswith(
+ "# Datafile created by QuickNXS 0.9.39"
+ ): # bug out oif we don't like the header
+ raise StonerLoadError("Not a file from the SNS BL4A line")
+ for line in data:
+ if line.startswith("# "): # We're in the header
+ line = line[2:].strip() # strip the header and whitespace
+
+ if line.startswith("["): # Look for a section header
+ section = line.strip().strip("[]")
+ if section == "Data": # The Data section has one line of column headers and then data
+ header = next(data)[2:].split("\t")
+ column_headers = [h.strip() for h in header]
+ new_data.data = np.genfromtxt(data) # we end by reading the raw data
+ elif section == "Global Options": # This section can go into metadata
+ for line in data:
+ line = line[2:].strip()
+ if line.strip() == "":
+ break
+ else:
+ new_data[line[2:10].strip()] = line[11:].strip()
+ elif (
+ section == "Direct Beam Runs" or section == "Data Runs"
+ ): # These are constructed into lists ofg dictionaries for each file
+ sec = list()
+ header = next(data)
+ header = header[2:].strip()
+ keys = [s.strip() for s in header.split(" ") if s.strip()]
+ for line in data:
+ line = line[2:].strip()
+ if line == "":
+ break
+ else:
+ values = [s.strip() for s in line.split(" ") if s.strip()]
+ sec.append(dict(zip(keys, values)))
+ new_data[section] = sec
+ else: # We must still be in the opening un-labelled section of meta data
+ if ":" in line:
+ i = line.index(":")
+ key = line[:i].strip()
+ value = line[i + 1 :].strip()
+ new_data[key.strip()] = value.strip()
+ new_data.column_headers = column_headers
+ return new_data
+
+
+if fabio:
+
+ @register_loader(
+ patterns=(".edf", 16),
+ mime_types=[("application/octet-stream", 16), ("text/plain", 15)],
+ name="ESRF_DataFile",
+ what="Data",
+ )
+ def load_esrf(self, filename=None, *args, **kargs):
+ """Load function. File format has space delimited columns from row 3 onwards."""
+ if filename is None or not filename:
+ self.get_filename("r")
+ else:
+ self.filename = filename
+ try:
+ img = fabio.edfimage.edfimage().read(self.filename)
+ self.data = img.data
+ self.metadata.update(img.header)
+ return self
+ except (OSError, ValueError, TypeError, IndexError) as err:
+ raise StonerLoadError("Not an ESRF data file !") from err
diff --git a/Stoner/formats/data/generic.py b/Stoner/formats/data/generic.py
new file mode 100755
index 000000000..b6ee18f12
--- /dev/null
+++ b/Stoner/formats/data/generic.py
@@ -0,0 +1,516 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+"""Implement DataFile classes for some generic file formats."""
+import csv
+import contextlib
+import copy
+import io
+from pathlib import Path
+import re
+from collections.abc import Mapping
+import sys
+import logging
+from typing import Optional
+import warnings
+
+import PIL
+import numpy as np
+
+from ...compat import str2bytes, Hyperspy_ok, hs, hsload
+from ...core.array import DataArray
+from ...core.exceptions import StonerLoadError
+from ...core.utils import tab_delimited
+from ...tools import make_Data
+
+from ...tools.file import FileManager
+
+from ..decorators import register_loader, register_saver
+
+
+class _refuse_log(logging.Filter):
+ """Refuse to log all records."""
+
+ def filter(self, record):
+ """Do not log anything."""
+ return False
+
+
+@contextlib.contextmanager
+def catch_sysout(*args):
+ """Temporarily redirect sys.stdout and.sys.stdin."""
+ stdout, stderr = sys.stdout, sys.stderr
+ out = io.StringIO()
+ sys.stdout, sys.stderr = out, out
+ logger = logging.getLogger("hyperspy.io")
+ logger.addFilter(_refuse_log)
+ yield None
+ logger.removeFilter(_refuse_log)
+ sys.stdout, sys.stderr = stdout, stderr
+ return
+
+
+def _delim_detect(line):
+ """Detect a delimiter in a line.
+
+ Args:
+ line(str):
+ String to search for delimiters in.
+
+ Returns:
+ (str):
+ Delimiter to use.
+
+ Raises:
+ StnerLoadError:
+ If delimiter cannot be located.
+ """
+ quotes = re.compile(r"([\"\'])[^\1]*\1")
+ line = quotes.sub("", line) # Remove quoted strings first
+ current = (None, len(line))
+ for delim in "\t ,;":
+ try:
+ idx = line.index(delim)
+ except ValueError:
+ continue
+ if idx < current[1]:
+ current = (delim, idx)
+ if current[0] is None:
+ raise StonerLoadError("Unable to find a delimiter in the line")
+ return current[0]
+
+
+@register_loader(
+ patterns=[(".dat", 8), (".txt", 8), ("*", 8)],
+ mime_types=[("application/tsv", 8), ("text/plain", 8), ("text/tab-separated-values", 8)],
+ name="DataFile",
+ what="Data",
+)
+def load_tdi_format(new_data, filename, *args, **kargs):
+ """Actually load the data from disc assuming a .tdi file format.
+
+ Args:
+ filename (str):
+ Path to filename to be loaded. If None or False, a dialog bax is raised to ask for the filename.
+
+ Returns:
+ DataFile:
+ A copy of the newly loaded :py:class`DataFile` object.
+
+ Exceptions:
+ StonerLoadError:
+ Raised if the first row does not start with 'TDI Format 1.5' or 'TDI Format=1.0'.
+
+ Note:
+ The *_load* methods shouldbe overridden in each child class to handle the process of loading data from
+ disc. If they encounter unexpected data, then they should raise StonerLoadError to signal this, so that
+ the loading class can try a different sub-class instead.
+ """
+ if filename is None or not filename:
+ new_data.get_filename("r")
+ else:
+ new_data.filename = filename
+ with FileManager(new_data.filename, "r", encoding="utf-8", errors="ignore") as datafile:
+ line = datafile.readline()
+ if line.startswith("TDI Format 1.5"):
+ fmt = 1.5
+ elif line.startswith("TDI Format=Text 1.0"):
+ fmt = 1.0
+ else:
+ raise StonerLoadError("Not a TDI File")
+
+ datafile.seek(0)
+ reader = csv.reader(datafile, dialect=tab_delimited())
+ cols = 0
+ for ix, metadata in enumerate(reader):
+ if ix == 0:
+ row = metadata
+ continue
+ if len(metadata) < 1:
+ continue
+ if cols == 0:
+ cols = len(metadata)
+ if len(metadata) > 1:
+ max_rows = ix + 1
+ if "=" in metadata[0]:
+ new_data.metadata.import_key(metadata[0])
+ col_headers_tmp = [x.strip() for x in row[1:]]
+ with warnings.catch_warnings():
+ datafile.seek(0)
+ warnings.filterwarnings("ignore", "Some errors were detected !")
+ data = np.genfromtxt(
+ datafile,
+ skip_header=1,
+ usemask=True,
+ delimiter="\t",
+ usecols=range(1, cols),
+ invalid_raise=False,
+ comments="\0",
+ missing_values=[""],
+ filling_values=[np.nan],
+ max_rows=max_rows,
+ )
+ if data.ndim < 2:
+ data = np.ma.atleast_2d(data)
+ retain = np.all(np.isnan(data), axis=1)
+ new_data.data = DataArray(data[~retain])
+ new_data["TDI Format"] = fmt
+ new_data["Stoner.class"] = "Data"
+ if new_data.data.ndim == 2 and new_data.data.shape[1] > 0:
+ new_data.column_headers = col_headers_tmp
+ new_data.metadata = copy.deepcopy(new_data.metadata) # This fixes some type issues TODO - work out why!
+ return new_data
+
+
+@register_saver(
+ patterns=[(".dat", 8), (".txt", 8), ("*", 8)],
+ name="DataFile",
+ what="Data",
+)
+def save_tdi_format(save_data, filename):
+ """Write out a DataFile to a tab delimited tdi text file."""
+
+ header = ["TDI Format 1.5"]
+ header.extend(save_data.column_headers[: save_data.data.shape[1]])
+ header = "\t".join(header)
+ mdkeys = sorted(save_data.metadata)
+ if len(mdkeys) > len(save_data):
+ mdremains = mdkeys[len(save_data) :]
+ mdkeys = mdkeys[0 : len(save_data)]
+ else:
+ mdremains = []
+ mdtext = np.array([save_data.metadata.export(k) for k in mdkeys])
+ if len(mdtext) < len(save_data):
+ mdtext = np.append(mdtext, np.zeros(len(save_data) - len(mdtext), dtype=str))
+ data_out = np.column_stack([mdtext, save_data.data])
+ fmt = ["%s"] * data_out.shape[1]
+ with io.open(filename, "w", errors="replace", encoding="utf-8") as f:
+ np.savetxt(f, data_out, fmt=fmt, header=header, delimiter="\t", comments="")
+ for k in mdremains:
+ f.write(save_data.metadata.export(k) + "\n") # (str2bytes(save_data.metadata.export(k) + "\n"))
+
+ save_data.filename = filename
+ return save_data
+
+
+@register_loader(
+ patterns=[(".csv", 32), (".txt", 256)],
+ mime_types=[("application/csv", 16), ("text/plain", 256), ("text/csv", 16)],
+ name="CSVFile",
+ what="Data",
+)
+def load_csvfile(new_data: "DataFile", filename: str, *args, **kargs) -> "DataFile":
+ """Load generic deliminated files.
+
+ Args:
+ filename (string or bool): File to load. If None then the existing filename is used,
+ if False, then a file dialog will be used.
+
+ Keyword Arguments:
+ header_line (int): The line in the file that contains the column headers.
+ If None, then column headers are automatically generated.
+ data_line (int): The line on which the data starts
+ data_delim (string): The delimiter used for separating data values
+ header_delim (strong): The delimiter used for separating header values
+
+ Returns:
+ A copy of the current object after loading the data.
+ """
+ _defaults = {"header_line": 0, "data_line": 1, "header_delim": ",", "data_delim": ","}
+
+ header_line = kargs.pop("header_line", _defaults["header_line"])
+ data_line = kargs.pop("data_line", _defaults["data_line"])
+ data_delim = kargs.pop("data_delim", _defaults["data_delim"])
+ header_delim = kargs.pop("header_delim", _defaults["header_delim"])
+
+ new_data.filename = filename
+
+ if data_delim is None or header_delim is None: #
+ with FileManager(new_data.filename, "r") as datafile:
+ lines = datafile.readlines()
+ if header_line is not None and header_delim is None:
+ header_delim = _delim_detect(lines[header_line])
+ if data_line is not None and data_delim is None:
+ data_delim = _delim_detect(lines[data_line])
+
+ with FileManager(new_data.filename, "r") as datafile:
+ if header_line is not None:
+ try:
+ for ix, line in enumerate(datafile):
+ if ix == header_line:
+ break
+ else:
+ raise StonerLoadError("Ran out of file before readching header")
+ header = line.strip()
+ column_headers = next(csv.reader(io.StringIO(header), delimiter=header_delim))
+ data = np.genfromtxt(datafile, delimiter=data_delim, skip_header=data_line - header_line)
+ except (TypeError, ValueError, csv.Error, StopIteration, UnicodeDecodeError) as err:
+ raise StonerLoadError("Header and data on the same line") from err
+ else: # Generate
+ try:
+ data = np.genfromtxt(datafile, delimiter=data_delim, skip_header=data_line)
+ except (TypeError, ValueError) as err:
+ raise StonerLoadError("Failed to open file as CSV File") from err
+ column_headers = ["Column" + str(x) for x in range(np.shape(data)[1])]
+
+ new_data.data = data
+ new_data.column_headers = column_headers
+ new_data.metadata |= kargs
+ return new_data
+
+
+@register_saver(patterns=[(".csv", 32), (".txt", 256)], name="CSVFile", what="Data")
+def save_csvfile(save_data: "DataFile", filename: Optional[str] = None, **kargs):
+ """Override the save method to allow CSVFiles to be written out to disc (as a mininmalist output).
+
+ Args:
+ filename (string): Filename to save as (using the same rules as for the load routines)
+
+ Keyword Arguments:
+ deliminator (string): Record deliniminator (defaults to a comma)
+ no_header (bool): Whether to skip the headers, defaults to False (include colu,n headers)
+
+ Returns:
+ A copy of itsave_data.
+ """
+ delimiter = kargs.pop("deliminator", ",")
+ if filename is None:
+ filename = save_data.filename
+ no_header = kargs.get("no_header", False)
+ if filename is None or (isinstance(filename, bool) and not filename): # now go and ask for one
+ filename = save_data.__file_dialog("w")
+ with FileManager(filename, "w") as outfile:
+ spamWriter = csv.writer(outfile, delimiter=delimiter, quotechar='"', quoting=csv.QUOTE_MINIMAL)
+ i = 0
+ if not no_header:
+ spamWriter.writerow(save_data.column_headers)
+ while i < save_data.data.shape[0]:
+ spamWriter.writerow(save_data.data[i, :])
+ i += 1
+ save_data.filename = filename
+ return save_data
+
+
+@register_loader(
+ patterns=[(".csv", 64), (".txt", 512)],
+ mime_types=[("application/csv", 64), ("text/plain", 512), ("text/csv", 64)],
+ name="JustNumbers",
+ what="Data",
+)
+def load_justnumbers(new_data: "DataFile", filename: str, *args, **kargs) -> "DataFile":
+ """Load generic deliminated files with no headers or metadata.
+
+ Args:
+ filename (str):
+ File to load.
+
+ Keyword Arguments:
+ data_delim (string): The delimiter used for separating data values
+ header_delim (strong): The delimiter used for separating header values
+
+ Returns:
+ A copy of the current object after loading the data.
+ """
+ _defaults = {"header_line": None, "data_line": 0, "data_delim": None}
+ for karg, val in _defaults.items():
+ kargs.setdefault(karg, val)
+ return load_csvfile(new_data, filename, *args, **kargs)
+
+
+@register_saver(patterns=[(".csv", 32), (".txt", 256)], name="JustNumbersFile", what="Data")
+def save_justnumbers(save_data: "DataFile", filename: Optional[str] = None, **kargs):
+ """Override the save method to allow JustNumbersFiles to be written out to disc (as a very mininmalist output).
+
+ Args:
+ filename (string): Filename to save as (using the same rules as for the load routines)
+
+ Keyword Arguments:
+ deliminator (string): Record deliniminator (defaults to a comma)
+
+ Returns:
+ A copy of itsave_data.
+ """
+ kargs["no_header"] = False
+ return save_csvfile(save_data, filename, **kargs)
+
+
+def _check_png_signature(filename):
+ """Check that this is a PNG file and raie a StonerLoadError if not."""
+ try:
+ with FileManager(filename, "rb") as test:
+ sig = test.read(8)
+ sig = [x for x in sig]
+ if sig != [137, 80, 78, 71, 13, 10, 26, 10]:
+ raise StonerLoadError("Signature mismatrch")
+ except (StonerLoadError, IOError) as err:
+ from traceback import format_exc
+
+ raise StonerLoadError(f"Not a PNG file!>\n{format_exc()}") from err
+ return True
+
+
+@register_loader(
+ patterns=(".png", 16),
+ mime_types=("image/png", 16),
+ name="KermitPNGFile",
+ what="Data",
+)
+def load_pngfile(new_data, filename=None, *args, **kargs):
+ """PNG file loader routine.
+
+ Args:
+ filename (string or bool): File to load. If None then the existing filename is used,
+ if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+ """
+ new_data.filename = filename
+ _check_png_signature(filename)
+ try:
+ with PIL.Image.open(new_data.filename, "r") as img:
+ for k in img.info:
+ new_data.metadata[k] = img.info[k]
+ new_data.data = np.asarray(img)
+ except IOError as err:
+ raise StonerLoadError("Unable to read as a PNG file.") from err
+ new_data["Stoner.class"] = "Data" # Reset my load class
+ new_data.metadata = copy.deepcopy(new_data.metadata) # Fixes up some data types (see TDI loader)
+ return new_data
+
+
+@register_saver(patterns=(".png", 8), name="ImageFile", what="Image")
+@register_saver(patterns=(".png", 16), name="KermitPngFile", what="Data")
+def save_pngfile(save_data, filename=None, **kargs):
+ """Override the save method to allow KermitPNGFiles to be written out to disc.
+
+ Args:
+ filename (string): Filename to save as (using the same rules as for the load routines)
+
+ Keyword Arguments:
+ deliminator (string): Record deliniminator (defaults to a comma)
+
+ Returns:
+ A copy of itsave_data.
+ """
+ if filename is None:
+ filename = save_data.filename
+ if filename is None or (isinstance(filename, bool) and not filename): # now go and ask for one
+ filename = save_data.__file_dialog("w")
+
+ metadata = PIL.PngImagePlugin.PngInfo()
+ for k in save_data.metadata:
+ parts = save_data.metadata.export(k).split("=")
+ key = parts[0]
+ val = str2bytes("=".join(parts[1:]))
+ metadata.add_text(key, val)
+ img = PIL.Image.fromarray(save_data.data)
+ img.save(filename, "png", pnginfo=metadata)
+ save_data.filename = filename
+ return save_data
+
+
+try: # Optional tdms support
+ from nptdms import TdmsFile
+
+ @register_loader(
+ patterns=[(".tdms", 16), (".tdms_index", 16)],
+ mime_types=("application/octet-stream", 16),
+ name="TDMSFile",
+ what="Data",
+ )
+ def load_tdms(new_data, filename=None, *args, **kargs):
+ """TDMS file loader routine.
+
+ Args:
+ filename (string or bool): File to load. If None then the existing filename is used,
+ if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+ """
+ filename = Path(filename)
+ if filename.suffix == ".tdms_index":
+ filename = filename.parent / f"{filename.stem}.tdms" # rewrite filename for not the index file!
+ new_data.filename = filename
+ # Open the file and read the main file header and unpack into a dict
+ try:
+ f = TdmsFile(new_data.filename)
+ for grp in f.groups():
+ if grp.path == "/":
+ pass # skip the rooot group
+ elif grp.path == "/'TDI Format 1.5'":
+ tmp = make_Data(grp.as_dataframe())
+ new_data.data = tmp.data
+ new_data.column_headers = tmp.column_headers
+ new_data.metadata.update(grp.properties)
+ else:
+ tmp = make_Data(grp.as_dataframe())
+ new_data.data = tmp.data
+ new_data.column_headers = tmp.column_headers
+ except (IOError, ValueError, TypeError, StonerLoadError) as err:
+ from traceback import format_exc
+
+ raise StonerLoadError(f"Not a TDMS File \n{format_exc()}") from err
+
+ return new_data
+
+except ImportError:
+ pass
+
+if Hyperspy_ok:
+
+ def _unpack_hyperspy_meta(new_data, root, value):
+ """Recursively unpack a nested dict of metadata and append keys to new_data.metadata."""
+ if isinstance(value, Mapping):
+ for item in value.keys():
+ if root != "":
+ _unpack_hyperspy_meta(new_data, f"{root}.{item}", value[item])
+ else:
+ _unpack_hyperspy_meta(new_data, f"{item}", value[item])
+ else:
+ new_data.metadata[root] = value
+
+ def _unpack_hyperspy_axes(newdata, ax_manager):
+ """Unpack the axes managber as metadata."""
+ _axes_keys = ["name", "scale", "low_index", "low_value", "high_index", "high_value"]
+ for ax in ax_manager.signal_axes:
+ for k in _axes_keys:
+ newdata.metadata[f"{ax.name}.{k}"] = getattr(ax, k)
+
+ @register_loader(
+ patterns=[(".emd", 32), (".dm4", 32)],
+ mime_types=[("application/x-hdf", 64), ("application/x-hdf5", 64)],
+ name="HyperSpyFile",
+ what="Data",
+ )
+ def hypersput_load(new_data, filename=None, *args, **kargs):
+ """Load HyperSpy file loader routine.
+
+ Args:
+ filename (string or bool): File to load. If None then the existing filename is used,
+ if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+ """
+ if filename is None or not filename:
+ new_data.get_filename("r")
+ else:
+ new_data.filename = filename
+ # Open the file and read the main file header and unpack into a dict
+ try:
+ with catch_sysout():
+ signal = hsload(new_data.filename)
+ if hasattr(hs, "signals"):
+ Signal2D = hs.signals.Signal2D
+ else:
+ Signal2D = hs.api.signals.Signal2D
+ if not isinstance(signal, Signal2D):
+ raise StonerLoadError("Not a 2D signal object - aborting!")
+ except Exception as err: # pylint: disable=W0703 Pretty generic error catcher
+ raise StonerLoadError(f"Not readable by HyperSpy error was {err}") from err
+ new_data.data = signal.data
+ _unpack_hyperspy_meta(new_data, "", signal.metadata.as_dictionary())
+ _unpack_hyperspy_axes(new_data, signal.axes_manager)
+ del new_data["General.FileIO.0.timestamp"]
+ return new_data
diff --git a/Stoner/formats/data/hdf5.py b/Stoner/formats/data/hdf5.py
new file mode 100755
index 000000000..4b767e41c
--- /dev/null
+++ b/Stoner/formats/data/hdf5.py
@@ -0,0 +1,271 @@
+# -*- coding: utf-8 -*-
+"""HDF5 format Data loader routines"""
+import os
+
+import h5py
+import numpy as np
+from importlib import import_module
+
+from ...compat import string_types, bytes2str, path_types, str2bytes
+from ...core.exceptions import StonerLoadError
+from ..decorators import register_loader, register_saver
+from ...tools.file import HDFFileManager
+
+
+@register_loader(
+ patterns=[(".hdf", 16), (".hf5", 16)],
+ mime_types=[("application/x-hdf", 16), ("application/x-hdf5", 16)],
+ name="HDF5File",
+ what="Data",
+)
+def load_hdf(new_data, filename, *args, **kargs): # pylint: disable=unused-argument
+ """Create a new HDF5File from an actual HDF file."""
+ with HDFFileManager(filename, "r") as f:
+ data = f["data"]
+ if np.prod(np.array(data.shape)) > 0:
+ new_data.data = data[...]
+ else:
+ new_data.data = [[]]
+ metadata = f.require_group("metadata")
+ typehints = f.get("typehints", None)
+ if not isinstance(typehints, h5py.Group):
+ typehints = {}
+ else:
+ typehints = typehints.attrs
+ if "column_headers" in f.attrs:
+ new_data.column_headers = [bytes2str(x) for x in f.attrs["column_headers"]]
+ if isinstance(new_data.column_headers, string_types):
+ new_data.column_headers = new_data.metadata.string_to_type(new_data.column_headers)
+ new_data.column_headers = [bytes2str(x) for x in new_data.column_headers]
+ else:
+ raise StonerLoadError("Couldn't work out where my column headers were !")
+ for i in sorted(metadata.attrs):
+ v = metadata.attrs[i]
+ t = typehints.get(i, "Detect")
+ if isinstance(v, string_types) and t != "Detect": # We have typehints and this looks like it got exported
+ new_data.metadata[f"{i}{{{t}}}".strip()] = f"{v}".strip()
+ else:
+ new_data[i] = metadata.attrs[i]
+ if isinstance(f, h5py.Group):
+ if f.name != "/":
+ new_data.filename = os.path.join(f.file.filename, f.name)
+ else:
+ new_data.filename = os.path.realpath(f.file.filename)
+ else:
+ new_data.filename = os.path.realpath(f.filename)
+ return new_data
+
+
+@register_saver(patterns=[(".hdf", 16), (".hf5", 16)], name="HDF5File", what="Data")
+def save_hdf(save_data, filename=None, **kargs): # pylint: disable=unused-argument
+ """Write the current object into an hdf5 file or group within a file.
+
+ Writes the data in afashion that is compatible with being loaded in again.
+
+ Args:
+ filename (string or h5py.Group):
+ Either a string, of h5py.File or h5py.Group object into which to save the file. If this is a string,
+ the corresponding file is opened for writing, written to and save again.
+
+ Returns
+ A copy of the object
+ """
+ if filename is None:
+ filename = save_data.filename
+ if filename is None or (isinstance(filename, bool) and not filename): # now go and ask for one
+ filename = save_data.__file_dialog("w")
+ save_data.filename = filename
+ if isinstance(filename, path_types):
+ mode = "r+" if os.path.exists(filename) else "w"
+ else:
+ mode = ""
+ compression = "gzip"
+ compression_opts = 6
+ with HDFFileManager(filename, mode) as f:
+ f.require_dataset(
+ "data",
+ data=save_data.data,
+ shape=save_data.data.shape,
+ dtype=save_data.data.dtype,
+ )
+ metadata = f.require_group("metadata")
+ typehints = f.require_group("typehints")
+ for k in save_data.metadata:
+ try:
+ typehints.attrs[k] = save_data.metadata._typehints[k]
+ metadata.attrs[k] = save_data[k]
+ except TypeError:
+ # We get this for trying to store a bad data type - fallback to metadata export to string
+ parts = save_data.metadata.export(k).split("=")
+ metadata.attrs[k] = "=".join(parts[1:])
+ f.attrs["column_headers"] = [str2bytes(x) for x in save_data.column_headers]
+ f.attrs["filename"] = save_data.filename
+ f.attrs["type"] = type(save_data).__name__
+ f.attrs["module"] = type(save_data).__module__
+ return save_data
+
+
+def _scan_SLS_meta(new_data, group):
+ """Scan the HDF5 Group for attributes and datasets and sub groups and recursively add them to the metadata."""
+ root = ".".join(group.name.split("/")[2:])
+ for name, thing in group.items():
+ parts = thing.name.split("/")
+ name = ".".join(parts[2:])
+ if isinstance(thing, h5py.Group):
+ _scan_SLS_meta(new_data, thing)
+ elif isinstance(thing, h5py.Dataset):
+ if thing.ndim > 1:
+ continue
+ if np.prod(thing.shape) == 1:
+ new_data.metadata[name] = thing[0]
+ else:
+ new_data.metadata[name] = thing[...]
+ for attr in group.attrs:
+ new_data.metadata[f"{root}.{attr}"] = group.attrs[attr]
+
+
+@register_loader(
+ patterns=[(".hdf", 16), (".hdf5", 16)],
+ mime_types=[("application/x-hdf", 16), ("application/x-hdf5", 16)],
+ name="SLS_STXMFile",
+ what="Data",
+)
+def load_sls_stxm(new_data, filename, *args, **kargs):
+ """Load data from the hdf5 file produced by Pollux.
+
+ Args:
+ h5file (string or h5py.Group):
+ Either a string or an h5py Group object to load data from
+
+ Returns:
+ itnew_data after having loaded the data
+ """
+ new_data.filename = filename
+ if isinstance(filename, path_types): # We got a string, so we'll treat it like a file...
+ try:
+ f = h5py.File(filename, "r")
+ except IOError as err:
+ raise StonerLoadError(f"Failed to open {filename} as a n hdf5 file") from err
+ elif isinstance(filename, h5py.File) or isinstance(filename, h5py.Group):
+ f = filename
+ else:
+ raise StonerLoadError(f"Couldn't interpret {filename} as a valid HDF5 file or group or filename")
+ items = [x for x in f.items()]
+ if len(items) == 1 and items[0][0] == "entry1":
+ group1 = [x for x in f["entry1"]]
+ if "definition" in group1 and bytes2str(f["entry1"]["definition"][0]) == "NXstxm": # Good HDF5
+ pass
+ else:
+ raise StonerLoadError("HDF5 file lacks single top level group called entry1")
+ else:
+ raise StonerLoadError("HDF5 file lacks single top level group called entry1")
+ root = f["entry1"]
+ data = root["counter0"]["data"]
+ if np.prod(np.array(data.shape)) > 0:
+ new_data.data = data[...]
+ else:
+ new_data.data = [[]]
+ _scan_SLS_meta(new_data, root)
+ if "file_name" in f.attrs:
+ new_data["original filename"] = f.attrs["file_name"]
+ elif isinstance(f, h5py.Group):
+ new_data["original filename"] = f.name
+ else:
+ new_data["original filename"] = f.file.filename
+
+ if isinstance(filename, path_types):
+ f.file.close()
+ new_data["Loaded from"] = new_data.filename
+
+ if "instrument.sample_x.data" in new_data.metadata:
+ new_data.metadata["actual_x"] = new_data.metadata["instrument.sample_x.data"].reshape(new_data.shape)
+ if "instrument.sample_y.data" in new_data.metadata:
+ new_data.metadata["actual_y"] = new_data.metadata["instrument.sample_y.data"].reshape(new_data.shape)
+ new_data.metadata["sample_x"], new_data.metadata["sample_y"] = np.meshgrid(
+ new_data.metadata["counter0.sample_x"], new_data.metadata["counter0.sample_y"]
+ )
+ if "control.data" in new_data.metadata:
+ mod = import_module("Stoner.Image")
+ ImageArray = getattr(mod, "ImageArray")
+
+ new_data.metadata["beam current"] = ImageArray(new_data.metadata["control.data"].reshape(new_data.data.shape))
+ new_data.metadata["beam current"].metadata = new_data.metadata
+ new_data.data = new_data.data[::-1]
+ return new_data
+
+
+def _hgx_scan_group(new_data, grp, pth):
+ """Recursively list HDF5 Groups."""
+ if pth in new_data.seen:
+ return None
+ new_data.seen.append(pth)
+ if not isinstance(grp, h5py.Group):
+ return None
+ if new_data.debug:
+ if new_data.debug:
+ print(f"Scanning in {pth}")
+ for x in grp:
+ if pth == "":
+ new_pth = x
+ else:
+ new_pth = pth + "." + x
+ if pth == "" and x == "data": # Special case for main data
+ continue
+ if isinstance(grp[x], type(grp)):
+ _hgx_scan_group(new_data, grp[x], new_pth)
+ elif isinstance(grp[x], h5py.Dataset):
+ y = grp[x][...]
+ new_data[new_pth] = y
+ return None
+
+
+def _hgx_main_data(new_data, data_grp):
+ """Work through the main data group and build something that looks like a numpy 2D array."""
+ if not isinstance(data_grp, h5py.Group) or data_grp.name != "/current/data":
+ raise StonerLoadError("HDF5 file not in expected format")
+ root = data_grp["datasets"]
+ for ix in root: # Hack - iterate over all items in root, but actually data is in Groups not DataSets
+ dataset = root[ix]
+ if isinstance(dataset, h5py.Dataset):
+ continue
+ x = dataset["x"][...]
+ y = dataset["y"][...]
+ e = dataset["error"][...]
+ new_data &= x
+ new_data &= y
+ new_data &= e
+ new_data.column_headers[-3] = bytes2str(dataset["x_command"][()])
+ new_data.column_headers[-2] = bytes2str(dataset["y_command"][()])
+ new_data.column_headers[-1] = bytes2str(dataset["error_command"][()])
+ new_data.column_headers = [str(ix) for ix in new_data.column_headers]
+
+
+@register_loader(
+ patterns=(".hgx", 16),
+ mime_types=[("application/x-hdf", 32), ("application/x-hdf5", 32)],
+ name="HGXFile",
+ what="Data",
+)
+def _load(new_data, filename, *args, **kargs):
+ """Load a GenX HDF file.
+
+ Args:
+ filename (string or bool):
+ File to load. If None then the existing filename is used, if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+ """
+ new_data.filename = filename
+ new_data.seen = []
+ with HDFFileManager(new_data.filename, "r") as f:
+ try:
+ if "current" in f and "config" in f["current"]:
+ pass
+ else:
+ raise StonerLoadError("Looks like an unexpected HDF layout!.")
+ _hgx_scan_group(new_data, f["current"], "")
+ _hgx_main_data(new_data, f["current"]["data"])
+ except IOError as err:
+ raise StonerLoadError("Looks like an unexpected HDF layout!.") from err
+ return new_data
diff --git a/Stoner/formats/data/instruments.py b/Stoner/formats/data/instruments.py
new file mode 100755
index 000000000..dbd2648f4
--- /dev/null
+++ b/Stoner/formats/data/instruments.py
@@ -0,0 +1,718 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+"""Implement DataFile like classes to represent Instrument Manufacturer's File Formats'."""
+
+# Standard Library imports
+from datetime import datetime
+import re
+import struct
+from ast import literal_eval
+
+import numpy as np
+
+from ...compat import str2bytes, bytes2str
+from ...core.exceptions import StonerAssertionError, assertion, StonerLoadError
+from ...core.base import string_to_type
+from ...tools.file import FileManager, SizedFileManager
+
+from ..decorators import register_loader, register_saver
+
+
+@register_loader(patterns=(".340", 16), mime_types=("text/plain", 32), name="LSTemperatureFile", what="Data")
+def load_340(new_data, filename=None, *args, **kargs):
+ """Load data for 340 files."""
+ new_data.filename = filename
+ with FileManager(new_data.filename, "rb") as data:
+ keys = []
+ vals = []
+ for line in data:
+ line = bytes2str(line)
+ if line.strip() == "":
+ break
+ parts = [p.strip() for p in line.split(":")]
+ if len(parts) != 2:
+ raise StonerLoadError(f"Header doesn't contain two parts at {line.strip()}")
+ keys.append(parts[0])
+ vals.append(parts[1])
+ else:
+ raise StonerLoadError("Overan the end of the file")
+ if keys != [
+ "Sensor Model",
+ "Serial Number",
+ "Data Format",
+ "SetPoint Limit",
+ "Temperature coefficient",
+ "Number of Breakpoints",
+ ]:
+ raise StonerLoadError("Header did not contain recognised keys.")
+ for k, v in zip(keys, vals):
+ v = v.split()[0]
+ new_data.metadata[k] = string_to_type(v)
+ headers = bytes2str(next(data)).strip().split()
+ column_headers = headers[1:]
+ dat = np.genfromtxt(data)
+ new_data.data = dat[:, 1:]
+ new_data.column_headers = column_headers
+ return new_data
+
+
+@register_saver(patterns=".340", name="LSTemperatureFile", what="Data")
+def save_340(save_data, filename=None, **kargs):
+ """Override the save method to allow CSVFiles to be written out to disc (as a mininmalist output).
+
+ Args:
+ filename (string):
+ Filename to save as (using the same rules as for the load routines)
+
+ Keyword Arguments:
+ deliminator (string):
+ Record deliniminator (defaults to a comma)
+
+ Returns:
+ A copy of itsave_data.
+ """
+ if filename is None:
+ filename = save_data.filename
+ if filename is None or (isinstance(filename, bool) and not filename): # now go and ask for one
+ filename = save_data.__file_dialog("w")
+ if save_data.shape[1] == 2: # 2 columns, let's hope they're the right way round!
+ cols = [0, 1]
+ elif (
+ save_data.setas.has_xcol and save_data.setas.has_ycol
+ ): # Use ycol, x col but assume x is real temperature and y is resistance
+ cols = [save_data.setas.ycol[0], save_data.setas.xcol]
+ else:
+ cols = range(save_data.shape[1])
+ with FileManager(filename, "w", errors="ignore", encoding="utf-8", newline="\r\n") as f:
+ for k, v in (
+ ("Sensor Model", "CX-1070-SD"),
+ ("Serial Number", "Unknown"),
+ ("Data Format", 4),
+ ("SetPoint Limit", 300.0),
+ ("Temperature coefficient", 1),
+ ("Number of Breakpoints", len(save_data)),
+ ):
+ if k in ["Sensor Model", "Serial Number", "Data Format", "SetPoint Limit"]:
+ kstr = f"{k+':':16s}"
+ else:
+ kstr = f"{k}: "
+ v = save_data.get(k, v)
+ if k == "Data Format":
+ units = ["()", "()", "()", "()", "(Log Ohms/Kelvin)", "(Log Ohms/Log Kelvin)"]
+ vstr = f"{v} {units[int(v)]}"
+ elif k == "SetPointLimit":
+ vstr = f"{v} (Kelvin)"
+ elif k == "Temperature coefficient":
+ vstr = f"{v} {['(positive)', '(negative)'][v]}"
+ elif k == "Number of Breakpoints":
+ vstr = str(len(save_data))
+ else:
+ vstr = str(v)
+ f.write(f"{kstr}{vstr}\n")
+ f.write("\n")
+ f.write("No. ")
+ for i in cols:
+ f.write(f"{save_data.column_headers[i]:11s}")
+ f.write("\n\n")
+ for i in range(
+ len(save_data.data)
+ ): # This is a slow way to write the data, but there should only ever be 200 lines
+ line = "\t".join([f"{n:<10.8f}" for n in save_data.data[i, cols]])
+ f.write(f"{i}\t")
+ f.write(f"{line}\n")
+ save_data.filename = filename
+ return save_data
+
+
+@register_loader(
+ patterns=(".dat", 16),
+ mime_types=[("application/x-wine-extension-ini", 15), ("text/plain", 16)],
+ name="QDFile",
+ what="Data",
+)
+def load_qdfile(new_data, filename=None, *args, **kargs):
+ """QD system file loader routine.
+
+ Args:
+ filename (string or bool):
+ File to load. If None then the existing filename is used, if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+ """
+ setas = {}
+ i = 0
+ new_data.filename = filename
+ with FileManager(new_data.filename, "r", encoding="utf-8", errors="ignore") as f: # Read filename linewise
+ for i, line in enumerate(f):
+ line = line.strip()
+ if i == 0 and line != "[Header]":
+ raise StonerLoadError("Not a Quantum Design File !")
+ if line == "[Header]" or line.startswith(";") or line == "":
+ continue
+ if "[Data]" in line:
+ break
+ if "," not in line:
+ raise StonerLoadError("No data in file!")
+ parts = [x.strip() for x in line.split(",")]
+ if parts[1].split(":")[0] == "SEQUENCE FILE":
+ key = parts[1].split(":")[0].title()
+ value = parts[1].split(":")[1]
+ elif parts[0] == "INFO":
+ if parts[1] == "APPNAME":
+ parts[1], parts[2] = parts[2], parts[1]
+ if len(parts) > 2:
+ key = f"{parts[0]}.{parts[2]}"
+ else:
+ raise StonerLoadError("No data in file!")
+ key = key.title()
+ value = parts[1]
+ elif parts[0] in ["BYAPP", "FILEOPENTIME"]:
+ key = parts[0].title()
+ value = " ".join(parts[1:])
+ elif parts[0] == "FIELDGROUP":
+ key = f"{parts[0]}.{parts[1]}".title()
+ value = f'[{",".join(parts[2:])}]'
+ elif parts[0] == "STARTUPAXIS":
+ axis = parts[1][0].lower()
+ setas[axis] = setas.get(axis, []) + [int(parts[2])]
+ key = f"Startupaxis-{parts[1].strip()}"
+ value = parts[2].strip()
+ else:
+ key = parts[0] + "," + parts[1]
+ key = key.title()
+ value = " ".join(parts[2:])
+ new_data.metadata[key] = string_to_type(value)
+ else:
+ raise StonerLoadError("No data in file!")
+ if "Byapp" not in new_data:
+ raise StonerLoadError("Not a Quantum Design File !")
+
+ column_headers = f.readline().strip().split(",")
+ data = np.genfromtxt([str2bytes(l) for l in f], dtype="float", delimiter=",", invalid_raise=False)
+ if data.shape[0] == 0:
+ raise StonerLoadError("No data in file!")
+ if data.shape[1] < len(column_headers): # Trap for buggy QD software not giving ewnough columns of data
+ data = np.append(data, np.ones((data.shape[0], len(column_headers) - data.shape[1])) * np.NaN, axis=1)
+ elif data.shape[1] > len(column_headers): # too much data
+ data = data[:, : len(column_headers) - data.shape[1]]
+ new_data.data = data
+ new_data.column_headers = column_headers
+ s = new_data.setas
+ for k in setas:
+ for ix in setas[k]:
+ s[ix - 1] = k
+ new_data.setas = s
+ return new_data
+
+
+def _to_Q(new_data, wavelength=1.540593):
+ """Add an additional function to covert an angualr scale to momentum transfer.
+
+ Returns:
+ a copy of itnew_data.
+ """
+ new_data.add_column(
+ (4 * np.pi / wavelength) * np.sin(np.pi * new_data.column(0) / 360), header="Momentum Transfer, Q ($\\AA$)"
+ )
+
+
+@register_loader(
+ patterns=(".ras", 16),
+ mime_types=[("application/x-wine-extension-ini", 16), ("text/plain", 16)],
+ name="RigakuFile",
+ what="Data",
+)
+def load_rigaku(new_data, filename=None, *args, **kargs):
+ """Read a Rigaku ras file including handling the metadata nicely.
+
+ Args:
+ filename (string or bool):
+ File to load. If None then the existing filename is used, if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+ """
+ sh = re.compile(r"^\*([^\s]+)\s+(.*)$") # Regexp to grab the keys
+ ka = re.compile(r"(.*)\-(\d+)$")
+ header = {}
+ i = 0
+ new_data.filename = filename
+ with SizedFileManager(new_data.filename, "rb") as (f, end):
+ for i, line in enumerate(f):
+ line = bytes2str(line).strip()
+ if i == 0 and not line.startswith("*RAS_"):
+ raise StonerLoadError("Not a Rigaku file!")
+ if line == "*RAS_HEADER_START":
+ break
+ for line in f:
+ line = bytes2str(line).strip()
+ m = sh.match(line)
+ if m:
+ key = m.groups()[0].lower().replace("_", ".")
+ try:
+ value = m.groups()[1].decode("utf-8", "ignore")
+ except AttributeError:
+ value = m.groups()[1]
+ header[key] = value
+ if "*RAS_INT_START" in line:
+ break
+ keys = list(header.keys())
+ keys.sort()
+ for key in keys:
+ m = ka.match(key)
+ value = header[key].strip()
+ try:
+ newvalue = literal_eval(value.strip('"'))
+ except (TypeError, ValueError, SyntaxError):
+ newvalue = literal_eval(value)
+ if newvalue == "-":
+ newvalue = np.nan # trap for missing float value
+ if m:
+ key = m.groups()[0]
+ idx = int(m.groups()[1])
+ if key in new_data.metadata and not (isinstance(new_data[key], (np.ndarray, list))):
+ if isinstance(new_data[key], str):
+ new_data[key] = list([new_data[key]])
+ if idx > 1:
+ new_data[key].extend([""] * idx - 1)
+ else:
+ new_data[key] = np.array(new_data[key])
+ if idx > 1:
+ new_data[key] = np.append(new_data[key], np.ones(idx - 1) * np.nan)
+ if key not in new_data.metadata:
+ if isinstance(newvalue, str):
+ listval = [""] * (idx + 1)
+ listval[idx] = newvalue
+ new_data[key] = listval
+ else:
+ arrayval = np.ones(idx + 1) * np.nan
+ arrayval = arrayval.astype(type(newvalue))
+ arrayval[idx] = newvalue
+ new_data[key] = arrayval
+ else:
+ if isinstance(new_data[key][0], str) and isinstance(new_data[key], list):
+ if len(new_data[key]) < idx + 1:
+ new_data[key].extend([""] * (idx + 1 - len(new_data[key])))
+ new_data[key][idx] = newvalue
+ else:
+ if idx + 1 > new_data[key].size:
+ new_data[key] = np.append(
+ new_data[key],
+ (np.ones(idx + 1 - new_data[key].size) * np.nan).astype(new_data[key].dtype),
+ )
+ try:
+ new_data[key][idx] = newvalue
+ except ValueError:
+ pass
+ else:
+ new_data.metadata[key] = newvalue
+
+ pos = f.tell()
+ max_rows = 0
+ for max_rows, line in enumerate(f):
+ line = bytes2str(line).strip()
+ if "RAS_INT_END" in line:
+ break
+ f.seek(pos)
+ if max_rows > 0:
+ new_data.data = np.genfromtxt(
+ f, dtype="float", delimiter=" ", invalid_raise=False, comments="*", max_rows=max_rows
+ )
+ column_headers = ["Column" + str(i) for i in range(new_data.data.shape[1])]
+ column_headers[0:2] = [new_data.metadata["meas.scan.unit.x"], new_data.metadata["meas.scan.unit.y"]]
+ for key in new_data.metadata:
+ if isinstance(new_data[key], list):
+ new_data[key] = np.array(new_data[key])
+ new_data.setas = "xy"
+ new_data.column_headers = column_headers
+ pos = f.tell()
+ if pos < end: # Trap for Rigaku files with multiple scans in them.
+ new_data["_endpos"] = pos
+ if hasattr(filename, "seekable") and filename.seekable():
+ filename.seek(pos)
+ if kargs.pop("add_Q", False):
+ _to_Q(new_data)
+ return new_data
+
+
+def _read_spc_xdata(new_data, f):
+ """Read the xdata from the spc file."""
+ new_data._pts = new_data._header["fnpts"]
+ if new_data._header["ftflgs"] & 128: # We need to read some X Data
+ if 4 * new_data._pts > new_data._filesize - f.tell():
+ raise StonerLoadError("Trying to read too much data!")
+ xvals = f.read(4 * new_data._pts) # I think storing X vals directly implies that each one is 4 bytes....
+ xdata = np.array(struct.unpack(str2bytes(str(new_data._pts) + "f"), xvals))
+ else: # Generate the X Data ourselves
+ first = new_data._header["ffirst"]
+ last = new_data._header["flast"]
+ if new_data._pts > 1e6: # Something not right here !
+ raise StonerLoadError("More than 1 million points requested. Bugging out now!")
+ xdata = np.linspace(first, last, new_data._pts)
+ return xdata
+
+
+def _read_spc_ydata(new_data, f, data, column_headers):
+ """Read the y data and column headers from spc file."""
+ n = new_data._header["fnsub"]
+ subhdr_keys = (
+ "subflgs",
+ "subexp",
+ "subindx",
+ "subtime",
+ "subnext",
+ "subnois",
+ "subnpts",
+ "subscan",
+ "subwlevel",
+ "subresv",
+ )
+ if new_data._header["ftflgs"] & 1:
+ y_width = 2
+ y_fmt = "h"
+ divisor = 2**16
+ else:
+ y_width = 4
+ y_fmt = "i"
+ divisor = 2**32
+ if n * (y_width * new_data._pts + 32) > new_data._filesize - f.tell():
+ raise StonerLoadError("No good, going to read too much data!")
+ for j in range(n): # We have n sub-scans
+ # Read the subheader and import into the main metadata dictionary as scan#:
+ subhdr = struct.unpack(b"BBHfffIIf4s", f.read(32))
+ subheader = dict(zip(["scan" + str(j) + ":" + x for x in subhdr_keys], subhdr))
+
+ # Now read the y-data
+ exponent = subheader["scan" + str(j) + ":subexp"]
+ if int(exponent) & -128: # Data is unscaled direct floats
+ ydata = np.array(struct.unpack(str2bytes(str(new_data._pts) + "f"), f.read(new_data._pts * y_width)))
+ else: # Data is scaled by exponent
+ yvals = struct.unpack(str2bytes(str(new_data._pts) + y_fmt), f.read(new_data._pts * y_width))
+ ydata = np.array(yvals, dtype="float64") * (2**exponent) / divisor
+ data[:, j + 1] = ydata
+ new_data._header = dict(new_data._header, **subheader)
+ column_headers.append("Scan" + str(j) + ":" + new_data._yvars[new_data._header["fytype"]])
+
+ return data
+
+
+def _read_spc_loginfo(new_data, f):
+ """Read the log info section of the spc file."""
+ logstc = struct.unpack(b"IIIII44s", f.read(64))
+ logstc_keys = ("logsizd", "logsizm", "logtxto", "logbins", "logdsks", "logrsvr")
+ logheader = dict(zip(logstc_keys, logstc))
+ new_data._header = dict(new_data._header, **logheader)
+
+ # Can't handle either binary log information or ion disk log information (wtf is this anyway !)
+ if new_data._header["logbins"] + new_data._header["logdsks"] > new_data._filesize - f.tell():
+ raise StonerLoadError("Too much logfile data to read")
+ f.read(new_data._header["logbins"] + new_data._header["logdsks"])
+
+ # The renishaw seems to put a 16 character timestamp next - it's not in the spec but never mind that.
+ new_data._header["Date-Time"] = f.read(16)
+ # Now read the rest of the file as log text
+ logtext = f.read()
+ # We expect things to be single lines terminated with a CR-LF of the format key=value
+ for line in re.split(b"[\r\n]+", logtext):
+ if b"=" in line:
+ parts = line.split(b"=")
+ key = parts[0].decode()
+ value = parts[1].decode()
+ new_data._header[key] = value
+
+
+@register_loader(patterns=(".spc", 16), mime_types=("application/octet-stream", 16), name="SPCFile", what="Data")
+def load_spc(new_data, filename=None, *args, **kargs):
+ """Read a .scf file produced by the Renishaw Raman system (among others).
+
+ Args:
+ filename (string or bool):
+ File to load. If None then the existing filename is used, if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+
+ Todo:
+ Implement the second form of the file that stores multiple x-y curves in the one file.
+
+ Notes:
+ Metadata keys are pretty much as specified in the spc.h file that defines the filerformat.
+ """
+ if filename is None or not filename:
+ new_data.get_filename("r")
+ else:
+ new_data.filename = filename
+ # Open the file and read the main file header and unpack into a dict
+ with SizedFileManager(filename, "rb") as (f, length):
+ new_data._filesize = length
+ spchdr = struct.unpack(b"BBBciddiBBBBi9s9sH8f30s130siiBBHf48sfifB187s", f.read(512))
+ keys = (
+ "ftflgs",
+ "fversn",
+ "fexper",
+ "fexp",
+ "fnpts",
+ "ffirst",
+ "flast",
+ "fnsub",
+ "fxtype",
+ "fytype",
+ "fztype",
+ "fpost",
+ "fres",
+ "fsource",
+ "fpeakpt",
+ "fspare1",
+ "fspare2",
+ "fspare3",
+ "fspare4",
+ "fspare5",
+ "fspare6",
+ "fspare7",
+ "fspare8",
+ "fcm",
+ "nt",
+ "fcatx",
+ "flogoff",
+ "fmods",
+ "fprocs",
+ "flevel",
+ "fsampin",
+ "ffactor",
+ "fmethod",
+ "fzinc",
+ "fwplanes",
+ "fwinc",
+ "fwtype",
+ "fwtype",
+ "fresv",
+ )
+ new_data._xvars = [
+ "Arbitrary",
+ "Wavenumber (cm-1)",
+ "Micrometers (um)",
+ "Nanometers (nm)",
+ "Seconds",
+ "Minutes",
+ "Hertz (Hz)",
+ "Kilohertz (KHz)",
+ "Megahertz (MHz)",
+ "Mass (M/z)",
+ "Parts per million (PPM)",
+ "Days",
+ "Years",
+ "Raman Shift (cm-1)",
+ "Raman Shift (cm-1)",
+ "eV",
+ "XYZ text labels in fcatxt (old 0x4D version only)",
+ "Diode Number",
+ "Channel",
+ "Degrees",
+ "Temperature (F)",
+ "Temperature (C)",
+ "Temperature (K)",
+ "Data Points",
+ "Milliseconds (mSec)",
+ "Microseconds (uSec)",
+ "Nanoseconds (nSec)",
+ "Gigahertz (GHz)",
+ "Centimeters (cm)",
+ "Meters (m)",
+ "Millimeters (mm)",
+ "Hours",
+ "Hours",
+ ]
+ new_data._yvars = [
+ "Arbitrary Intensity",
+ "Interferogram",
+ "Absorbance",
+ "Kubelka-Monk",
+ "Counts",
+ "Volts",
+ "Degrees",
+ "Milliamps",
+ "Millimeters",
+ "Millivolts",
+ "Log(1/R)",
+ "Percent",
+ "Percent",
+ "Intensity",
+ "Relative Intensity",
+ "Energy",
+ "Decibel",
+ "Temperature (F)",
+ "Temperature (C)",
+ "Temperature (K)",
+ "Index of Refraction [N]",
+ "Extinction Coeff. [K]",
+ "Real",
+ "Imaginary",
+ "Complex",
+ "Complex",
+ "Transmission (ALL HIGHER MUST HAVE VALLEYS!)",
+ "Reflectance",
+ "Arbitrary or Single Beam with Valley Peaks",
+ "Emission",
+ "Emission",
+ ]
+
+ new_data._header = dict(zip(keys, spchdr))
+ n = new_data._header["fnsub"]
+
+ if new_data._header["ftflgs"] & 64 == 64 or not (
+ 75 <= new_data._header["fversn"] <= 77
+ ): # This is the multiple XY curves in file flag.
+ raise StonerLoadError(
+ f"Filetype not implemented yet ! {new_data._header['ftflgs']=}, {new_data._header['fversn']=}"
+ )
+ # Read the xdata and add it to the file.
+ xdata = _read_spc_xdata(new_data, f)
+ data = np.zeros((new_data._pts, (n + 1))) # initialise the data soace
+ data[:, 0] = xdata # Put in the X-Data
+ column_headers = [new_data._xvars[new_data._header["fxtype"]]] # And label the X column correctly
+
+ # Now we're going to read the Y-data
+ data = _read_spc_ydata(new_data, f, data, column_headers)
+ if new_data._header["flogoff"] != 0: # Ok, we've got a log, so read the log header and merge into metadata
+ _read_spc_loginfo(new_data, f)
+ # Ok now build the Stoner.DataFile instance to return
+ new_data.data = data
+ # The next bit generates the metadata. We don't just copy the metadata because we need to figure out
+ # the typehints first - hence the loop
+ # here to call DataFile.__setitem()
+ for x in new_data._header:
+ new_data[x] = new_data._header[x]
+ new_data.column_headers = column_headers
+ if len(new_data.column_headers) == 2:
+ new_data.setas = "xy"
+ return new_data
+
+
+@register_loader(patterns=[(".fld", 16), (".dat", 32)], mime_types=("text/plain", 16), name="VSMFile", what="Data")
+def load_vsm(new_data, filename=None, *args, header_line=3, data_line=3, header_delim=",", **kargs):
+ """VSM file loader routine.
+
+ Args:
+ filename (string or bool):
+ File to load. If None then the existing filename is used, if False, then a file dialog will be used.
+
+ Keyword Arguments:
+ header_line (int):
+ The line in the file that contains the column headers. If None, then column headers are automatically
+ generated.
+ data_line (int):
+ The line on which the data starts
+ header_delim (strong):
+ The delimiter used for separating header values
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+ """
+ new_data.filename = filename
+ try:
+ with FileManager(filename, errors="ignore", encoding="utf-8") as f:
+ for i, line in enumerate(f):
+ if i == 0:
+ first = line.strip()
+ new_data["Timestamp"] = first
+ check = datetime.strptime(first, "%a %b %d %H:%M:%S %Y")
+ if check is None:
+ raise StonerLoadError("Not a VSM file ?")
+ elif i == 1:
+ assertion(line.strip() == "")
+ elif i == 2:
+ header_string = line.strip()
+ elif i == header_line:
+ unit_string = line.strip()
+ column_headers = [
+ f"{h.strip()} ({u.strip()})"
+ for h, u in zip(header_string.split(header_delim), unit_string.split(header_delim))
+ ]
+ elif i > 3:
+ break
+ except (StonerAssertionError, ValueError, AssertionError, TypeError) as err:
+ raise StonerLoadError(f"Not a VSM File {err}") from err
+ new_data.data = np.genfromtxt(
+ new_data.filename,
+ dtype="float",
+ usemask=True,
+ skip_header=data_line - 1,
+ missing_values=["6:0", "---"],
+ invalid_raise=False,
+ )
+
+ new_data.data = np.ma.mask_rows(new_data.data)
+ cols = new_data.data.shape[1]
+ new_data.data = np.reshape(new_data.data.compressed(), (-1, cols))
+ new_data.column_headers = column_headers
+ new_data.setas(x="H_vsm (T)", y="m (emu)") # pylint: disable=not-callable
+ return new_data
+
+
+@register_loader(
+ patterns=".dql",
+ mime_types=[("application/x-wine-extension-ini", 16), ("text/plain", 16)],
+ name="XRDFile",
+ what="Data",
+)
+def load_xrd(new_data, filename=None, *args, **kargs):
+ """Read an XRD DataFile as produced by the Brucker diffractometer.
+
+ Args:
+ filename (string or bool):
+ File to load. If None then the existing filename is used, if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+
+ Notes:
+ Format is ini file like but not enough to do standard inifile processing - in particular
+ one can have multiple sections with the same name (!)
+ """
+ if filename is None or not filename:
+ new_data.get_filename("r")
+ else:
+ new_data.filename = filename
+ sh = re.compile(r"\[(.+)\]") # Regexp to grab section name
+ with FileManager(new_data.filename, errors="ignore", encoding="utf-8") as f: # Read filename linewise
+ if f.readline().strip() != ";RAW4.00": # Check we have the correct fileformat
+ raise StonerLoadError("File Format Not Recognized !")
+ drive = 0
+ for line in f: # for each line
+ m = sh.search(line)
+ if m: # This is a new section
+ section = m.group(1)
+ if section == "Drive": # If this is a Drive section we need to know which Drive Section it is
+ section = section + str(drive)
+ drive = drive + 1
+ elif section == "Data": # Data section contains the business but has a redundant first line
+ f.readline()
+ for line in f: # Now start reading lines in this section...
+ if line.strip() == "":
+ # A blank line marks the end of the section, so go back to the outer loop which will
+ # handle a new section
+ break
+ if section == "Data": # In the Data section read lines of data value,vale
+ parts = line.split(",")
+ angle = parts[0].strip()
+ counts = parts[1].strip()
+ dataline = np.array([float(angle), float(counts)])
+ new_data.data = np.append(new_data.data, dataline)
+ else: # Other sections contain metadata
+ parts = line.split("=")
+ key = parts[0].strip()
+ data = parts[1].strip()
+ # Keynames in main metadata are section:key - use theDataFile magic to do type
+ # determination
+ new_data[section + ":" + key] = string_to_type(data)
+ column_headers = ["Angle", "Counts"] # Assume the columns were Angles and Counts
+
+ new_data.data = np.reshape(new_data.data, (-1, 2))
+ new_data.setas = "xy"
+ new_data._public_attrs = {"four_bounce": bool}
+ new_data.four_bounce = new_data["HardwareConfiguration:Monochromator"] == 1
+ new_data.column_headers = column_headers
+ if kargs.pop("Q", False):
+ _to_Q(new_data)
+ return new_data
diff --git a/Stoner/formats/data/maximus.py b/Stoner/formats/data/maximus.py
new file mode 100755
index 000000000..234ce14ff
--- /dev/null
+++ b/Stoner/formats/data/maximus.py
@@ -0,0 +1,83 @@
+# -*- coding: utf-8 -*-
+"""Loader for maximus spectra files."""
+from pathlib import Path
+
+import numpy as np
+
+from ..decorators import register_loader
+from ...core.exceptions import StonerLoadError
+from ..utils.maximus import read_scan, flatten_header, hdr_to_dict
+
+
+@register_loader(
+ patterns=[(".hdr", 16), (".xsp", 16)], mime_types=("text/plain", 16), name="MaximusSpectra", what="Data"
+)
+def load_maximus_spectra(new_data, *args, **kargs):
+ """Maximus xsp file loader routine.
+
+ Args:
+ filename (string or bool): File to load. If None then the existing filename is used,
+ if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+ """
+ filename = kargs.get("filename", args[0])
+ if filename is None or not filename:
+ new_data.get_filename("r")
+ else:
+ new_data.filename = filename
+ # Open the file and read the main file header and unpack into a dict
+ try:
+ pth = Path(new_data.filename)
+ except (TypeError, ValueError) as err:
+ raise StonerLoadError("Can only open things that can be converted to paths!") from err
+ if pth.suffix != ".hdr": # Passed a .xim or .xsp file in instead of the hdr file.
+ pth = Path("_".join(str(pth).split("_")[:-1]) + ".hdr")
+ stem = pth.parent / pth.stem
+
+ try:
+ hdr = flatten_header(hdr_to_dict(pth))
+ if "Point Scan" not in hdr["ScanDefinition.Type"]:
+ raise StonerLoadError("Not an Maximus Single Image File")
+ except (StonerLoadError, ValueError, TypeError, IOError) as err:
+ raise StonerLoadError("Error loading as Maximus File") from err
+ header, data, dims = read_scan(stem)
+ new_data.metadata.update(flatten_header(header))
+ new_data.data = np.column_stack((dims[0], data))
+ headers = [new_data.metadata["ScanDefinition.Regions.PAxis.Name"]]
+ if len(dims) == 2:
+ headers.extend([str(x) for x in dims[1]])
+ else:
+ headers.append(new_data.metadata["ScanDefinition.Channels.Name"])
+ new_data.column_headers = headers
+ new_data.setas = "xy"
+ return new_data
+
+
+@register_loader(
+ patterns=[(".hdr", 16), (".xim", 16)], mime_types=("text/plain", 16), name="MaximusImage", what="Data"
+)
+def load_maximus_data(new_data, filename, *args, **kargs):
+ """Load a maximus image, but to a Data object."""
+ try:
+ new_data.filename = filename
+ pth = Path(new_data.filename)
+ except TypeError as err:
+ raise StonerLoadError(f"UUnable to interpret {filename} as a path like object") from err
+ if pth.suffix != ".hdr": # Passed a .xim or .xsp file in instead of the hdr file.
+ pth = Path("_".join(str(pth).split("_")[:-1]) + ".hdr")
+ stem = pth.parent / pth.stem
+
+ try:
+ hdr = flatten_header(hdr_to_dict(pth))
+ if "Image Scan" not in hdr["ScanDefinition.Type"]:
+ raise StonerLoadError("Not an Maximus Single Image File")
+ except (StonerLoadError, ValueError, TypeError, IOError) as err:
+ raise StonerLoadError("Error loading as Maximus File") from err
+ data = read_scan(stem)[1]
+ new_data.metadata.update(hdr)
+ if data.ndim == 3:
+ data = data[:, :, 0]
+ new_data.data = data
+ return new_data
diff --git a/Stoner/formats/data/rigs.py b/Stoner/formats/data/rigs.py
new file mode 100755
index 000000000..04360d9dd
--- /dev/null
+++ b/Stoner/formats/data/rigs.py
@@ -0,0 +1,279 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+"""Implement DataFile like classes for Various experimental rigs."""
+
+import re
+import csv
+
+import numpy as np
+
+from ...compat import bytes2str
+from ...core.base import string_to_type
+from ...core.exceptions import StonerLoadError
+from ..data.generic import load_csvfile
+from ...tools.file import FileManager
+
+from ..decorators import register_loader
+
+
+@register_loader(
+ patterns=[(".dat", 64), (".iv", 64), (".rvt", 64)], mime_types=("text/plain", 64), name="BigBlueFile", what="Data"
+)
+def load_bigblue(new_data, filename, *args, **kargs):
+ """Just call the parent class but with the right parameters set.
+
+ Args:
+ filename (string or bool): File to load. If None then the existing filename is used,
+ if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+ """
+ new_data.filename = filename
+
+ new_data = load_csvfile(new_data, filename, *args, header_line=3, data_line=7, data_delim=" ", header_delim=",")
+ if np.all(np.isnan(new_data.data)):
+ raise StonerLoadError("All data was NaN in Big Blue format")
+ return new_data
+
+
+@register_loader(patterns=(".dat", 32), mime_types=("text/plain", 32), name="BirgeIVFile", what="Data")
+def load_birge(new_data, filename, *args, **kargs):
+ """File loader for PinkLib.
+
+ Args:
+ filename (string or bool): File to load. If None then the existing filename is used,
+ if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+ """
+ new_data.filename = filename
+ ix = 0
+ with FileManager(new_data.filename, "r", errors="ignore", encoding="utf-8") as f: # Read filename linewise
+ if not re.compile(r"\d{1,2}/\d{1,2}/\d{4}").match(f.readline()):
+ raise StonerLoadError("Not a BirgeIVFile as no date on first line")
+ data = f.readlines()
+ expected = ["Vo(-))", "Vo(+))", "Ic(+)", "Ic(-)"]
+ for length, m in zip(data[-4:], expected):
+ if not length.startswith(m):
+ raise StonerLoadError("Not a BirgeIVFile as wrong footer line")
+ key = length[: len(m)]
+ val = length[len(m) :]
+ if "STDEV" in val:
+ ix2 = val.index("STDEV")
+ key2 = val[ix2 : ix2 + 4 + len(key)]
+ val2 = val[ix2 + 4 + len(key) :]
+ new_data.metadata[key2] = string_to_type(val2.strip())
+ val = val[:ix2]
+ new_data.metadata[key] = string_to_type(val.strip())
+ for ix, line in enumerate(data): # Scan the ough lines to get metadata
+ if ":" in line:
+ parts = line.split(":")
+ new_data.metadata[parts[0].strip()] = string_to_type(parts[1].strip())
+ elif "," in line:
+ for part in line.split(","):
+ parts = part.split(" ")
+ new_data.metadata[parts[0].strip()] = string_to_type(parts[1].strip())
+ elif line.startswith("H "):
+ new_data.metadata["H"] = string_to_type(line.split(" ")[1].strip())
+ else:
+ headers = [x.strip() for x in line.split(" ")]
+ break
+ else:
+ raise StonerLoadError("Oops ran off the end of the file!")
+ new_data.data = np.genfromtxt(filename, skip_header=ix + 2, skip_footer=4)
+ new_data.column_headers = headers
+
+ new_data.setas = "xy"
+ return new_data
+
+
+@register_loader(patterns=[(".dat", 16), (".txt", 16)], mime_types=("text/plain", 16), name="MokeFile", what="Data")
+def load_old_moke(new_data, filename, *args, **kargs):
+ """Leeds MOKE file loader routine.
+
+ Args:
+ filename (string or bool): File to load. If None then the existing filename is used,
+ if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+ """
+ new_data.filename = filename
+ with FileManager(new_data.filename, mode="rb") as f:
+ line = bytes2str(f.readline()).strip()
+ if line != "#Leeds CM Physics MOKE":
+ raise StonerLoadError("Not a Core.DataFile from the Leeds MOKE")
+ while line.startswith("#") or line == "":
+ parts = line.split(":")
+ if len(parts) > 1:
+ key = parts[0][1:]
+ data = ":".join(parts[1:]).strip()
+ new_data[key] = data
+ line = bytes2str(f.readline()).strip()
+ column_headers = [x.strip() for x in line.split(",")]
+ new_data.data = np.genfromtxt(f, delimiter=",")
+ new_data.setas = "xy.de"
+ new_data.column_headers = column_headers
+ return new_data
+
+
+@register_loader(patterns=(".dat", 16), mime_types=("text/plain", 16), name="FmokeFile", what="Data")
+def load_fmoke(new_data, filename, *args, **kargs):
+ """Sheffield Focussed MOKE file loader routine.
+
+ Args:
+ filename (string or bool): File to load. If None then the existing filename is used,
+ if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+ """
+ if filename is None or not filename:
+ new_data.get_filename("r")
+ else:
+ new_data.filename = filename
+ with FileManager(new_data.filename, mode="rb") as f:
+ try:
+ value = [float(x.strip()) for x in bytes2str(f.readline()).split("\t")]
+ except (TypeError, ValueError) as err:
+ f.close()
+ raise StonerLoadError("Not an FMOKE file?") from err
+ label = [x.strip() for x in bytes2str(f.readline()).split("\t")]
+ if label[0] != "Header:":
+ f.close()
+ raise StonerLoadError("Not a Focussed MOKE file !")
+ del label[0]
+ for k, v in zip(label, value):
+ new_data.metadata[k] = v # Create metadata from first 2 lines
+ column_headers = [x.strip() for x in bytes2str(f.readline()).split("\t")]
+ new_data.data = np.genfromtxt(f, dtype="float", delimiter="\t", invalid_raise=False)
+ new_data.column_headers = column_headers
+ return new_data
+
+
+def _extend_columns(new_data, i):
+ """Ensure the column headers are at least i long."""
+ if len(new_data.column_headers) < i:
+ length = len(new_data.column_headers)
+ new_data.data = np.append(
+ new_data.data, np.zeros((new_data.shape[0], i - length)), axis=1
+ ) # Need to expand the array first
+ new_data.column_headers.extend([f"Column {x}" for x in range(length, i)])
+
+
+def _et_cmd(new_data, parts):
+ """Handle axis labellling command."""
+ if parts[0] == "x":
+ _extend_columns(new_data, 1)
+ new_data.column_headers[0] = parts[1]
+ elif parts[0] == "y":
+ _extend_columns(new_data, 2)
+ new_data.column_headers[1] = parts[1]
+ elif parts[0] == "g":
+ new_data["title"] = parts[1]
+
+
+def _td_cmd(new_data, parts):
+ new_data.setas = parts[0]
+
+
+def _sa_cmd(new_data, parts):
+ """Implement the sa (set-axis?) command."""
+ if parts[0] == "l": # Legend
+ col = int(parts[2])
+ _extend_columns(new_data, col + 1)
+ new_data.column_headers[col] = parts[1]
+
+
+@register_loader(patterns=("*", 64), mime_types=("text/plain", 64), name="EasyPlotFile", what="Data")
+def load_easyplot(new_data, filename, *args, **kargs):
+ """Private loader method."""
+ new_data.filename = filename
+
+ datastart = -1
+ dataend = -1
+
+ i = 0
+ with FileManager(new_data.filename, "r", errors="ignore", encoding="utf-8") as data:
+ if "******** EasyPlot save file ********" not in data.read(1024):
+ raise StonerLoadError("Not an EasyPlot Save file?")
+ data.seek(0)
+ for i, line in enumerate(data):
+ line = line.strip()
+ if line == "":
+ continue
+ if line[0] not in "-0123456789" and dataend < 0 <= datastart:
+ dataend = i
+ if line.startswith('"') and ":" in line:
+ parts = [x.strip() for x in line.strip('"').split(":")]
+ new_data[parts[0]] = string_to_type(":".join(parts[1:]))
+ elif line.startswith("/"): # command
+ parts = [x.strip('"') for x in next(csv.reader([line], delimiter=" ")) if x != ""]
+ cmd = parts[0].strip("/")
+ if len(cmd) > 1:
+ cmdname = f"_{cmd}_cmd"
+ if cmdname in globals():
+ cmd = globals()[cmdname]
+ cmd(new_data, parts[1:])
+ else:
+ if len(parts[1:]) > 1:
+ cmd = cmd + "." + parts[1]
+ value = ",".join(parts[2:])
+ elif len(parts[1:]) == 1:
+ value = parts[1]
+ else:
+ value = True
+ new_data[cmd] = value
+ elif line[0] in "-0123456789" and datastart < 0: # start of data
+ datastart = i
+ if "," in line:
+ delimiter = ","
+ else:
+ delimiter = None
+ if dataend < 0:
+ dataend = i
+ new_data.data = np.genfromtxt(
+ new_data.filename, skip_header=datastart, skip_footer=i - dataend, delimiter=delimiter
+ )
+ if new_data.data.shape[1] == 2:
+ new_data.setas = "xy"
+ return new_data
+
+
+@register_loader(patterns=(".dat", 64), mime_types=("text/plain", 64), name="PinkLibFile", what="Data")
+def load_pinklib(new_data, filename=None, *args, **kargs):
+ """File loader for PinkLib.
+
+ Args:
+ filename (string or bool): File to load. If None then the existing filename is used,
+ if False, then a file dialog will be used.
+
+ Returns:
+ A copy of the itnew_data after loading the data.
+ """
+ if filename is None or not filename:
+ new_data.get_filename("r")
+ else:
+ new_data.filename = filename
+ with FileManager(new_data.filename, "r", errors="ignore", encoding="utf-8") as f: # Read filename linewise
+ if "PINKlibrary" not in f.readline():
+ raise StonerLoadError("Not a PINK file")
+ f = f.readlines()
+ happened_before = False
+ for i, line in enumerate(f):
+ if line[0] != "#" and not happened_before:
+ header_line = i - 2 # -2 because there's a commented out data line
+ happened_before = True
+ continue # want to get the metadata at the bottom of the file too
+ if any(s in line for s in ("Start time", "End time", "Title")):
+ tmp = line.strip("#").split(":")
+ new_data.metadata[tmp[0].strip()] = ":".join(tmp[1:]).strip()
+ column_headers = f[header_line].strip("#\t ").split("\t")
+ data = np.genfromtxt(f, dtype="float", delimiter="\t", invalid_raise=False, comments="#")
+ new_data.data = data[:, 0:-2] # Deal with an errant tab at the end of each line
+ new_data.column_headers = column_headers
+ if np.all([h in column_headers for h in ("T (C)", "R (Ohm)")]):
+ new_data.setas(x="T (C)", y="R (Ohm)") # pylint: disable=not-callable
+ return new_data
diff --git a/Stoner/formats/data/simulations.py b/Stoner/formats/data/simulations.py
new file mode 100755
index 000000000..bc1311761
--- /dev/null
+++ b/Stoner/formats/data/simulations.py
@@ -0,0 +1,158 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+"""Implements classes to load file formats from various simulation packages."""
+
+import re
+
+import numpy as np
+
+from ...core.exceptions import StonerLoadError
+from ...tools.file import FileManager
+
+from ..decorators import register_loader
+
+
+def _read_line(data, metadata):
+ """Read a single line and add to a metadata dictionary."""
+ line = data.readline().decode("utf-8", errors="ignore").strip("#\n \t\r")
+ if line == "" or ":" not in line:
+ return True
+ parts = line.split(":")
+ field = parts[0].strip()
+ value = ":".join(parts[1:]).strip()
+ if field == "Begin" and value.startswith("Data "):
+ value = value.split(" ")
+ metadata["representation"] = value[1]
+ if value[1] == "Binary":
+ metadata["representation size"] = value[2]
+ return False
+ if field not in ["Begin", "End"]:
+ metadata[field] = value
+ return True
+
+
+@register_loader(patterns=(".dat", 16), mime_types=("text/plain", 16), name="GenXFile", what="Data")
+def load_genx(new_data, filename=None, *args, **kargs):
+ """Load function. File format has space delimited columns from row 3 onwards."""
+ new_data.filename = filename
+ pattern = re.compile(r'# Dataset "([^\"]*)" exported from GenX on (.*)$')
+ pattern2 = re.compile(r"#\sFile\sexported\sfrom\sGenX\'s\sReflectivity\splugin")
+ i = 0
+ ix = 0
+ with FileManager(new_data.filename, "r", errors="ignore", encoding="utf-8") as datafile:
+ line = datafile.readline()
+ match = pattern.match(line)
+ match2 = pattern2.match(line)
+ if match is not None:
+ dataset = match.groups()[0]
+ date = match.groups()[1]
+ new_data["date"] = date
+ i = 2
+ elif match2 is not None:
+ line = datafile.readline()
+ new_data["date"] = line.split(":")[1].strip()
+ dataset = datafile.readline()[1:].strip()
+ i = 3
+ else:
+ raise StonerLoadError("Not a GenXFile")
+ for ix, line in enumerate(datafile):
+ line = line.strip()
+ if line in ["# Headers:", "# Column labels:"]:
+ line = next(datafile)[1:].strip()
+ break
+ else:
+ raise StonerLoadError("Cannot find headers")
+ skip = ix + i + 2
+ column_headers = [f.strip() for f in line.strip().split("\t")]
+ new_data.data = np.real(np.genfromtxt(new_data.filename, skip_header=skip, dtype=complex))
+ new_data["dataset"] = dataset
+ if "sld" in dataset.lower():
+ new_data["type"] = "SLD"
+ elif "asymmetry" in dataset.lower():
+ new_data["type"] = "Asymmetry"
+ elif "dd" in dataset.lower():
+ new_data["type"] = "Down"
+ elif "uu" in dataset.lower():
+ new_data["type"] = "Up"
+ new_data.column_headers = column_headers
+ return new_data
+
+
+@register_loader(
+ patterns=(".ovf", 16),
+ mime_types=[("text/plain", 16), ("application/octet-stream", 16)],
+ name="OVFFile",
+ what="Data",
+)
+def _load(new_data, filename=None, *args, **kargs):
+ """Load function. File format has space delimited columns from row 3 onwards.
+
+ Notes:
+ This code can handle only the first segment in the data file.
+ """
+ if filename is None or not filename:
+ new_data.get_filename("r")
+ else:
+ new_data.filename = filename
+ # Reading in binary, converting to utf-8 where we should be in the text header
+ with FileManager(new_data.filename, "rb") as data:
+ line = data.readline().decode("utf-8", errors="ignore").strip("#\n \t\r")
+ if line not in ["OOMMF: rectangular mesh v1.0", "OOMMF OVF 2.0"]:
+ raise StonerLoadError("Not an OVF 1.0 or 2.0 file.")
+ while _read_line(data, new_data.metadata):
+ pass # Read the file until we reach the start of the data block.
+ if new_data.metadata["representation"] == "Binary":
+ size = ( # Work out the size of the data block to read
+ new_data.metadata["xnodes"]
+ * new_data.metadata["ynodes"]
+ * new_data.metadata["znodes"]
+ * new_data.metadata["valuedim"]
+ + 1
+ ) * new_data.metadata["representation size"]
+ bin_data = data.read(size)
+ numbers = np.frombuffer(bin_data, dtype=f"f{new_data.metadata['representation size']}")
+ chk = numbers[0]
+ if chk != [1234567.0, 123456789012345.0][new_data.metadata["representation size"] // 4 - 1]:
+ raise StonerLoadError("Bad binary data for ovf gile.")
+
+ data = np.reshape(
+ numbers[1:],
+ (new_data.metadata["xnodes"] * new_data.metadata["ynodes"] * new_data.metadata["znodes"], 3),
+ )
+ else:
+ data = np.genfromtxt(
+ data,
+ max_rows=new_data.metadata["xnodes"] * new_data.metadata["ynodes"] * new_data.metadata["znodes"],
+ )
+ xmin, xmax, xstep = (
+ new_data.metadata["xmin"],
+ new_data.metadata["xmax"],
+ new_data.metadata["xstepsize"],
+ )
+ ymin, ymax, ystep = (
+ new_data.metadata["ymin"],
+ new_data.metadata["ymax"],
+ new_data.metadata["ystepsize"],
+ )
+ zmin, zmax, zstep = (
+ new_data.metadata["zmin"],
+ new_data.metadata["zmax"],
+ new_data.metadata["zstepsize"],
+ )
+ Z, Y, X = np.meshgrid(
+ np.arange(zmin + zstep / 2, zmax, zstep) * 1e9,
+ np.arange(ymin + ystep / 2, ymax, ystep) * 1e9,
+ np.arange(xmin + xstep / 2, xmax, xstep) * 1e9,
+ indexing="ij",
+ )
+ new_data.data = np.column_stack((X.ravel(), Y.ravel(), Z.ravel(), data))
+
+ column_headers = ["X (nm)", "Y (nm)", "Z (nm)", "U", "V", "W"]
+ new_data.setas = "xyzuvw"
+ new_data.column_headers = column_headers
+ return new_data
diff --git a/Stoner/formats/data/zip.py b/Stoner/formats/data/zip.py
new file mode 100755
index 000000000..4184937a0
--- /dev/null
+++ b/Stoner/formats/data/zip.py
@@ -0,0 +1,120 @@
+# -*- coding: utf-8 -*-
+"""Loader for zip files."""
+import zipfile as zf
+import os.path as path
+from traceback import format_exc
+
+from ...compat import str2bytes, path_types
+from ...core.exceptions import StonerLoadError
+from ...tools import copy_into, make_Data
+from ..decorators import register_loader, register_saver
+from ..utils.zip import test_is_zip
+
+
+@register_loader(patterns=(".zip", 16), mime_types=("application/zip", 16), name="ZippedFile", what="Data")
+def load_zipfile(new_data, filename=None, *args, **kargs):
+ """Load a file from the zip file, opening it as necessary."""
+ new_data.filename = filename
+ try:
+ if isinstance(new_data.filename, zf.ZipFile): # Loading from an ZipFile
+ if not new_data.filename.fp: # Open zipfile if necessary
+ other = zf.ZipFile(new_data.filename.filename, "r")
+ close_me = True
+ else: # Zip file is already open
+ other = new_data.filename
+ close_me = False
+ member = kargs.get("member", other.namelist()[0])
+ solo_file = len(other.namelist()) == 1
+ elif isinstance(new_data.filename, path_types) and zf.is_zipfile(
+ new_data.filename
+ ): # filename is a string that is a zip file
+ other = zf.ZipFile(new_data.filename, "a")
+ member = kargs.get("member", other.namelist()[0])
+ close_me = True
+ solo_file = len(other.namelist()) == 1
+ else:
+ raise StonerLoadError(f"{new_data.filename} does not appear to be a real zip file")
+ except StonerLoadError:
+ raise
+ except Exception as err: # pylint: disable=W0703 # Catching everything else here
+ try:
+ exc = format_exc()
+ other.close()
+ except (AttributeError, NameError, ValueError, TypeError, zf.BadZipFile, zf.LargeZipFile):
+ pass
+ raise StonerLoadError(f"{new_data.filename} threw an error when opening\n{exc}") from err
+ # Ok we can try reading now
+ info = other.getinfo(member)
+ data = other.read(info) # In Python 3 this would be a bytes
+ tmp = make_Data() << data.decode("utf-8")
+ copy_into(tmp, new_data)
+ # new_data.__init__(tmp << data)
+ new_data.filename = path.join(other.filename, member)
+ if close_me:
+ other.close()
+ if solo_file:
+ new_data.filename = str(filename)
+ return new_data
+
+
+@register_saver(patterns=(".zip", 16), name="ZippedFile", what="Data")
+def save(save_data, filename=None, **kargs):
+ """Override the save method to allow ZippedFile to be written out to disc (as a mininmalist output).
+
+ Args:
+ filename (string or zipfile.ZipFile instance):
+ Filename to save as (using the same rules as for the load routines)
+
+ Returns:
+ A copy of itsave_data.
+ """
+ if filename is None:
+ filename = save_data.filename
+ if filename is None or (isinstance(filename, bool) and not filename): # now go and ask for one
+ filename = save_data.__file_dialog("w")
+ compression = kargs.pop("compression", zf.ZIP_DEFLATED)
+ try:
+ if isinstance(filename, path_types): # We;ve got a string filename
+ if test_is_zip(filename): # We can find an existing zip file somewhere in the filename
+ zipfile, member = test_is_zip(filename)
+ zipfile = zf.ZipFile(zipfile, "a")
+ close_me = True
+ elif path.exists(filename): # The fiule exists but isn't a zip file
+ raise IOError(f"{filename} Should either be a zip file or a new zip file")
+ else: # Path doesn't exist, use extension of file part to find where the zip file should be
+ parts = path.split(filename)
+ for i, part in enumerate(parts):
+ if path.splitext(part)[1].lower() == ".zip":
+ break
+ else:
+ raise IOError(f"Can't figure out where the zip file is in {filename}")
+ zipfile = zf.ZipFile(path.join(*parts[: i + 1]), "w", compression, True)
+ close_me = True
+ member = path.join("/", *parts[i + 1 :])
+ elif isinstance(filename, zf.ZipFile): # Handle\ zipfile instance, opening if necessary
+ if not filename.fp:
+ filename = zf.ZipFile(filename.filename, "a")
+ close_me = True
+ else:
+ close_me = False
+ zipfile = filename
+ member = ""
+
+ if member == "" or member == "/": # Is our file object a bare zip file - if so create a default member name
+ if len(zipfile.namelist()) > 0:
+ member = zipfile.namelist()[-1]
+ save_data.filename = path.join(filename, member)
+ else:
+ member = "DataFile.txt"
+ save_data.filename = filename
+
+ zipfile.writestr(member, str2bytes(str(save_data)))
+ if close_me:
+ zipfile.close()
+ except (zipfile.BadZipFile, IOError, TypeError, ValueError) as err:
+ error = format_exc()
+ try:
+ zipfile.close()
+ finally:
+ raise IOError(f"Error saving zipfile\n{error}") from err
+ return save_data
diff --git a/Stoner/formats/facilities.py b/Stoner/formats/facilities.py
index 363a3c324..b4f0ec82b 100755
--- a/Stoner/formats/facilities.py
+++ b/Stoner/formats/facilities.py
@@ -22,7 +22,6 @@
class BNLFile(Core.DataFile):
-
"""Reader of files in the SPEC format given by BNL (specifically u4b beamline but hopefully generalisable).
Author RCT 12/2011
@@ -137,7 +136,6 @@ def _load(self, filename, *args, **kargs): # fileType omitted, implicit in clas
class MDAASCIIFile(Core.DataFile):
-
"""Reads files generated from the APS."""
priority = 16
@@ -223,7 +221,6 @@ def _load(self, filename=None, *args, **kargs):
class OpenGDAFile(Core.DataFile):
-
"""Extends Core.DataFile to load files from RASOR."""
priority = 16 # Makes a positive ID of it's file type so give priority
@@ -264,12 +261,10 @@ def _load(self, filename=None, *args, **kargs):
class RasorFile(OpenGDAFile):
-
"""Just an alias for OpenGDAFile."""
class SNSFile(Core.DataFile):
-
"""Reads the ASCII exported PNR reduced files from BL-4A line at the SSNS at Oak Ridge National Lab.
File has a large header marked up with # prefixes which include several section is []
@@ -343,7 +338,6 @@ def _load(self, filename=None, *args, **kargs):
if fabio:
class ESRF_DataFile(Core.DataFile):
-
"""Utilise the fabIO library to read an edf file has a DataFile."""
priority = 16
@@ -365,7 +359,6 @@ def _load(self, filename=None, *args, **kargs):
raise StonerLoadError("Not an ESRF data file !") from err
class FabioImageFile(Image.ImageFile):
-
"""Utilise the fabIO library to read an edf file has a DataFile."""
priority = 32
diff --git a/Stoner/formats/generic.py b/Stoner/formats/generic.py
index 572f4588f..53264f484 100755
--- a/Stoner/formats/generic.py
+++ b/Stoner/formats/generic.py
@@ -8,6 +8,23 @@
import re
from collections.abc import Mapping
import sys
+import logging
+
+import PIL
+import numpy as np
+
+from ..Core import DataFile
+from ..compat import str2bytes, Hyperspy_ok, hs, hsload
+from ..core.exceptions import StonerLoadError
+from ..tools.file import FileManager
+
+
+class _refuse_log(logging.Filter):
+ """Refuse to log all records."""
+
+ def filter(self, record):
+ """Do not log anything."""
+ return False
@contextlib.contextmanager
@@ -16,20 +33,14 @@ def catch_sysout(*args):
stdout, stderr = sys.stdout, sys.stderr
out = io.StringIO()
sys.stdout, sys.stderr = out, out
+ logger = logging.getLogger("hyperspy.io")
+ logger.addFilter(_refuse_log)
yield None
+ logger.removeFilter(_refuse_log)
sys.stdout, sys.stderr = stdout, stderr
return
-import PIL
-import numpy as np
-
-from ..Core import DataFile
-from ..compat import str2bytes, Hyperspy_ok, hs, hsload
-from ..core.exceptions import StonerLoadError
-from ..tools.file import FileManager
-
-
def _delim_detect(line):
"""Detect a delimiter in a line.
@@ -61,7 +72,6 @@ def _delim_detect(line):
class CSVFile(DataFile):
-
"""A subclass of DataFiule for loading generic deliminated text fiules without metadata."""
#: priority (int): is the load order for the class, smaller numbers are tried before larger numbers.
@@ -166,7 +176,6 @@ def save(self, filename=None, **kargs):
class JustNumbersFile(CSVFile):
-
"""A reader format for things which are just a block of numbers with no headers or metadata."""
priority = 256 # Rather generic file format so make it a low priority
@@ -178,7 +187,6 @@ class JustNumbersFile(CSVFile):
class KermitPNGFile(DataFile):
-
"""Loads PNG files with additional metadata embedded in them and extracts as metadata."""
#: priority (int): is the load order for the class, smaller numbers are tried before larger numbers.
@@ -266,7 +274,6 @@ def save(self, filename=None, **kargs):
from nptdms import TdmsFile
class TDMSFile(DataFile):
-
"""First stab at writing a file that will import TDMS files."""
#: priority (int): is the load order for the class, smaller numbers are tried before larger numbers.
@@ -322,7 +329,6 @@ def _load(self, filename=None, *args, **kargs):
if Hyperspy_ok:
class HyperSpyFile(DataFile):
-
"""Wrap the HyperSpy file to map to DataFile."""
priority = 64 # Makes an ID check but is quite generic
diff --git a/Stoner/formats/image/__init__.py b/Stoner/formats/image/__init__.py
new file mode 100755
index 000000000..d43bc84df
--- /dev/null
+++ b/Stoner/formats/image/__init__.py
@@ -0,0 +1,8 @@
+"""Sub-package of routines to load ImageFile objects in different formats.
+
+This sub-package is lazy-loaded but then pulls in all of the modules in order to register the routines.
+"""
+
+__all__ = ["generic", "hdf5", "facilities", "maximus"]
+
+from . import generic, hdf5, facilities, maximus
diff --git a/Stoner/formats/image/facilities.py b/Stoner/formats/image/facilities.py
new file mode 100755
index 000000000..41fff219d
--- /dev/null
+++ b/Stoner/formats/image/facilities.py
@@ -0,0 +1,35 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+"""Implements DataFile like classes for various large scale facilities."""
+
+# Standard Library imports
+from ...core.exceptions import StonerLoadError
+
+from ..decorators import register_loader
+
+try:
+ import fabio
+except ImportError:
+ fabio = None
+
+if fabio:
+
+ @register_loader(
+ patterns=(".edf", 32),
+ mime_types=[("text/plain", 32), ("application/octet-stream", 32)],
+ name="FabioImage",
+ what="Image",
+ )
+ def load_fabio(new_data, filename=None, *args, **kargs):
+ """Load function. File format has space delimited columns from row 3 onwards."""
+ if filename is None or not filename:
+ new_data.get_filename("r")
+ else:
+ new_data.filename = filename
+ try:
+ img = fabio.open(new_data.filename)
+ new_data.image = img.data
+ new_data.metadata.update(img.header)
+ return new_data
+ except (OSError, ValueError, TypeError, IndexError) as err:
+ raise StonerLoadError("Not a Fabio Image file !") from err
diff --git a/Stoner/formats/image/generic.py b/Stoner/formats/image/generic.py
new file mode 100755
index 000000000..da1c13994
--- /dev/null
+++ b/Stoner/formats/image/generic.py
@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+"""Implement DataFile classes for some generic file formats."""
+import contextlib
+import io
+import re
+import sys
+import logging
+
+from ...core.exceptions import StonerLoadError
+from ...Image import ImageArray
+from ...formats.decorators import register_loader
+
+
+class _refuse_log(logging.Filter):
+ """Refuse to log all records."""
+
+ def filter(self, record):
+ """Do not log anything."""
+ return False
+
+
+@contextlib.contextmanager
+def catch_sysout(*args):
+ """Temporarily redirect sys.stdout and.sys.stdin."""
+ stdout, stderr = sys.stdout, sys.stderr
+ out = io.StringIO()
+ sys.stdout, sys.stderr = out, out
+ logger = logging.getLogger("hyperspy.io")
+ logger.addFilter(_refuse_log)
+ yield None
+ logger.removeFilter(_refuse_log)
+ sys.stdout, sys.stderr = stdout, stderr
+ return
+
+
+def _delim_detect(line):
+ """Detect a delimiter in a line.
+
+ Args:
+ line(str):
+ String to search for delimiters in.
+
+ Returns:
+ (str):
+ Delimiter to use.
+
+ Raises:
+ StnerLoadError:
+ If delimiter cannot be located.
+ """
+ quotes = re.compile(r"([\"\'])[^\1]*\1")
+ line = quotes.sub("", line) # Remove quoted strings first
+ current = (None, len(line))
+ for delim in "\t ,;":
+ try:
+ idx = line.index(delim)
+ except ValueError:
+ continue
+ if idx < current[1]:
+ current = (delim, idx)
+ if current[0] is None:
+ raise StonerLoadError("Unable to find a delimiter in the line")
+ return current[0]
+
+
+@register_loader(
+ patterns=[(".tif", 8), (".tiff", 8), (".png", 8), (".npy", 8)],
+ mime_types=[("image/tiff", 8), ("image/png", 8), ("application/octet-stream", 8)],
+ name="ImageFile",
+ what="Image",
+)
+def load_imagefile(new_image, filename, *args, **kargs):
+ """Load an ImageFile by calling the ImageArray method instead."""
+ new_image._image = ImageArray(filename, *args, **kargs)
+ for k in new_image._image._public_attrs:
+ setattr(new_image, k, getattr(new_image._image, k, None))
+ return new_image
diff --git a/Stoner/formats/image/hdf5.py b/Stoner/formats/image/hdf5.py
new file mode 100755
index 000000000..ed0dc05d5
--- /dev/null
+++ b/Stoner/formats/image/hdf5.py
@@ -0,0 +1,42 @@
+# -*- coding: utf-8 -*-
+"""Image Loading routines for HDF5 format."""
+from copy import deepcopy
+
+import numpy as np
+
+from ..decorators import register_loader
+from ...tools import make_Data
+
+
+@register_loader(
+ patterns=[(".hdf5", 16), (".hdf", 16)],
+ mime_types=[("application/x-hdf", 16), ("application/x-hdf5", 16)],
+ name="STXMImage",
+ what="Image",
+)
+def load_stxm_image(new_data, filename, *args, **kargs):
+ """Initialise and load a STXM image produced by Pollux.
+
+ Keyword Args:
+ regrid (bool):
+ If set True, the gridimage() method is automatically called to re-grid the image to known coordinates.
+ """
+ regrid = kargs.pop("regrid", False)
+ kargs.setdefault("filetype", "SLS_STXMFile")
+ bcn = kargs.pop("bcn", False)
+ d = make_Data(filename, *args, **kargs)
+ new_data.image = d.data
+ new_data.metadata = deepcopy(d.metadata)
+ new_data.filename = d.filename
+ if isinstance(regrid, tuple):
+ new_data.gridimage(*regrid)
+ elif isinstance(regrid, dict):
+ new_data.gridimage(**regrid)
+ elif regrid:
+ new_data.gridimage()
+ if bcn:
+ if regrid:
+ new_data.metadata["beam current"] = new_data.metadata["beam current"].gridimage()
+ new_data.image /= new_data["beam current"]
+ new_data.polarization = np.sign(new_data.get("collection.polarization.value", 0))
+ return new_data
diff --git a/Stoner/formats/image/maximus.py b/Stoner/formats/image/maximus.py
new file mode 100755
index 000000000..123865109
--- /dev/null
+++ b/Stoner/formats/image/maximus.py
@@ -0,0 +1,33 @@
+# -*- coding: utf-8 -*-
+"""Loader for maximus scan image."""
+from pathlib import Path
+
+
+from ..decorators import register_loader
+from ...core.exceptions import StonerLoadError
+from ..utils.maximus import read_scan, flatten_header, hdr_to_dict
+
+
+@register_loader(
+ patterns=[(".hdr", 16), (".xim", 16)], mime_types=("text/plain", 16), name="MaximusImage", what="Image"
+)
+def load_maximus_image(new_data, filename, *args, **kargs):
+ """Load an ImageFile by calling the ImageArray method instead."""
+ try:
+ new_data.filename = filename
+ pth = Path(new_data.filename)
+ except TypeError as err:
+ raise StonerLoadError(f"UUnable to interpret {filename} as a path like object") from err
+ if pth.suffix != ".hdr": # Passed a .xim or .xsp file in instead of the hdr file.
+ pth = Path("_".join(str(pth).split("_")[:-1]) + ".hdr")
+ stem = pth.parent / pth.stem
+
+ try:
+ hdr = flatten_header(hdr_to_dict(pth))
+ if "Image Scan" not in hdr["ScanDefinition.Type"]:
+ raise StonerLoadError("Not an Maximus Single Image File")
+ except (StonerLoadError, ValueError, TypeError, IOError) as err:
+ raise StonerLoadError("Error loading as Maximus File") from err
+ new_data.metadata.update(hdr)
+ new_data.image = read_scan(stem)[1]
+ return new_data
diff --git a/Stoner/formats/instruments.py b/Stoner/formats/instruments.py
index 40f2575fa..4e7295233 100755
--- a/Stoner/formats/instruments.py
+++ b/Stoner/formats/instruments.py
@@ -18,7 +18,6 @@
class LSTemperatureFile(Core.DataFile):
-
"""A class that reads and writes Lakeshore Temperature Calibration Curves.
.. warning::
@@ -145,7 +144,6 @@ def save(self, filename=None, **kargs):
class QDFile(Core.DataFile):
-
"""Extends Core.DataFile to load files from Quantum Design Systems - including PPMS, MPMS and SQUID-VSM."""
#: priority (int): is the load order for the class, smaller numbers are tried before larger numbers.
@@ -239,7 +237,6 @@ def _load(self, filename=None, *args, **kargs):
class RigakuFile(Core.DataFile):
-
"""Loads a .ras file as produced by Rigaku X-ray diffractormeters."""
#: priority (int): is the load order for the class, smaller numbers are tried before larger numbers.
@@ -377,7 +374,6 @@ def to_Q(self, wavelength=1.540593):
class SPCFile(Core.DataFile):
-
"""Extends Core.DataFile to load SPC files from Raman."""
#: priority (int): is the load order for the class, smaller numbers are tried before larger numbers.
@@ -650,7 +646,6 @@ def _load(self, filename=None, *args, **kargs):
class VSMFile(Core.DataFile):
-
"""Extends Core.DataFile to open VSM Files."""
#: priority (int): is the load order for the class, smaller numbers are tried before larger numbers.
@@ -737,7 +732,6 @@ def _load(self, filename=None, *args, **kargs):
class XRDFile(Core.DataFile):
-
"""Loads Files from a Brucker D8 Discovery X-Ray Diffractometer."""
#: priority (int): is the load order for the class, smaller numbers are tried before larger numbers.
diff --git a/Stoner/formats/maximus.py b/Stoner/formats/maximus.py
index 64a8383d2..9dfb39678 100755
--- a/Stoner/formats/maximus.py
+++ b/Stoner/formats/maximus.py
@@ -37,7 +37,6 @@ def _raise_error(openfile, message=""):
class MaximusSpectra(DataFile):
-
"""Provides a :py:class:`Stoner.DataFile` subclass for loading Point spectra from Maximus."""
# We treat the hdr file as the key file type
@@ -90,7 +89,6 @@ def _load(self, *args, **kargs):
class MaximusImage(ImageFile):
-
"""Provide a STXMImage like class for the Maximus Beamline."""
_patterns = ["*.hdr", "*.xim"]
@@ -121,7 +119,6 @@ def _load(self, filename, **kargs):
class MaximusStackMixin:
-
"""Handle a stack of Maximus Images."""
_defaults = {"type": MaximusImage, "pattern": "*.hdr"}
@@ -355,7 +352,6 @@ def read_hdf5(cls, filename, *args, **kargs):
class MaximusStack(MaximusStackMixin, ImageStack):
-
"""Process an image scan stack from the Bessy Maximus beamline as an ImageStack subclass."""
@@ -374,7 +370,7 @@ def hdr_to_dict(filename, to_python=True):
(dict or str):
Either the header file as a python dictionary, or a json string.
"""
- bare = re.compile("([\s\{])([A-Za-z][A-Za-z0-9_]*)\s\:") # Match for keys
+ bare = re.compile(r"([\s\{])([A-Za-z][A-Za-z0-9_]*)\s\:") # Match for keys
term = re.compile(r",\s*([\]\}])") # match for extra , at the end of a dict or list
nan = re.compile(r"([\-0-9\.]+\#QNAN)") # Handle NaN values
diff --git a/Stoner/formats/rigs.py b/Stoner/formats/rigs.py
index 0c440c27b..624807c27 100755
--- a/Stoner/formats/rigs.py
+++ b/Stoner/formats/rigs.py
@@ -16,7 +16,6 @@
class BigBlueFile(CSVFile):
-
"""Extends CSVFile to load files from Nick Porter's old BigBlue code."""
#: priority (int): is the load order for the class, smaller numbers are tried before larger numbers.
@@ -50,7 +49,6 @@ def _load(self, filename, *args, **kargs):
class BirgeIVFile(Core.DataFile):
-
"""Implements the IV File format used by the Birge Group in Michigan State University Condesned Matter Physiscs."""
patterns = ["*.dat"]
@@ -110,7 +108,6 @@ def _load(self, filename, *args, **kargs):
class MokeFile(Core.DataFile):
-
"""Class that extgends Core.DataFile to load files from the Leeds MOKE system."""
#: priority (int): is the load order for the class, smaller numbers are tried before larger numbers.
@@ -155,7 +152,6 @@ def _load(self, filename, *args, **kargs):
class FmokeFile(Core.DataFile):
-
"""Extends Core.DataFile to open Fmoke Files."""
#: priority (int): is the load order for the class, smaller numbers are tried before larger numbers.
@@ -201,7 +197,6 @@ def _load(self, filename, *args, **kargs):
class EasyPlotFile(Core.DataFile):
-
"""A class that will extract as much as it can from an EasyPlot save File."""
#: priority (int): is the load order for the class, smaller numbers are tried before larger numbers.
@@ -296,7 +291,6 @@ def _sa_cmd(self, parts):
class PinkLibFile(Core.DataFile):
-
"""Extends Core.DataFile to load files from MdV's PINK library - as used by the GMR anneal rig."""
#: priority (int): is the load order for the class, smaller numbers are tried before larger numbers.
diff --git a/Stoner/formats/simulations.py b/Stoner/formats/simulations.py
index 9ffb7c6e0..fc229b9ab 100755
--- a/Stoner/formats/simulations.py
+++ b/Stoner/formats/simulations.py
@@ -32,7 +32,6 @@ def _read_line(data, metadata):
class GenXFile(DataFile):
-
"""Extends DataFile for GenX Exported data."""
#: priority (int): is the load order for the class, smaller numbers are tried before larger numbers.
@@ -42,7 +41,7 @@ class GenXFile(DataFile):
priority = 16
#: pattern (list of str): A list of file extensions that might contain this type of file. Used to construct
# the file load/save dialog boxes.
- patterns = ["*.dat"] # Recognised filename patterns
+ patterns = ["*.dat", "*.txt"] # Recognised filename patterns
def _load(self, filename=None, *args, **kargs):
"""Load function. File format has space delimited columns from row 3 onwards."""
@@ -50,8 +49,8 @@ def _load(self, filename=None, *args, **kargs):
self.get_filename("r")
else:
self.filename = filename
- pattern = re.compile(r'# Dataset "([^\"]*)" exported from GenX on (.*)$')
- pattern2 = re.compile(r"#\sFile\sexported\sfrom\sGenX\'s\sReflectivity\splugin")
+ pattern = re.compile(r'# Dataset "([^\"]*)" exported from GenX3? on (.*)$')
+ pattern2 = re.compile(r"#\sFile\sexported\sfrom\sGenX3?\'s\sReflectivity\splugin")
i = 0
ix = 0
with FileManager(self.filename, "r", errors="ignore", encoding="utf-8") as datafile:
@@ -72,14 +71,14 @@ def _load(self, filename=None, *args, **kargs):
raise StonerLoadError("Not a GenXFile")
for ix, line in enumerate(datafile):
line = line.strip()
- if line in ["# Headers:", "# Column labels:"]:
+ if line in ["# Headers:", "# Column lables:"]:
line = next(datafile)[1:].strip()
break
else:
raise StonerLoadError("Cannot find headers")
- skip = ix + i + 2
- column_headers = [f.strip() for f in line.strip().split("\t")]
- self.data = np.real(np.genfromtxt(self.filename, skip_header=skip, dtype=complex))
+ skip = ix + i + 2
+ column_headers = [f.strip() for f in re.split(r"[\s\t]+", line.strip())]
+ self.data = np.real(np.genfromtxt(datafile, dtype=complex))
self["dataset"] = dataset
if "sld" in dataset.lower():
self["type"] = "SLD"
@@ -90,11 +89,11 @@ def _load(self, filename=None, *args, **kargs):
elif "uu" in dataset.lower():
self["type"] = "Up"
self.column_headers = column_headers
+ self.setas = "xyye"
return self
class OVFFile(DataFile):
-
"""A class that reads OOMMF vector format files and constructs x,y,z,u,v,w data.
OVF 1 and OVF 2 files with text or binary data and only files with a meshtype rectangular are supported
diff --git a/Stoner/formats/utils/__init__.py b/Stoner/formats/utils/__init__.py
new file mode 100755
index 000000000..792d60054
--- /dev/null
+++ b/Stoner/formats/utils/__init__.py
@@ -0,0 +1 @@
+#
diff --git a/Stoner/formats/utils/maximus.py b/Stoner/formats/utils/maximus.py
new file mode 100755
index 000000000..772b95b64
--- /dev/null
+++ b/Stoner/formats/utils/maximus.py
@@ -0,0 +1,168 @@
+# -*- coding: utf-8 -*-
+"""Common routines for reading maximus files."""
+__all__ = ["read_scan", "hdr_to_dict", "flatten_header", "process_key", "read_images", "read_pointscan"]
+import json
+from pathlib import Path
+import re
+
+import numpy as np
+
+from ...tools.file import FileManager
+
+
+def read_scan(file_root):
+ """Find the .hdr and .xim/,xsp files for a scan load them into memory.
+
+ Args:
+ file_root (str): This is the part of the hdr filename beore the extension.
+
+ Returns:
+ (dict,ndarray, (n-1d arrays)):
+ Returns the metadata and an ndarray of the scan data and then n 1D arrays of the axes.
+ """
+ hdr = Path(str(file_root) + ".hdr")
+ header = hdr_to_dict(hdr, to_python=True)
+ scan_type = header["ScanDefinition"]["Type"]
+
+ if "Image Scan" in scan_type:
+ data, dims = read_images(hdr.parent.glob(f"{hdr.stem}*.xim"), header)
+ elif "Point Scan" in scan_type:
+ data, dims = read_pointscan(hdr.parent.glob(f"{hdr.stem}*.xsp"), header)
+ else:
+ raise ValueError(f"Unrecognised scan type {scan_type}")
+
+ return header, data, dims
+
+
+def hdr_to_dict(filename, to_python=True):
+ """Convert .hdr metadata file to json or python dictionary.
+
+ Args:
+ filename (str):
+ Name of file to read (can also be a pathlib.Path).
+
+ Keyword Arguments:
+ to_python (bool):
+ If true, return a python dictionary, otherwise return a json text string.
+
+ Returns:
+ (dict or str):
+ Either the header file as a python dictionary, or a json string.
+ """
+ bare = re.compile(r"([\s\{])([A-Za-z][A-Za-z0-9_]*)\s\:") # Match for keys
+ term = re.compile(r",\s*([\]\}])") # match for extra , at the end of a dict or list
+ nan = re.compile(r"([\-0-9\.]+\#QNAN)") # Handle NaN values
+
+ # Use oathlib to suck in the file
+ with FileManager(filename, "r") as f:
+ hdr = f.read()
+ # Simple string replacements first
+ stage1 = hdr.replace("=", ":").replace(";", ",").replace("(", "[").replace(")", "]")
+ # Wrap in { }
+ stage2 = f"{{{stage1}}}"
+ # Regexp replacements next
+ stage3 = bare.sub('\\1"\\2" :', stage2)
+ stage4 = term.sub("\\n\\1", stage3)
+ stage5 = nan.sub("NaN", stage4)
+
+ if to_python:
+ ret = process_key(json.loads(stage5))
+ else: # orettyify the json
+ ret = json.dumps(json.loads(stage5), indent=4)
+
+ return ret
+
+
+def flatten_header(value):
+ """Flatten nested dictionaries."""
+ if isinstance(value, list) and len(value) == 1:
+ value = value[0]
+ if not isinstance(value, dict):
+ return value
+ dels = []
+ adds = {}
+ for key, item in value.items():
+ item = flatten_header(item)
+ if isinstance(item, dict):
+ for item_key, item_val in item.items():
+ adds[f"{key}.{item_key}"] = item_val
+ dels.append(key)
+ for key in dels:
+ del value[key]
+ value.update(adds)
+ return value
+
+
+def process_key(value):
+ """Carry out post loading processing of data structures."""
+ if isinstance(value, dict):
+ for key, val in value.items():
+ value[key] = process_key(val)
+ return value
+ if isinstance(value, list):
+ if len(value) > 0 and isinstance(value[0], int) and len(value) == value[0] + 1: # Lists prepended with size
+ del value[0]
+ a = np.array(value)
+ if a.dtype.kind in ["f", "i", "u", "U"]: # convert arrays to arrays
+ return a
+ value = [process_key(v) for v in value]
+ return value
+
+
+def read_images(files, header):
+ """Read one or more .xim files and construct the data array.
+
+ Args:
+ files (glob): glob pattern of xim files to read.
+ header (dict): contents of the .jdr file.
+
+ Returns:
+ data (ndarray): 2D or 3D data.
+ dims (tuple of 1D arrays): 2 or 3 1D arrays corresponding to the dimensions of data.
+ """
+ xims = list(files)
+ scandef = header["ScanDefinition"]
+ region = scandef["Regions"][0] # FIXME assumes a single region in the data
+ if len(xims) > 1:
+ data = np.stack([np.genfromtxt(x)[::-1] for x in xims]).T
+ elif len(xims) == 1:
+ data = np.genfromtxt(xims[0])[::-1]
+ else: # no files !
+ raise IOError("No Images located")
+ xpts = region["PAxis"]["Points"]
+ ypts = region["QAxis"]["Points"]
+ if data.ndim == 3:
+ zpts = scandef["StackAxis"]["Points"]
+ dims = (xpts, ypts, zpts)
+ else:
+ dims = (xpts, ypts)
+ return data, dims
+
+
+def read_pointscan(files, header):
+ """Read one or more .xsp files and construct the data array.
+
+ Args:
+ files (glob): glob pattern of xim files to read.
+ header (dict): contents of the .jdr file.
+
+ Returns:
+ data (ndarray): 2D or 3D data.
+ dims (tuple of 1D arrays): 2 or 3 1D arrays corresponding to the dimensions of data.
+ """
+ xsps = list(files)
+ scandef = header["ScanDefinition"]
+ region = scandef["Regions"][0] # FIXME assumes a single region in the data
+ if len(xsps) > 1:
+ data = np.stack([np.genfromtxt(x)[:, 1] for x in xsps]).T
+ elif len(xsps) == 1:
+ data = np.genfromtxt(xsps[0])[:, 1]
+ else: # No files !
+ raise IOError("No Spectra located")
+ xpts = region["PAxis"]["Points"]
+ if data.ndim == 2:
+ zpts = scandef["StackAxis"]["Points"]
+ dims = (xpts, zpts)
+ else:
+ dims = (xpts,)
+ return data, dims
diff --git a/Stoner/formats/utils/zip.py b/Stoner/formats/utils/zip.py
new file mode 100755
index 000000000..06fb1ad90
--- /dev/null
+++ b/Stoner/formats/utils/zip.py
@@ -0,0 +1,37 @@
+# -*- coding: utf-8 -*-
+"""Support routine for reading zip files."""
+import zipfile as zf
+from os import path
+
+
+def test_is_zip(filename, member=""):
+ """Recursively searches for a zipfile in the tree.
+
+ Args:
+ filename (str):
+ Path to test whether it is a zip file or not.
+
+ Keyword Arguments:
+ member (str):
+ Used in recursive calls to identify the path within the zip file
+
+ Returns:
+ False or (filename,member):
+ Returns False if not a zip file, otherwise the actual filename of the zip file and the nanme of the
+ member within that
+ zipfile.
+ """
+ if not filename or str(filename) == "":
+ return False
+ if zf.is_zipfile(filename):
+ return filename, member
+ part = path.basename(filename)
+ newfile = path.dirname(filename)
+ if newfile == filename: # reached the end of the line
+ part = filename
+ newfile = ""
+ if member != "":
+ newmember = path.join(part, member)
+ else:
+ newmember = part
+ return test_is_zip(newfile, newmember)
diff --git a/Stoner/plot/core.py b/Stoner/plot/core.py
index 9a0954654..ef513ba31 100755
--- a/Stoner/plot/core.py
+++ b/Stoner/plot/core.py
@@ -1,9 +1,10 @@
-"""Provides the a class to facilitate easier plotting of Stoner Data.
+"""Provides the a class to facilitte easier plotting of Stoner Data.
Classes:
PlotMixin:
A class that uses matplotlib to plot data
"""
+
# pylint: disable=C0413
from __future__ import division
@@ -12,6 +13,7 @@
from collections.abc import Mapping
from functools import wraps
import copy
+from inspect import getfullargspec
import numpy as np
from scipy.interpolate import griddata
@@ -20,7 +22,7 @@
from matplotlib import figure as mplfig
from matplotlib import cm, colors, colormaps
-from Stoner.compat import string_types, index_types, int_types, getargspec
+from Stoner.compat import string_types, index_types, int_types
from Stoner.tools import AttributeStore, isnone, isanynone, all_type, isiterable, typedList, get_option, fix_signature
from .formats import DefaultPlotStyle
from .utils import errorfill
@@ -37,6 +39,24 @@
_3D = False
+def _getargspec(*args, **kargs):
+ """Get the function signature spec."""
+ ret = getfullargspec(*args, **kargs)
+ if ret.args and ret.args[0] == "self": # remove self for bound methods
+ del ret.args[0]
+ deflen = len(ret.defaults) if ret.defaults else 0
+ kwargs = ret.args[-deflen:]
+ kwargs.extend(ret.kwonlyargs)
+ args = ret.args[: len(ret.args) - deflen]
+ defaults = list(ret.defaults) if ret.defaults else []
+ if ret.kwonlydefaults:
+ defaults.extend(list(ret.kwonlydefaults.values()))
+ return args, kwargs, defaults, len(kwargs) - len(ret.kwonlyargs)
+
+
+FIG_KARGS = _getargspec(plt.figure)[1] + ["ax"]
+
+
def __mpl3DQuiver(x_coord, y_coord, z_coord, u_comp, v_comp, w_comp, **kargs):
"""Plot vector fields using mpltoolkit.quiver.
@@ -211,7 +231,8 @@ def labels(self, value):
def showfig(self):
"""Return either the current figure or self or None.
- The return value depends on whether the attribute is True or False or None."""
+ The return value depends on whether the attribute is True or False or None.
+ """
if self._showfig is None or get_option("no_figs"):
return None
if self._showfig:
@@ -305,10 +326,14 @@ def _surface_plotter(self, x_coord, y_coord, z_coord, **kargs):
ReturnsL
A matplotib Figure
- This function attempts to work the same as the 2D surface plotter pcolor, but draws a 3D axes set"""
+ This function attempts to work the same as the 2D surface plotter pcolor, but draws a 3D axes set
+ """
if not _3D:
raise RuntimeError("3D plotting Not available. Install matplotlib toolkits")
- ax = plt.axes(projection="3d")
+ if not isinstance(self.__figure.gca(), Axes3D):
+ ax = plt.axes(projection="3d")
+ else:
+ ax = self.__figure.gca()
z_coord = np.nan_to_num(z_coord)
surf = ax.plot_surface(x_coord, y_coord, z_coord, **kargs)
self.fig.colorbar(surf, shrink=0.5, aspect=5, extend="both")
@@ -328,7 +353,7 @@ def _vector_color(self, xcol=None, ycol=None, ucol=None, vcol=None, wcol=None, *
qdata = 0.5 + (np.arctan2(self.column(c.ucol), self.column(c.vcol)) / (2 * np.pi))
rdata = np.sqrt(self.column(c.ucol) ** 2 + self.column(c.vcol) ** 2 + wdata**2)
rdata = rdata / rdata.max()
- Z = hsl2rgb(qdata, rdata, phidata).astype("f") / 255.0
+ Z = hsl2rgb(qdata, rdata, phidata).astype("f") / 255.0001 + 1e-7
return Z
def _span_slice(self, col, num):
@@ -436,6 +461,7 @@ def _fix_fig(self, figure, **kargs):
else:
figure, ax = self.template.new_figure(None, **kargs)
self.__figure = figure
+ figure.sca(ax) # Esur4e we're set for plotting on the correct axes
return figure, ax
def _fix_kargs(self, function=None, defaults=None, otherkargs=None, **kargs):
@@ -448,10 +474,8 @@ def _fix_kargs(self, function=None, defaults=None, otherkargs=None, **kargs):
defaults = dict()
defaults.update(kargs)
- fig_kargs = ["num", "figsize", "dpi", "facecolor", "edgecolor", "frameon", "FigureClass", "clear", "ax"]
-
pass_fig_kargs = {}
- for k in set(fig_kargs) & set(kargs.keys()):
+ for k in set(FIG_KARGS) & set(kargs.keys()):
pass_fig_kargs[k] = kargs[k]
if k not in otherkargs and k not in defaults:
del kargs[k]
@@ -469,16 +493,18 @@ def _fix_kargs(self, function=None, defaults=None, otherkargs=None, **kargs):
if self.__figure is not plt.gcf():
plt.close(plt.gcf())
- (args, _, kwargs) = getargspec(function)[:3]
+ (args, kwargs) = _getargspec(function)[:2]
# Manually override the list of arguments that the plotting function takes if it takes keyword dictionary
- if isinstance(otherkargs, (list, tuple)) and kwargs is not None:
- args.extend(otherkargs)
- nonkargs = dict()
- for k in list(defaults.keys()):
- nonkargs[k] = defaults[k]
- if k not in args:
- del defaults[k]
- return defaults, nonkargs, pass_fig_kargs
+ if isinstance(otherkargs, (list, tuple)):
+ kwargs.extend(otherkargs)
+ nonkargs = {}
+ func_kwargs = {}
+ for key, value in defaults.items():
+ if key in kwargs:
+ func_kwargs[key] = value
+ else:
+ nonkargs[key] = value
+ return func_kwargs, nonkargs, pass_fig_kargs
def _fix_titles(self, ix, multiple, **kargs):
"""Do the titling and labelling for a matplotlib plot."""
@@ -736,7 +762,6 @@ def colormap_xyz(self, xcol=None, ycol=None, zcol=None, **kargs):
ax = self.plot_xyz(xcol, ycol, zcol, shape, xlim, ylim, **kargs)
if colorbar:
plt.colorbar()
- plt.tight_layout()
return ax
def contour_xyz(self, xcol=None, ycol=None, zcol=None, shape=None, xlim=None, ylim=None, plotter=None, **kargs):
@@ -1306,6 +1331,7 @@ def plot_xy(self, xcol=None, ycol=None, fmt=None, xerr=None, yerr=None, **kargs)
title = kargs.pop("title", self.basename)
defaults = {
+ "capsize": 4,
"plotter": plt.plot,
"show_plot": True,
"figure": self.__figure,
@@ -1456,11 +1482,13 @@ def plot_xy(self, xcol=None, ycol=None, fmt=None, xerr=None, yerr=None, **kargs)
c.ycol = [c.ycol]
if len(c.ycol) > 1:
if multiple == "panels":
- self.__figure, _ = plt.subplots(nrows=len(c.ycol), sharex=True, gridspec_kw={"hspace": 0})
+ self.__figure, _ = plt.subplots(
+ nrows=len(c.ycol), sharex=True, gridspec_kw={"hspace": 0}, layout="constrained", **fig_kargs
+ )
elif multiple == "subplots":
m = int(np.floor(np.sqrt(len(c.ycol))))
n = int(np.ceil(len(c.ycol) / m))
- self.__figure, _ = plt.subplots(nrows=m, ncols=n)
+ self.__figure, _ = plt.subplots(nrows=m, ncols=n, layout="constrained", **fig_kargs)
else:
self.__figure, _ = self._fix_fig(nonkargs["figure"], **fig_kargs)
else:
@@ -1508,7 +1536,7 @@ def plot_xy(self, xcol=None, ycol=None, fmt=None, xerr=None, yerr=None, **kargs)
if ix > 0: # Hooks for multiple subplots
if multiple == "panels":
loc, lab = plt.yticks()
- lab = [l.get_text() for l in lab]
+ lab = [label.get_text() for label in lab]
plt.yticks(loc[:-1], lab[:-1])
return self.showfig
@@ -1602,16 +1630,15 @@ def plot_xyz(self, xcol=None, ycol=None, zcol=None, shape=None, xlim=None, ylim=
"shade",
"linewidth",
"ax",
+ "alpha",
]
else:
otherkargs = ["vmin", "vmax", "shade", "color", "linewidth", "marker"]
- kargs, nonkargs, _ = self._fix_kargs(
- kargs.get("plotter", None), defaults, otherkargs=otherkargs, projection=projection, **kargs
- )
- plotter = nonkargs["plotter"]
- self.__figure, ax = self._fix_fig(nonkargs["figure"], projection=projection)
+ plotter = kargs.get("plotter", defaults["plotter"])
+ self.__figure, ax = self._fix_fig(kargs.get("figure", defaults["figure"]), projection=projection)
if isinstance(plotter, string_types):
plotter = ax.__getattribute__(plotter)
+ kargs, nonkargs, _ = self._fix_kargs(plotter, defaults, otherkargs=otherkargs, projection=projection, **kargs)
self.plot3d = plotter(xdata, ydata, zdata, **kargs)
if plotter is not self._surface_plotter:
del nonkargs["zlabel"]
@@ -1621,38 +1648,38 @@ def plot_xyz(self, xcol=None, ycol=None, zcol=None, shape=None, xlim=None, ylim=
def plot_xyuv(self, xcol=None, ycol=None, ucol=None, vcol=None, wcol=None, **kargs):
"""Make an overlaid image and quiver plot.
- Args:
- xcol (index):
- Xcolumn index or label
- ycol (index):
- Y column index or label
- zcol (index):
- Z column index or label
- ucol (index):
- U column index or label
- vcol (index):
- V column index or label
- wcol (index):
- W column index or label
-
- Keyword Arguments:
- show_plot (bool):
- True Turns on interactive plot control
- title (string):
- Optional parameter that specifies the plot title - otherwise the current DataFile filename is used
- save_filename (string):
- Filename used to save the plot
- figure (matplotlib figure):
- Controls what matplotlib figure to use. Can be an integer, or a matplotlib.figure or False. If False
- then a new figure is always used, otherwise it will default to using the last figure used by this
- DataFile object.
- no_quiver (bool):
- Do not overlay quiver plot (in cases of dense meshes of points)
- plotter (callable):
- Optional argument that passes a plotting function into the routine. Default is a 3d surface plotter,
- but contour plot and pcolormesh also work.
- **kargs (dict):
- A dictionary of other keyword arguments to pass into the plot function.
+ Args:
+ !c xcol (index):
+ Xcolumn index or label
+ ycol (index):
+ Y column index or label
+ zcol (index):
+ Z column index or label
+ ucol (index):
+ U column index or label
+ vcol (index):
+ V column index or label
+ wcol (index):
+ W column index or label
+
+ Keyword Arguments:
+ show_plot (bool):
+ True Turns on interactive plot control
+ title (string):
+ Optional parameter that specifies the plot title - otherwise the current DataFile filename is used
+ save_filename (string):
+ Filename used to save the plot
+ figure (matplotlib figure):
+ Controls what matplotlib figure to use. Can be an integer, or a matplotlib.figure or False. If False
+ then a new figure is always used, otherwise it will default to using the last figure used by this
+ DataFile object.
+ no_quiver (bool):
+ Do not overlay quiver plot (in cases of dense meshes of points)
+ plotter (callable):
+ Optional argument that passes a plotting function into the routine. Default is a 3d surface plotter,
+ but contour plot and pcolormesh also work.
+ **kargs (dict):
+ A dictionary of other keyword arguments to pass into the plot function.
"""
c = self._fix_cols(xcol=xcol, ycol=ycol, ucol=ucol, vcol=vcol, wcol=wcol, **kargs)
Z = self._vector_color(xcol=xcol, ycol=ycol, ucol=ucol, vcol=vcol, wcol=wcol)
@@ -2042,7 +2069,7 @@ def subplot(self, *args, **kargs):
def subplot2grid(self, *args, **kargs):
"""Provide a pass through to :py:func:`matplotlib.pyplot.subplot2grid`."""
if self.__figure is None:
- self.figure()
+ self.figure(no_axes=True)
figure = self.template.new_figure(self.__figure.number, no_axes=True)[0]
@@ -2069,6 +2096,6 @@ def y2(self):
"""
ax = self.fig.gca()
ax2 = ax.twinx()
- plt.subplots_adjust(right=self.__figure.subplotpars.right - 0.05)
+ # plt.subplots_adjust(right=self.__figure.subplotpars.right - 0.05)
plt.sca(ax2)
return ax2
diff --git a/Stoner/plot/formats.py b/Stoner/plot/formats.py
index 74cedc135..8c911b81c 100755
--- a/Stoner/plot/formats.py
+++ b/Stoner/plot/formats.py
@@ -19,6 +19,7 @@
import matplotlib.pyplot as plt
from matplotlib.ticker import EngFormatter, Formatter
from matplotlib.ticker import AutoLocator
+from mpl_toolkits.mplot3d import Axes3D
import numpy as np
from numpy.random import normal
@@ -82,7 +83,6 @@ def format_data_short(self, value): # pylint: disable=r0201
class TexEngFormatter(EngFormatter):
-
"""An axis tick label formatter that emits Tex formula mode code.
Formatting is set so that large numbers are registered as with SI prefixes
@@ -145,7 +145,6 @@ def format_data_short(self, value): # pylint: disable=r0201
class DefaultPlotStyle(MutableMapping):
-
"""Produces a default plot style.
To produce alternative plot styles, create subclasses of this plot. Either override or
@@ -418,14 +417,13 @@ def new_figure(self, figure=False, **kargs):
params[attrname] = value
projection = kargs.pop("projection", "rectilinear")
self.template_figure__figsize = kargs.pop("figsize", self.template_figure__figsize) # pylint: disable=W0201
- if "ax" in kargs: # Giving an axis instance in kargs means we can use that as our figure
- ax = kargs.get("ax")
- plt.sca(ax)
- figure = plt.gcf().number
+ if "ax" in kargs and isinstance(kargs["ax"], (Axes3D, plt.Axes)):
+ # Giving an axis instance in kargs means we can use that as our figure
+ figure = kargs["ax"].figure.number
if isinstance(figure, bool) and not figure:
return None, None
elif figure is not None:
- fig = plt.figure(figure, figsize=self.template_figure__figsize)
+ fig = plt.figure(figure, figsize=self.template_figure__figsize, layout="constrained")
if len(fig.axes) == 0:
rect = [plt.rcParams[f"figure.subplot.{i}"] for i in ["left", "bottom", "right", "top"]]
rect[2] = rect[2] - rect[0]
@@ -446,7 +444,7 @@ def new_figure(self, figure=False, **kargs):
ax = kargs.pop("ax")
else:
for ax in plt.gcf().axes:
- if "zaxis" in ax.properties():
+ if isinstance(ax, Axes3D):
break
else:
ax = plt.axes(projection="3d")
@@ -457,6 +455,7 @@ def new_figure(self, figure=False, **kargs):
else:
no_axes = kargs.pop("no_axes", False)
if projection == "3d":
+ kargs.setdefault("layout", "constrained")
ret = plt.figure(figsize=self.template_figure__figsize, **kargs)
if not no_axes:
ax = ret.add_subplot(111, projection="3d")
@@ -466,6 +465,7 @@ def new_figure(self, figure=False, **kargs):
ax.remove()
return ret, None
else:
+ kargs.setdefault("layout", "constrained")
if not no_axes:
return plt.subplots(figsize=self.template_figure__figsize, **kargs)
else:
@@ -541,7 +541,7 @@ def annotate(self, ix, multiple, plot, **kargs):
if multiple in self.subplot_settings:
if ix == 0:
i = 0
- elif ix == len(plot.axes):
+ elif ix == len(plot.axes) - 1:
i = 2
else:
i = 1
@@ -564,7 +564,6 @@ def annotate(self, ix, multiple, plot, **kargs):
class GBPlotStyle(DefaultPlotStyle):
-
"""Template developed for Gavin's plotting.
This is largely an experimental class for trying things out rather than
@@ -593,7 +592,6 @@ def customise_axes(self, ax, plot):
class JTBPlotStyle(DefaultPlotStyle):
-
"""Template class for Joe's Plot settings.
Example:
@@ -610,7 +608,6 @@ def customise_axes(self, ax, plot):
class JTBinsetStyle(DefaultPlotStyle):
-
"""Template class for Joe's Plot settings."""
show_title = False
@@ -621,7 +618,6 @@ def customise_axes(self, ax, plot):
class ThesisPlotStyle(DefaultPlotStyle):
-
"""Template class for Joe's Plot settings."""
show_title = False
@@ -629,7 +625,6 @@ class ThesisPlotStyle(DefaultPlotStyle):
class PRBPlotStyle(DefaultPlotStyle):
-
"""A figure Style for making figures for Phys Rev * Jounrals.
Example:
@@ -647,7 +642,6 @@ def customise_axes(self, ax, plot):
class SketchPlot(DefaultPlotStyle):
-
"""Turn on xkcd plot style.
Implemented as a bit of a joke, but perhaps someone will use this in a real
@@ -690,7 +684,6 @@ def customise_axes(self, ax, plot):
if SEABORN: # extra classes if we have seaborn available
class SeabornPlotStyle(DefaultPlotStyle):
-
"""A plotdtyle that makes use of the seaborn plotting package to make visually attractive plots.
Attributes:
diff --git a/Stoner/tools/classes.py b/Stoner/tools/classes.py
index 0fd8a4b7d..f8f7c3bec 100755
--- a/Stoner/tools/classes.py
+++ b/Stoner/tools/classes.py
@@ -31,7 +31,6 @@
class attributeStore(dict):
-
"""A dictionary=like class that provides attributes that work like indices.
Used to implement the mapping of column types to indices in the setas attriobutes.
@@ -124,7 +123,6 @@ def subclasses(cls: Optional[type] = None) -> Dict: # pylint: disable=no-self-a
class typedList(MutableSequence):
-
"""Subclass list to make setitem enforce strict typing of members of the list."""
def __init__(self, *args: Any, **kargs: Any) -> None:
@@ -248,7 +246,6 @@ def set_option(name: str, value: bool) -> None:
class Options:
-
"""Dead simple class to allow access to package options."""
def __init__(self):
diff --git a/Stoner/tools/file.py b/Stoner/tools/file.py
index 193d7c289..4a316132d 100755
--- a/Stoner/tools/file.py
+++ b/Stoner/tools/file.py
@@ -196,7 +196,6 @@ def get_mime_type(filename: Union[pathlib.Path, str], debug: bool = False) -> Op
class FileManager:
-
"""Simple context manager that allows opening files or working with already open string buffers."""
def __init__(self, filename, *args, **kargs):
@@ -265,7 +264,6 @@ def __exit__(self, exc_type, exc_value, exc_traceback):
class SizedFileManager(FileManager):
-
"""Context manager that figures out the size of the file as well as opening it."""
def __enter__(self):
diff --git a/Stoner/tools/widgets.py b/Stoner/tools/widgets.py
index 93ae24d3e..b050a23c6 100755
--- a/Stoner/tools/widgets.py
+++ b/Stoner/tools/widgets.py
@@ -18,7 +18,6 @@
except ImportError:
class App:
-
"""Mock App that raises an error when you try to call openDialog on it."""
modes: Dict = dict()
@@ -36,7 +35,6 @@ def openDialog(
else:
class App(QApplication):
-
"""Placehold PyQT5 Application for producing filedialog boxes."""
modes = {
@@ -141,7 +139,6 @@ def openDialog(
class RangeSelect:
-
"""A simple class to allow a matplotlib graph to be used to select data."""
def __init__(self):
diff --git a/doc/.coveralls.yml b/doc/.coveralls.yml
deleted file mode 100755
index f2a441ff3..000000000
--- a/doc/.coveralls.yml
+++ /dev/null
@@ -1,2 +0,0 @@
-service_name: travis-pro
-repo_token: kwYbCcg0mDbFK4fLXaV7JayClxaOAcWPd
diff --git a/doc/Makefile b/doc/Makefile
index aa705faa2..986b64d96 100755
--- a/doc/Makefile
+++ b/doc/Makefile
@@ -42,7 +42,7 @@ clean:
-rm -rf classes/* functions/*
readme:
- cp readme.rst ../README.rst
+ cp ../README.rst readme.rst
html: readme
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
diff --git a/doc/plot_cache/3D.hires.png b/doc/plot_cache/3D.hires.png
index 21c286cc3..f28924a50 100755
Binary files a/doc/plot_cache/3D.hires.png and b/doc/plot_cache/3D.hires.png differ
diff --git a/doc/plot_cache/3D.pdf b/doc/plot_cache/3D.pdf
index 6d100b43f..17aa45c30 100755
Binary files a/doc/plot_cache/3D.pdf and b/doc/plot_cache/3D.pdf differ
diff --git a/doc/plot_cache/3D.png b/doc/plot_cache/3D.png
index 1e436e10a..837b59292 100755
Binary files a/doc/plot_cache/3D.png and b/doc/plot_cache/3D.png differ
diff --git a/doc/plot_cache/arrhenius-class.hires.png b/doc/plot_cache/arrhenius-class.hires.png
index 3bbec3f64..1f37a415d 100755
Binary files a/doc/plot_cache/arrhenius-class.hires.png and b/doc/plot_cache/arrhenius-class.hires.png differ
diff --git a/doc/plot_cache/arrhenius-class.pdf b/doc/plot_cache/arrhenius-class.pdf
index 944e83c2f..0239429fc 100755
Binary files a/doc/plot_cache/arrhenius-class.pdf and b/doc/plot_cache/arrhenius-class.pdf differ
diff --git a/doc/plot_cache/arrhenius-class.png b/doc/plot_cache/arrhenius-class.png
index 3a232d030..9b7875a6b 100755
Binary files a/doc/plot_cache/arrhenius-class.png and b/doc/plot_cache/arrhenius-class.png differ
diff --git a/doc/plot_cache/bdr-class.hires.png b/doc/plot_cache/bdr-class.hires.png
index 56d0082f7..07d847785 100755
Binary files a/doc/plot_cache/bdr-class.hires.png and b/doc/plot_cache/bdr-class.hires.png differ
diff --git a/doc/plot_cache/bdr-class.pdf b/doc/plot_cache/bdr-class.pdf
index 039f749ba..b6d8be279 100755
Binary files a/doc/plot_cache/bdr-class.pdf and b/doc/plot_cache/bdr-class.pdf differ
diff --git a/doc/plot_cache/bdr-class.png b/doc/plot_cache/bdr-class.png
index f363b0d87..0353f407d 100755
Binary files a/doc/plot_cache/bdr-class.png and b/doc/plot_cache/bdr-class.png differ
diff --git a/doc/plot_cache/bins.hires.png b/doc/plot_cache/bins.hires.png
index 6b9152e9c..428058919 100755
Binary files a/doc/plot_cache/bins.hires.png and b/doc/plot_cache/bins.hires.png differ
diff --git a/doc/plot_cache/bins.pdf b/doc/plot_cache/bins.pdf
index bd75e146f..7c9528a49 100755
Binary files a/doc/plot_cache/bins.pdf and b/doc/plot_cache/bins.pdf differ
diff --git a/doc/plot_cache/bins.png b/doc/plot_cache/bins.png
index babf86ac9..884d87f37 100755
Binary files a/doc/plot_cache/bins.png and b/doc/plot_cache/bins.png differ
diff --git a/doc/plot_cache/blochgruneisen-class.hires.png b/doc/plot_cache/blochgruneisen-class.hires.png
index 2959ebb04..05cfb12c0 100755
Binary files a/doc/plot_cache/blochgruneisen-class.hires.png and b/doc/plot_cache/blochgruneisen-class.hires.png differ
diff --git a/doc/plot_cache/blochgruneisen-class.pdf b/doc/plot_cache/blochgruneisen-class.pdf
index fa670ec12..e7e9fc87d 100755
Binary files a/doc/plot_cache/blochgruneisen-class.pdf and b/doc/plot_cache/blochgruneisen-class.pdf differ
diff --git a/doc/plot_cache/blochgruneisen-class.png b/doc/plot_cache/blochgruneisen-class.png
index a5b815f45..5e37c4d04 100755
Binary files a/doc/plot_cache/blochgruneisen-class.png and b/doc/plot_cache/blochgruneisen-class.png differ
diff --git a/doc/plot_cache/channel_math.hires.png b/doc/plot_cache/channel_math.hires.png
index 6bbea04cb..5de2e521c 100755
Binary files a/doc/plot_cache/channel_math.hires.png and b/doc/plot_cache/channel_math.hires.png differ
diff --git a/doc/plot_cache/channel_math.pdf b/doc/plot_cache/channel_math.pdf
index 0a9b77a48..12c09832e 100755
Binary files a/doc/plot_cache/channel_math.pdf and b/doc/plot_cache/channel_math.pdf differ
diff --git a/doc/plot_cache/channel_math.png b/doc/plot_cache/channel_math.png
index 6ba743115..796953415 100755
Binary files a/doc/plot_cache/channel_math.png and b/doc/plot_cache/channel_math.png differ
diff --git a/doc/plot_cache/colormap.hires.png b/doc/plot_cache/colormap.hires.png
index 4e3355cb1..ed6c881d1 100755
Binary files a/doc/plot_cache/colormap.hires.png and b/doc/plot_cache/colormap.hires.png differ
diff --git a/doc/plot_cache/colormap.pdf b/doc/plot_cache/colormap.pdf
index f4497cff1..56915f475 100755
Binary files a/doc/plot_cache/colormap.pdf and b/doc/plot_cache/colormap.pdf differ
diff --git a/doc/plot_cache/colormap.png b/doc/plot_cache/colormap.png
index 03619ba4e..3f6d10440 100755
Binary files a/doc/plot_cache/colormap.png and b/doc/plot_cache/colormap.png differ
diff --git a/doc/plot_cache/common_y.hires.png b/doc/plot_cache/common_y.hires.png
index 8955bbf19..6e69e3af2 100755
Binary files a/doc/plot_cache/common_y.hires.png and b/doc/plot_cache/common_y.hires.png differ
diff --git a/doc/plot_cache/common_y.pdf b/doc/plot_cache/common_y.pdf
index 06cc8f9d6..7ba319747 100755
Binary files a/doc/plot_cache/common_y.pdf and b/doc/plot_cache/common_y.pdf differ
diff --git a/doc/plot_cache/common_y.png b/doc/plot_cache/common_y.png
index 7eb3f0779..c94896d38 100755
Binary files a/doc/plot_cache/common_y.png and b/doc/plot_cache/common_y.png differ
diff --git a/doc/plot_cache/contour.hires.png b/doc/plot_cache/contour.hires.png
index a1641135b..51600e91b 100755
Binary files a/doc/plot_cache/contour.hires.png and b/doc/plot_cache/contour.hires.png differ
diff --git a/doc/plot_cache/contour.pdf b/doc/plot_cache/contour.pdf
index e8b01e900..7c5a1cf49 100755
Binary files a/doc/plot_cache/contour.pdf and b/doc/plot_cache/contour.pdf differ
diff --git a/doc/plot_cache/contour.png b/doc/plot_cache/contour.png
index 4697e598c..bb01b9db7 100755
Binary files a/doc/plot_cache/contour.png and b/doc/plot_cache/contour.png differ
diff --git a/doc/plot_cache/curve_fit_line.hires.png b/doc/plot_cache/curve_fit_line.hires.png
index d24fede7d..009f6516d 100755
Binary files a/doc/plot_cache/curve_fit_line.hires.png and b/doc/plot_cache/curve_fit_line.hires.png differ
diff --git a/doc/plot_cache/curve_fit_line.pdf b/doc/plot_cache/curve_fit_line.pdf
index 92a3d9701..f73745298 100755
Binary files a/doc/plot_cache/curve_fit_line.pdf and b/doc/plot_cache/curve_fit_line.pdf differ
diff --git a/doc/plot_cache/curve_fit_line.png b/doc/plot_cache/curve_fit_line.png
index 2d33665cb..ddafa5dce 100755
Binary files a/doc/plot_cache/curve_fit_line.png and b/doc/plot_cache/curve_fit_line.png differ
diff --git a/doc/plot_cache/curvefit_plane.hires.png b/doc/plot_cache/curvefit_plane.hires.png
index 06159702a..ce0886fc9 100755
Binary files a/doc/plot_cache/curvefit_plane.hires.png and b/doc/plot_cache/curvefit_plane.hires.png differ
diff --git a/doc/plot_cache/curvefit_plane.pdf b/doc/plot_cache/curvefit_plane.pdf
index f12b4a029..bc8957b1f 100755
Binary files a/doc/plot_cache/curvefit_plane.pdf and b/doc/plot_cache/curvefit_plane.pdf differ
diff --git a/doc/plot_cache/curvefit_plane.png b/doc/plot_cache/curvefit_plane.png
index 98ec7eb1c..2b888c891 100755
Binary files a/doc/plot_cache/curvefit_plane.png and b/doc/plot_cache/curvefit_plane.png differ
diff --git a/doc/plot_cache/curvefit_plane2.hires.png b/doc/plot_cache/curvefit_plane2.hires.png
index 9e62eee5a..5fb2a9443 100755
Binary files a/doc/plot_cache/curvefit_plane2.hires.png and b/doc/plot_cache/curvefit_plane2.hires.png differ
diff --git a/doc/plot_cache/curvefit_plane2.pdf b/doc/plot_cache/curvefit_plane2.pdf
index a86898647..3cd012ab5 100755
Binary files a/doc/plot_cache/curvefit_plane2.pdf and b/doc/plot_cache/curvefit_plane2.pdf differ
diff --git a/doc/plot_cache/curvefit_plane2.png b/doc/plot_cache/curvefit_plane2.png
index 5c70980ad..9ddd42a8b 100755
Binary files a/doc/plot_cache/curvefit_plane2.png and b/doc/plot_cache/curvefit_plane2.png differ
diff --git a/doc/plot_cache/curvefit_sphere.hires.png b/doc/plot_cache/curvefit_sphere.hires.png
index a38eb718d..d050e14ff 100755
Binary files a/doc/plot_cache/curvefit_sphere.hires.png and b/doc/plot_cache/curvefit_sphere.hires.png differ
diff --git a/doc/plot_cache/curvefit_sphere.pdf b/doc/plot_cache/curvefit_sphere.pdf
index cd6c3d3e9..8667ecd7f 100755
Binary files a/doc/plot_cache/curvefit_sphere.pdf and b/doc/plot_cache/curvefit_sphere.pdf differ
diff --git a/doc/plot_cache/curvefit_sphere.png b/doc/plot_cache/curvefit_sphere.png
index 460818afc..6fbf25bcb 100755
Binary files a/doc/plot_cache/curvefit_sphere.png and b/doc/plot_cache/curvefit_sphere.png differ
diff --git a/doc/plot_cache/curvefit_sphere_2.hires.png b/doc/plot_cache/curvefit_sphere_2.hires.png
index a38eb718d..d050e14ff 100755
Binary files a/doc/plot_cache/curvefit_sphere_2.hires.png and b/doc/plot_cache/curvefit_sphere_2.hires.png differ
diff --git a/doc/plot_cache/curvefit_sphere_2.pdf b/doc/plot_cache/curvefit_sphere_2.pdf
index 4bd353966..255b4a497 100755
Binary files a/doc/plot_cache/curvefit_sphere_2.pdf and b/doc/plot_cache/curvefit_sphere_2.pdf differ
diff --git a/doc/plot_cache/curvefit_sphere_2.png b/doc/plot_cache/curvefit_sphere_2.png
index 460818afc..6fbf25bcb 100755
Binary files a/doc/plot_cache/curvefit_sphere_2.png and b/doc/plot_cache/curvefit_sphere_2.png differ
diff --git a/doc/plot_cache/customising.hires.png b/doc/plot_cache/customising.hires.png
index f145da5c5..22814734c 100755
Binary files a/doc/plot_cache/customising.hires.png and b/doc/plot_cache/customising.hires.png differ
diff --git a/doc/plot_cache/customising.pdf b/doc/plot_cache/customising.pdf
index 2f975fe81..c526749e0 100755
Binary files a/doc/plot_cache/customising.pdf and b/doc/plot_cache/customising.pdf differ
diff --git a/doc/plot_cache/customising.png b/doc/plot_cache/customising.png
index 3c531c424..af6b911cd 100755
Binary files a/doc/plot_cache/customising.png and b/doc/plot_cache/customising.png differ
diff --git a/doc/plot_cache/decompose.hires.png b/doc/plot_cache/decompose.hires.png
index e910b1456..fd97d49ae 100755
Binary files a/doc/plot_cache/decompose.hires.png and b/doc/plot_cache/decompose.hires.png differ
diff --git a/doc/plot_cache/decompose.pdf b/doc/plot_cache/decompose.pdf
index b61a260d5..21fdcae2e 100755
Binary files a/doc/plot_cache/decompose.pdf and b/doc/plot_cache/decompose.pdf differ
diff --git a/doc/plot_cache/decompose.png b/doc/plot_cache/decompose.png
index 82d2b1f40..1ac2dda72 100755
Binary files a/doc/plot_cache/decompose.png and b/doc/plot_cache/decompose.png differ
diff --git a/doc/plot_cache/defaultstyle.hires.png b/doc/plot_cache/defaultstyle.hires.png
index 2a5ca1166..d6e1c19a3 100755
Binary files a/doc/plot_cache/defaultstyle.hires.png and b/doc/plot_cache/defaultstyle.hires.png differ
diff --git a/doc/plot_cache/defaultstyle.pdf b/doc/plot_cache/defaultstyle.pdf
index d62d66019..dc2824e09 100755
Binary files a/doc/plot_cache/defaultstyle.pdf and b/doc/plot_cache/defaultstyle.pdf differ
diff --git a/doc/plot_cache/defaultstyle.png b/doc/plot_cache/defaultstyle.png
index b2f0a0394..bb8104a42 100755
Binary files a/doc/plot_cache/defaultstyle.png and b/doc/plot_cache/defaultstyle.png differ
diff --git a/doc/plot_cache/diffev1.hires.png b/doc/plot_cache/diffev1.hires.png
index acd2d4dbc..cc4f81705 100755
Binary files a/doc/plot_cache/diffev1.hires.png and b/doc/plot_cache/diffev1.hires.png differ
diff --git a/doc/plot_cache/diffev1.pdf b/doc/plot_cache/diffev1.pdf
index a523c8a40..b107b5c97 100755
Binary files a/doc/plot_cache/diffev1.pdf and b/doc/plot_cache/diffev1.pdf differ
diff --git a/doc/plot_cache/diffev1.png b/doc/plot_cache/diffev1.png
index 9b343910f..ff7d59c9e 100755
Binary files a/doc/plot_cache/diffev1.png and b/doc/plot_cache/diffev1.png differ
diff --git a/doc/plot_cache/diffev2.hires.png b/doc/plot_cache/diffev2.hires.png
index acd2d4dbc..cc4f81705 100755
Binary files a/doc/plot_cache/diffev2.hires.png and b/doc/plot_cache/diffev2.hires.png differ
diff --git a/doc/plot_cache/diffev2.pdf b/doc/plot_cache/diffev2.pdf
index 838283547..3804922e8 100755
Binary files a/doc/plot_cache/diffev2.pdf and b/doc/plot_cache/diffev2.pdf differ
diff --git a/doc/plot_cache/diffev2.png b/doc/plot_cache/diffev2.png
index 9b343910f..ff7d59c9e 100755
Binary files a/doc/plot_cache/diffev2.png and b/doc/plot_cache/diffev2.png differ
diff --git a/doc/plot_cache/double_y.hires.png b/doc/plot_cache/double_y.hires.png
index 4cde1a3f4..ab6c8af9f 100755
Binary files a/doc/plot_cache/double_y.hires.png and b/doc/plot_cache/double_y.hires.png differ
diff --git a/doc/plot_cache/double_y.pdf b/doc/plot_cache/double_y.pdf
index 4c8b1998f..9c5391089 100755
Binary files a/doc/plot_cache/double_y.pdf and b/doc/plot_cache/double_y.pdf differ
diff --git a/doc/plot_cache/double_y.png b/doc/plot_cache/double_y.png
index 1298d7361..66ee348e0 100755
Binary files a/doc/plot_cache/double_y.png and b/doc/plot_cache/double_y.png differ
diff --git a/doc/plot_cache/extrapolate-demo.hires.png b/doc/plot_cache/extrapolate-demo.hires.png
index 96085dd6c..9ea3dfa69 100755
Binary files a/doc/plot_cache/extrapolate-demo.hires.png and b/doc/plot_cache/extrapolate-demo.hires.png differ
diff --git a/doc/plot_cache/extrapolate-demo.pdf b/doc/plot_cache/extrapolate-demo.pdf
index dbdf41d5f..0d21b4766 100755
Binary files a/doc/plot_cache/extrapolate-demo.pdf and b/doc/plot_cache/extrapolate-demo.pdf differ
diff --git a/doc/plot_cache/extrapolate-demo.png b/doc/plot_cache/extrapolate-demo.png
index 2cbc8b5e5..3329ad9b3 100755
Binary files a/doc/plot_cache/extrapolate-demo.png and b/doc/plot_cache/extrapolate-demo.png differ
diff --git a/doc/plot_cache/fluchsdondheimer-class.hires.png b/doc/plot_cache/fluchsdondheimer-class.hires.png
index 821187604..032e8ed0b 100755
Binary files a/doc/plot_cache/fluchsdondheimer-class.hires.png and b/doc/plot_cache/fluchsdondheimer-class.hires.png differ
diff --git a/doc/plot_cache/fluchsdondheimer-class.pdf b/doc/plot_cache/fluchsdondheimer-class.pdf
index bd2ab48bc..04f01249c 100755
Binary files a/doc/plot_cache/fluchsdondheimer-class.pdf and b/doc/plot_cache/fluchsdondheimer-class.pdf differ
diff --git a/doc/plot_cache/fluchsdondheimer-class.png b/doc/plot_cache/fluchsdondheimer-class.png
index a24075f01..69fb293ca 100755
Binary files a/doc/plot_cache/fluchsdondheimer-class.png and b/doc/plot_cache/fluchsdondheimer-class.png differ
diff --git a/doc/plot_cache/fowlernordheim-class.hires.png b/doc/plot_cache/fowlernordheim-class.hires.png
index 9d018976c..5048b1780 100755
Binary files a/doc/plot_cache/fowlernordheim-class.hires.png and b/doc/plot_cache/fowlernordheim-class.hires.png differ
diff --git a/doc/plot_cache/fowlernordheim-class.pdf b/doc/plot_cache/fowlernordheim-class.pdf
index 0e5de2bf9..9815077b4 100755
Binary files a/doc/plot_cache/fowlernordheim-class.pdf and b/doc/plot_cache/fowlernordheim-class.pdf differ
diff --git a/doc/plot_cache/fowlernordheim-class.png b/doc/plot_cache/fowlernordheim-class.png
index 4667dccf1..d6683f71f 100755
Binary files a/doc/plot_cache/fowlernordheim-class.png and b/doc/plot_cache/fowlernordheim-class.png differ
diff --git a/doc/plot_cache/gbstyle.hires.png b/doc/plot_cache/gbstyle.hires.png
index 3d1522a7c..55f52fb97 100755
Binary files a/doc/plot_cache/gbstyle.hires.png and b/doc/plot_cache/gbstyle.hires.png differ
diff --git a/doc/plot_cache/gbstyle.pdf b/doc/plot_cache/gbstyle.pdf
index 98f6a28c1..31ec83e2c 100755
Binary files a/doc/plot_cache/gbstyle.pdf and b/doc/plot_cache/gbstyle.pdf differ
diff --git a/doc/plot_cache/gbstyle.png b/doc/plot_cache/gbstyle.png
index 71d6c4858..7b588f9c5 100755
Binary files a/doc/plot_cache/gbstyle.png and b/doc/plot_cache/gbstyle.png differ
diff --git a/doc/plot_cache/ic_b_airy_class.hires.png b/doc/plot_cache/ic_b_airy_class.hires.png
index 0b97fde99..63d3befc8 100755
Binary files a/doc/plot_cache/ic_b_airy_class.hires.png and b/doc/plot_cache/ic_b_airy_class.hires.png differ
diff --git a/doc/plot_cache/ic_b_airy_class.pdf b/doc/plot_cache/ic_b_airy_class.pdf
index be93e9e6d..74ff7ad87 100755
Binary files a/doc/plot_cache/ic_b_airy_class.pdf and b/doc/plot_cache/ic_b_airy_class.pdf differ
diff --git a/doc/plot_cache/ic_b_airy_class.png b/doc/plot_cache/ic_b_airy_class.png
index fccaa7ba2..005bb1eb9 100755
Binary files a/doc/plot_cache/ic_b_airy_class.png and b/doc/plot_cache/ic_b_airy_class.png differ
diff --git a/doc/plot_cache/inset.hires.png b/doc/plot_cache/inset.hires.png
index a4c16ae37..da289cc0d 100755
Binary files a/doc/plot_cache/inset.hires.png and b/doc/plot_cache/inset.hires.png differ
diff --git a/doc/plot_cache/inset.pdf b/doc/plot_cache/inset.pdf
index fd0101afa..7b3505c2a 100755
Binary files a/doc/plot_cache/inset.pdf and b/doc/plot_cache/inset.pdf differ
diff --git a/doc/plot_cache/inset.png b/doc/plot_cache/inset.png
index b44100a1c..c56e56e47 100755
Binary files a/doc/plot_cache/inset.png and b/doc/plot_cache/inset.png differ
diff --git a/doc/plot_cache/jtbstyle.hires.png b/doc/plot_cache/jtbstyle.hires.png
index 93f281935..39d883000 100755
Binary files a/doc/plot_cache/jtbstyle.hires.png and b/doc/plot_cache/jtbstyle.hires.png differ
diff --git a/doc/plot_cache/jtbstyle.pdf b/doc/plot_cache/jtbstyle.pdf
index 9808916fa..bdacb8c0d 100755
Binary files a/doc/plot_cache/jtbstyle.pdf and b/doc/plot_cache/jtbstyle.pdf differ
diff --git a/doc/plot_cache/jtbstyle.png b/doc/plot_cache/jtbstyle.png
index bead362fa..3f88a0268 100755
Binary files a/doc/plot_cache/jtbstyle.png and b/doc/plot_cache/jtbstyle.png differ
diff --git a/doc/plot_cache/kittel-class.hires.png b/doc/plot_cache/kittel-class.hires.png
index c7a19fef1..9a0da6591 100755
Binary files a/doc/plot_cache/kittel-class.hires.png and b/doc/plot_cache/kittel-class.hires.png differ
diff --git a/doc/plot_cache/kittel-class.pdf b/doc/plot_cache/kittel-class.pdf
index 7cde029ce..31cee92a2 100755
Binary files a/doc/plot_cache/kittel-class.pdf and b/doc/plot_cache/kittel-class.pdf differ
diff --git a/doc/plot_cache/kittel-class.png b/doc/plot_cache/kittel-class.png
index 4270401fd..2d154df57 100755
Binary files a/doc/plot_cache/kittel-class.png and b/doc/plot_cache/kittel-class.png differ
diff --git a/doc/plot_cache/langevin-class.hires.png b/doc/plot_cache/langevin-class.hires.png
index 840168ee3..73dae35d8 100755
Binary files a/doc/plot_cache/langevin-class.hires.png and b/doc/plot_cache/langevin-class.hires.png differ
diff --git a/doc/plot_cache/langevin-class.pdf b/doc/plot_cache/langevin-class.pdf
index db59ec9df..8a18495a6 100755
Binary files a/doc/plot_cache/langevin-class.pdf and b/doc/plot_cache/langevin-class.pdf differ
diff --git a/doc/plot_cache/langevin-class.png b/doc/plot_cache/langevin-class.png
index da4f331fe..4d2d288e9 100755
Binary files a/doc/plot_cache/langevin-class.png and b/doc/plot_cache/langevin-class.png differ
diff --git a/doc/plot_cache/lmfit2.hires.png b/doc/plot_cache/lmfit2.hires.png
index 710cb36d3..bb0e3cd88 100755
Binary files a/doc/plot_cache/lmfit2.hires.png and b/doc/plot_cache/lmfit2.hires.png differ
diff --git a/doc/plot_cache/lmfit2.pdf b/doc/plot_cache/lmfit2.pdf
index 9a2245f9e..109a93dd6 100755
Binary files a/doc/plot_cache/lmfit2.pdf and b/doc/plot_cache/lmfit2.pdf differ
diff --git a/doc/plot_cache/lmfit2.png b/doc/plot_cache/lmfit2.png
index fd8f12389..f368a28d5 100755
Binary files a/doc/plot_cache/lmfit2.png and b/doc/plot_cache/lmfit2.png differ
diff --git a/doc/plot_cache/lmfit_example.hires.png b/doc/plot_cache/lmfit_example.hires.png
index 2ccac1979..8d828a52c 100755
Binary files a/doc/plot_cache/lmfit_example.hires.png and b/doc/plot_cache/lmfit_example.hires.png differ
diff --git a/doc/plot_cache/lmfit_example.pdf b/doc/plot_cache/lmfit_example.pdf
index 22de19fae..0de5e7418 100755
Binary files a/doc/plot_cache/lmfit_example.pdf and b/doc/plot_cache/lmfit_example.pdf differ
diff --git a/doc/plot_cache/lmfit_example.png b/doc/plot_cache/lmfit_example.png
index b15ebb493..ddd516006 100755
Binary files a/doc/plot_cache/lmfit_example.png and b/doc/plot_cache/lmfit_example.png differ
diff --git a/doc/plot_cache/lorentzian_diff_class.hires.png b/doc/plot_cache/lorentzian_diff_class.hires.png
index 3d2e8b110..3c1b58433 100755
Binary files a/doc/plot_cache/lorentzian_diff_class.hires.png and b/doc/plot_cache/lorentzian_diff_class.hires.png differ
diff --git a/doc/plot_cache/lorentzian_diff_class.pdf b/doc/plot_cache/lorentzian_diff_class.pdf
index 5a058947c..df5cb1416 100755
Binary files a/doc/plot_cache/lorentzian_diff_class.pdf and b/doc/plot_cache/lorentzian_diff_class.pdf differ
diff --git a/doc/plot_cache/lorentzian_diff_class.png b/doc/plot_cache/lorentzian_diff_class.png
index 75564d25b..f7a2fe763 100755
Binary files a/doc/plot_cache/lorentzian_diff_class.png and b/doc/plot_cache/lorentzian_diff_class.png differ
diff --git a/doc/plot_cache/make_model.hires.png b/doc/plot_cache/make_model.hires.png
index d947933a8..16525085a 100755
Binary files a/doc/plot_cache/make_model.hires.png and b/doc/plot_cache/make_model.hires.png differ
diff --git a/doc/plot_cache/make_model.pdf b/doc/plot_cache/make_model.pdf
index 28a1d9811..2575dfdb0 100755
Binary files a/doc/plot_cache/make_model.pdf and b/doc/plot_cache/make_model.pdf differ
diff --git a/doc/plot_cache/make_model.png b/doc/plot_cache/make_model.png
index 54a0ce9cb..0b54b20a9 100755
Binary files a/doc/plot_cache/make_model.png and b/doc/plot_cache/make_model.png differ
diff --git a/doc/plot_cache/matrix.hires.png b/doc/plot_cache/matrix.hires.png
index 6e26ab7c3..4e53020cc 100755
Binary files a/doc/plot_cache/matrix.hires.png and b/doc/plot_cache/matrix.hires.png differ
diff --git a/doc/plot_cache/matrix.pdf b/doc/plot_cache/matrix.pdf
index ae2f3c981..204184858 100755
Binary files a/doc/plot_cache/matrix.pdf and b/doc/plot_cache/matrix.pdf differ
diff --git a/doc/plot_cache/matrix.png b/doc/plot_cache/matrix.png
index afc50eab1..7ac9d3876 100755
Binary files a/doc/plot_cache/matrix.png and b/doc/plot_cache/matrix.png differ
diff --git a/doc/plot_cache/modarrhenius-class.hires.png b/doc/plot_cache/modarrhenius-class.hires.png
index cc61048e1..d1ed4ea1a 100755
Binary files a/doc/plot_cache/modarrhenius-class.hires.png and b/doc/plot_cache/modarrhenius-class.hires.png differ
diff --git a/doc/plot_cache/modarrhenius-class.pdf b/doc/plot_cache/modarrhenius-class.pdf
index d9428db8d..d48019220 100755
Binary files a/doc/plot_cache/modarrhenius-class.pdf and b/doc/plot_cache/modarrhenius-class.pdf differ
diff --git a/doc/plot_cache/modarrhenius-class.png b/doc/plot_cache/modarrhenius-class.png
index 72ef0dc52..bbd27331c 100755
Binary files a/doc/plot_cache/modarrhenius-class.png and b/doc/plot_cache/modarrhenius-class.png differ
diff --git a/doc/plot_cache/multiple_y.hires.png b/doc/plot_cache/multiple_y.hires.png
index 6f30ca797..1413bbfdc 100755
Binary files a/doc/plot_cache/multiple_y.hires.png and b/doc/plot_cache/multiple_y.hires.png differ
diff --git a/doc/plot_cache/multiple_y.pdf b/doc/plot_cache/multiple_y.pdf
index 7e0325cff..f64a58776 100755
Binary files a/doc/plot_cache/multiple_y.pdf and b/doc/plot_cache/multiple_y.pdf differ
diff --git a/doc/plot_cache/multiple_y.png b/doc/plot_cache/multiple_y.png
index a31c1fb53..0def0147d 100755
Binary files a/doc/plot_cache/multiple_y.png and b/doc/plot_cache/multiple_y.png differ
diff --git a/doc/plot_cache/nDimarrhenius-class.hires.png b/doc/plot_cache/nDimarrhenius-class.hires.png
index 74f54f514..6e53cb94d 100755
Binary files a/doc/plot_cache/nDimarrhenius-class.hires.png and b/doc/plot_cache/nDimarrhenius-class.hires.png differ
diff --git a/doc/plot_cache/nDimarrhenius-class.pdf b/doc/plot_cache/nDimarrhenius-class.pdf
index bcbdfd4ea..40d22928f 100755
Binary files a/doc/plot_cache/nDimarrhenius-class.pdf and b/doc/plot_cache/nDimarrhenius-class.pdf differ
diff --git a/doc/plot_cache/nDimarrhenius-class.png b/doc/plot_cache/nDimarrhenius-class.png
index 05353794c..78b5333c3 100755
Binary files a/doc/plot_cache/nDimarrhenius-class.png and b/doc/plot_cache/nDimarrhenius-class.png differ
diff --git a/doc/plot_cache/odr_demo.hires.png b/doc/plot_cache/odr_demo.hires.png
index 249c3a858..1f8c7dd95 100755
Binary files a/doc/plot_cache/odr_demo.hires.png and b/doc/plot_cache/odr_demo.hires.png differ
diff --git a/doc/plot_cache/odr_demo.pdf b/doc/plot_cache/odr_demo.pdf
index 9ee341556..9af4c5a0d 100755
Binary files a/doc/plot_cache/odr_demo.pdf and b/doc/plot_cache/odr_demo.pdf differ
diff --git a/doc/plot_cache/odr_demo.png b/doc/plot_cache/odr_demo.png
index 0a2f47002..f301fa619 100755
Binary files a/doc/plot_cache/odr_demo.png and b/doc/plot_cache/odr_demo.png differ
diff --git a/doc/plot_cache/odrfit1.hires.png b/doc/plot_cache/odrfit1.hires.png
index 559696766..126a8e6c3 100755
Binary files a/doc/plot_cache/odrfit1.hires.png and b/doc/plot_cache/odrfit1.hires.png differ
diff --git a/doc/plot_cache/odrfit1.pdf b/doc/plot_cache/odrfit1.pdf
index 8187c17ff..acd7e9880 100755
Binary files a/doc/plot_cache/odrfit1.pdf and b/doc/plot_cache/odrfit1.pdf differ
diff --git a/doc/plot_cache/odrfit1.png b/doc/plot_cache/odrfit1.png
index 579ef0f1c..c99bad256 100755
Binary files a/doc/plot_cache/odrfit1.png and b/doc/plot_cache/odrfit1.png differ
diff --git a/doc/plot_cache/odrfit2.hires.png b/doc/plot_cache/odrfit2.hires.png
index 559696766..126a8e6c3 100755
Binary files a/doc/plot_cache/odrfit2.hires.png and b/doc/plot_cache/odrfit2.hires.png differ
diff --git a/doc/plot_cache/odrfit2.pdf b/doc/plot_cache/odrfit2.pdf
index adb6e0a8c..e2de39acc 100755
Binary files a/doc/plot_cache/odrfit2.pdf and b/doc/plot_cache/odrfit2.pdf differ
diff --git a/doc/plot_cache/odrfit2.png b/doc/plot_cache/odrfit2.png
index 579ef0f1c..c99bad256 100755
Binary files a/doc/plot_cache/odrfit2.png and b/doc/plot_cache/odrfit2.png differ
diff --git a/doc/plot_cache/outlier.hires.png b/doc/plot_cache/outlier.hires.png
index 285f291df..86f855e96 100755
Binary files a/doc/plot_cache/outlier.hires.png and b/doc/plot_cache/outlier.hires.png differ
diff --git a/doc/plot_cache/outlier.pdf b/doc/plot_cache/outlier.pdf
index 6000697a8..a06069680 100755
Binary files a/doc/plot_cache/outlier.pdf and b/doc/plot_cache/outlier.pdf differ
diff --git a/doc/plot_cache/outlier.png b/doc/plot_cache/outlier.png
index 89e14e2e5..56a7ad748 100755
Binary files a/doc/plot_cache/outlier.png and b/doc/plot_cache/outlier.png differ
diff --git a/doc/plot_cache/panels.hires.png b/doc/plot_cache/panels.hires.png
index dace9f364..a1f5d56ed 100755
Binary files a/doc/plot_cache/panels.hires.png and b/doc/plot_cache/panels.hires.png differ
diff --git a/doc/plot_cache/panels.pdf b/doc/plot_cache/panels.pdf
index cf1d3b9ce..8dd66a958 100755
Binary files a/doc/plot_cache/panels.pdf and b/doc/plot_cache/panels.pdf differ
diff --git a/doc/plot_cache/panels.png b/doc/plot_cache/panels.png
index 247867149..ce5caac90 100755
Binary files a/doc/plot_cache/panels.png and b/doc/plot_cache/panels.png differ
diff --git a/doc/plot_cache/peaks.hires.png b/doc/plot_cache/peaks.hires.png
index 2c3a9a6e0..93795ec92 100755
Binary files a/doc/plot_cache/peaks.hires.png and b/doc/plot_cache/peaks.hires.png differ
diff --git a/doc/plot_cache/peaks.pdf b/doc/plot_cache/peaks.pdf
index d80e23dc6..11556e7b6 100755
Binary files a/doc/plot_cache/peaks.pdf and b/doc/plot_cache/peaks.pdf differ
diff --git a/doc/plot_cache/peaks.png b/doc/plot_cache/peaks.png
index f328ca457..2bdc006c2 100755
Binary files a/doc/plot_cache/peaks.png and b/doc/plot_cache/peaks.png differ
diff --git a/doc/plot_cache/powerlaw-class.hires.png b/doc/plot_cache/powerlaw-class.hires.png
index ba4c57471..a357e10f7 100755
Binary files a/doc/plot_cache/powerlaw-class.hires.png and b/doc/plot_cache/powerlaw-class.hires.png differ
diff --git a/doc/plot_cache/powerlaw-class.pdf b/doc/plot_cache/powerlaw-class.pdf
index 250a90fb9..5c5de4cb8 100755
Binary files a/doc/plot_cache/powerlaw-class.pdf and b/doc/plot_cache/powerlaw-class.pdf differ
diff --git a/doc/plot_cache/powerlaw-class.png b/doc/plot_cache/powerlaw-class.png
index c42e0e736..ff83668f2 100755
Binary files a/doc/plot_cache/powerlaw-class.png and b/doc/plot_cache/powerlaw-class.png differ
diff --git a/doc/plot_cache/prbstyle.hires.png b/doc/plot_cache/prbstyle.hires.png
index d6494df55..bb6e89909 100755
Binary files a/doc/plot_cache/prbstyle.hires.png and b/doc/plot_cache/prbstyle.hires.png differ
diff --git a/doc/plot_cache/prbstyle.pdf b/doc/plot_cache/prbstyle.pdf
index c04bbfbfd..32a8b0153 100755
Binary files a/doc/plot_cache/prbstyle.pdf and b/doc/plot_cache/prbstyle.pdf differ
diff --git a/doc/plot_cache/prbstyle.png b/doc/plot_cache/prbstyle.png
index 529176504..345a3d501 100755
Binary files a/doc/plot_cache/prbstyle.png and b/doc/plot_cache/prbstyle.png differ
diff --git a/doc/plot_cache/quadratic-class.hires.png b/doc/plot_cache/quadratic-class.hires.png
index 05974e536..a41cd5a4b 100755
Binary files a/doc/plot_cache/quadratic-class.hires.png and b/doc/plot_cache/quadratic-class.hires.png differ
diff --git a/doc/plot_cache/quadratic-class.pdf b/doc/plot_cache/quadratic-class.pdf
index 49099bce2..f6cec5fe5 100755
Binary files a/doc/plot_cache/quadratic-class.pdf and b/doc/plot_cache/quadratic-class.pdf differ
diff --git a/doc/plot_cache/quadratic-class.png b/doc/plot_cache/quadratic-class.png
index ff91acc4d..1575e28f1 100755
Binary files a/doc/plot_cache/quadratic-class.png and b/doc/plot_cache/quadratic-class.png differ
diff --git a/doc/plot_cache/rsj_noiseless_class.hires.png b/doc/plot_cache/rsj_noiseless_class.hires.png
index c8c08ef3f..5b9e99fb1 100755
Binary files a/doc/plot_cache/rsj_noiseless_class.hires.png and b/doc/plot_cache/rsj_noiseless_class.hires.png differ
diff --git a/doc/plot_cache/rsj_noiseless_class.pdf b/doc/plot_cache/rsj_noiseless_class.pdf
index a941e1784..3b9df2b30 100755
Binary files a/doc/plot_cache/rsj_noiseless_class.pdf and b/doc/plot_cache/rsj_noiseless_class.pdf differ
diff --git a/doc/plot_cache/rsj_noiseless_class.png b/doc/plot_cache/rsj_noiseless_class.png
index b2c35e5fe..df9862fa9 100755
Binary files a/doc/plot_cache/rsj_noiseless_class.png and b/doc/plot_cache/rsj_noiseless_class.png differ
diff --git a/doc/plot_cache/rsj_simple_class.hires.png b/doc/plot_cache/rsj_simple_class.hires.png
index c8c08ef3f..5b9e99fb1 100755
Binary files a/doc/plot_cache/rsj_simple_class.hires.png and b/doc/plot_cache/rsj_simple_class.hires.png differ
diff --git a/doc/plot_cache/rsj_simple_class.pdf b/doc/plot_cache/rsj_simple_class.pdf
index a9f901458..d20d29ef2 100755
Binary files a/doc/plot_cache/rsj_simple_class.pdf and b/doc/plot_cache/rsj_simple_class.pdf differ
diff --git a/doc/plot_cache/rsj_simple_class.png b/doc/plot_cache/rsj_simple_class.png
index b2c35e5fe..df9862fa9 100755
Binary files a/doc/plot_cache/rsj_simple_class.png and b/doc/plot_cache/rsj_simple_class.png differ
diff --git a/doc/plot_cache/scale.hires.png b/doc/plot_cache/scale.hires.png
index 097f48c07..94a0723b9 100755
Binary files a/doc/plot_cache/scale.hires.png and b/doc/plot_cache/scale.hires.png differ
diff --git a/doc/plot_cache/scale.pdf b/doc/plot_cache/scale.pdf
index 9cd386a3b..0bd4896ec 100755
Binary files a/doc/plot_cache/scale.pdf and b/doc/plot_cache/scale.pdf differ
diff --git a/doc/plot_cache/scale.png b/doc/plot_cache/scale.png
index cd78d1dbe..b9b958b80 100755
Binary files a/doc/plot_cache/scale.png and b/doc/plot_cache/scale.png differ
diff --git a/doc/plot_cache/seabornstyle.hires.png b/doc/plot_cache/seabornstyle.hires.png
index 0a0f1d93d..531e1637f 100755
Binary files a/doc/plot_cache/seabornstyle.hires.png and b/doc/plot_cache/seabornstyle.hires.png differ
diff --git a/doc/plot_cache/seabornstyle.pdf b/doc/plot_cache/seabornstyle.pdf
index f218e122c..83dbad8af 100755
Binary files a/doc/plot_cache/seabornstyle.pdf and b/doc/plot_cache/seabornstyle.pdf differ
diff --git a/doc/plot_cache/seabornstyle.png b/doc/plot_cache/seabornstyle.png
index f7cd48171..38c503eeb 100755
Binary files a/doc/plot_cache/seabornstyle.png and b/doc/plot_cache/seabornstyle.png differ
diff --git a/doc/plot_cache/select.hires.png b/doc/plot_cache/select.hires.png
index eff2c8437..9b28ffdd7 100755
Binary files a/doc/plot_cache/select.hires.png and b/doc/plot_cache/select.hires.png differ
diff --git a/doc/plot_cache/select.pdf b/doc/plot_cache/select.pdf
index a35c6c626..1977b37d3 100755
Binary files a/doc/plot_cache/select.pdf and b/doc/plot_cache/select.pdf differ
diff --git a/doc/plot_cache/select.png b/doc/plot_cache/select.png
index ebb78439f..777284717 100755
Binary files a/doc/plot_cache/select.png and b/doc/plot_cache/select.png differ
diff --git a/doc/plot_cache/simmons-class.hires.png b/doc/plot_cache/simmons-class.hires.png
index 927f4ce44..031a0d52e 100755
Binary files a/doc/plot_cache/simmons-class.hires.png and b/doc/plot_cache/simmons-class.hires.png differ
diff --git a/doc/plot_cache/simmons-class.pdf b/doc/plot_cache/simmons-class.pdf
index be54223e3..23a7cff60 100755
Binary files a/doc/plot_cache/simmons-class.pdf and b/doc/plot_cache/simmons-class.pdf differ
diff --git a/doc/plot_cache/simmons-class.png b/doc/plot_cache/simmons-class.png
index 64d8fdea5..d8e8f1fb8 100755
Binary files a/doc/plot_cache/simmons-class.png and b/doc/plot_cache/simmons-class.png differ
diff --git a/doc/plot_cache/single.hires.png b/doc/plot_cache/single.hires.png
index fa86dcec2..b9d132fef 100755
Binary files a/doc/plot_cache/single.hires.png and b/doc/plot_cache/single.hires.png differ
diff --git a/doc/plot_cache/single.pdf b/doc/plot_cache/single.pdf
index 29ef6e472..aad493f6c 100755
Binary files a/doc/plot_cache/single.pdf and b/doc/plot_cache/single.pdf differ
diff --git a/doc/plot_cache/single.png b/doc/plot_cache/single.png
index 8fb63e673..378b67215 100755
Binary files a/doc/plot_cache/single.png and b/doc/plot_cache/single.png differ
diff --git a/doc/plot_cache/sketchstyle.hires.png b/doc/plot_cache/sketchstyle.hires.png
index 0eb1f5ff9..7c8d724ab 100755
Binary files a/doc/plot_cache/sketchstyle.hires.png and b/doc/plot_cache/sketchstyle.hires.png differ
diff --git a/doc/plot_cache/sketchstyle.pdf b/doc/plot_cache/sketchstyle.pdf
index 92905b11d..ae4e80eae 100755
Binary files a/doc/plot_cache/sketchstyle.pdf and b/doc/plot_cache/sketchstyle.pdf differ
diff --git a/doc/plot_cache/sketchstyle.png b/doc/plot_cache/sketchstyle.png
index 26a38532a..5c86a9a43 100755
Binary files a/doc/plot_cache/sketchstyle.png and b/doc/plot_cache/sketchstyle.png differ
diff --git a/doc/plot_cache/smooth.hires.png b/doc/plot_cache/smooth.hires.png
index 350000a0e..56675fec4 100755
Binary files a/doc/plot_cache/smooth.hires.png and b/doc/plot_cache/smooth.hires.png differ
diff --git a/doc/plot_cache/smooth.pdf b/doc/plot_cache/smooth.pdf
index 8709bd973..45c87a7dd 100755
Binary files a/doc/plot_cache/smooth.pdf and b/doc/plot_cache/smooth.pdf differ
diff --git a/doc/plot_cache/smooth.png b/doc/plot_cache/smooth.png
index 874eb7be4..739135150 100755
Binary files a/doc/plot_cache/smooth.png and b/doc/plot_cache/smooth.png differ
diff --git a/doc/plot_cache/stitch.hires.png b/doc/plot_cache/stitch.hires.png
index fadd7dd4e..b7d9117a3 100755
Binary files a/doc/plot_cache/stitch.hires.png and b/doc/plot_cache/stitch.hires.png differ
diff --git a/doc/plot_cache/stitch.pdf b/doc/plot_cache/stitch.pdf
index 5aa0d7331..1be33a903 100755
Binary files a/doc/plot_cache/stitch.pdf and b/doc/plot_cache/stitch.pdf differ
diff --git a/doc/plot_cache/stitch.png b/doc/plot_cache/stitch.png
index a541d9be6..b987acaec 100755
Binary files a/doc/plot_cache/stitch.png and b/doc/plot_cache/stitch.png differ
diff --git a/doc/plot_cache/strijkers_class.hires.png b/doc/plot_cache/strijkers_class.hires.png
index fa3486a31..e1c83f691 100755
Binary files a/doc/plot_cache/strijkers_class.hires.png and b/doc/plot_cache/strijkers_class.hires.png differ
diff --git a/doc/plot_cache/strijkers_class.pdf b/doc/plot_cache/strijkers_class.pdf
index 9850296bd..fcba0c50a 100755
Binary files a/doc/plot_cache/strijkers_class.pdf and b/doc/plot_cache/strijkers_class.pdf differ
diff --git a/doc/plot_cache/strijkers_class.png b/doc/plot_cache/strijkers_class.png
index 50b3c8847..869a0f61d 100755
Binary files a/doc/plot_cache/strijkers_class.png and b/doc/plot_cache/strijkers_class.png differ
diff --git a/doc/plot_cache/subplots.hires.png b/doc/plot_cache/subplots.hires.png
index eb002ccd8..d6dccc622 100755
Binary files a/doc/plot_cache/subplots.hires.png and b/doc/plot_cache/subplots.hires.png differ
diff --git a/doc/plot_cache/subplots.pdf b/doc/plot_cache/subplots.pdf
index 7b08c92e3..101bca780 100755
Binary files a/doc/plot_cache/subplots.pdf and b/doc/plot_cache/subplots.pdf differ
diff --git a/doc/plot_cache/subplots.png b/doc/plot_cache/subplots.png
index bd96d5c0d..bf4741b4e 100755
Binary files a/doc/plot_cache/subplots.png and b/doc/plot_cache/subplots.png differ
diff --git a/doc/plot_cache/template.hires.png b/doc/plot_cache/template.hires.png
index f0922fa53..d5778aa25 100755
Binary files a/doc/plot_cache/template.hires.png and b/doc/plot_cache/template.hires.png differ
diff --git a/doc/plot_cache/template.pdf b/doc/plot_cache/template.pdf
index bd30e2b5d..42c53ef58 100755
Binary files a/doc/plot_cache/template.pdf and b/doc/plot_cache/template.pdf differ
diff --git a/doc/plot_cache/template.png b/doc/plot_cache/template.png
index dd1a49c51..34b483503 100755
Binary files a/doc/plot_cache/template.png and b/doc/plot_cache/template.png differ
diff --git a/doc/plot_cache/template2.hires.png b/doc/plot_cache/template2.hires.png
index 7d5da135b..7c324b770 100755
Binary files a/doc/plot_cache/template2.hires.png and b/doc/plot_cache/template2.hires.png differ
diff --git a/doc/plot_cache/template2.pdf b/doc/plot_cache/template2.pdf
index 9989b9749..95e23661d 100755
Binary files a/doc/plot_cache/template2.pdf and b/doc/plot_cache/template2.pdf differ
diff --git a/doc/plot_cache/template2.png b/doc/plot_cache/template2.png
index 91323a70c..72787290a 100755
Binary files a/doc/plot_cache/template2.png and b/doc/plot_cache/template2.png differ
diff --git a/doc/plot_cache/vector_field.hires.png b/doc/plot_cache/vector_field.hires.png
index 76257f583..045c88f6c 100755
Binary files a/doc/plot_cache/vector_field.hires.png and b/doc/plot_cache/vector_field.hires.png differ
diff --git a/doc/plot_cache/vector_field.pdf b/doc/plot_cache/vector_field.pdf
index 9f3b7db75..aefcda7cb 100755
Binary files a/doc/plot_cache/vector_field.pdf and b/doc/plot_cache/vector_field.pdf differ
diff --git a/doc/plot_cache/vector_field.png b/doc/plot_cache/vector_field.png
index b923275e7..d47c30202 100755
Binary files a/doc/plot_cache/vector_field.png and b/doc/plot_cache/vector_field.png differ
diff --git a/doc/plot_cache/vft-class.hires.png b/doc/plot_cache/vft-class.hires.png
index 0300af267..ad94d487a 100755
Binary files a/doc/plot_cache/vft-class.hires.png and b/doc/plot_cache/vft-class.hires.png differ
diff --git a/doc/plot_cache/vft-class.pdf b/doc/plot_cache/vft-class.pdf
index 1dd9f0570..97a3cb6f7 100755
Binary files a/doc/plot_cache/vft-class.pdf and b/doc/plot_cache/vft-class.pdf differ
diff --git a/doc/plot_cache/vft-class.png b/doc/plot_cache/vft-class.png
index f9d907c33..1b4bae1e2 100755
Binary files a/doc/plot_cache/vft-class.png and b/doc/plot_cache/vft-class.png differ
diff --git a/doc/plot_cache/voxel2.hires.png b/doc/plot_cache/voxel2.hires.png
index ebfd76413..f603240fd 100755
Binary files a/doc/plot_cache/voxel2.hires.png and b/doc/plot_cache/voxel2.hires.png differ
diff --git a/doc/plot_cache/voxel2.pdf b/doc/plot_cache/voxel2.pdf
index 5be499d4d..43b04109a 100755
Binary files a/doc/plot_cache/voxel2.pdf and b/doc/plot_cache/voxel2.pdf differ
diff --git a/doc/plot_cache/voxel2.png b/doc/plot_cache/voxel2.png
index f108a0c5b..5dff5ab36 100755
Binary files a/doc/plot_cache/voxel2.png and b/doc/plot_cache/voxel2.png differ
diff --git a/doc/plot_cache/voxels.hires.png b/doc/plot_cache/voxels.hires.png
index ebfd76413..f603240fd 100755
Binary files a/doc/plot_cache/voxels.hires.png and b/doc/plot_cache/voxels.hires.png differ
diff --git a/doc/plot_cache/voxels.pdf b/doc/plot_cache/voxels.pdf
index a885da6ca..7d74a8f9b 100755
Binary files a/doc/plot_cache/voxels.pdf and b/doc/plot_cache/voxels.pdf differ
diff --git a/doc/plot_cache/voxels.png b/doc/plot_cache/voxels.png
index f108a0c5b..5dff5ab36 100755
Binary files a/doc/plot_cache/voxels.png and b/doc/plot_cache/voxels.png differ
diff --git a/doc/plot_cache/wlfit.hires.png b/doc/plot_cache/wlfit.hires.png
index 581b4bb51..6dfba9de6 100755
Binary files a/doc/plot_cache/wlfit.hires.png and b/doc/plot_cache/wlfit.hires.png differ
diff --git a/doc/plot_cache/wlfit.pdf b/doc/plot_cache/wlfit.pdf
index 6e5a17c3c..efeccb70c 100755
Binary files a/doc/plot_cache/wlfit.pdf and b/doc/plot_cache/wlfit.pdf differ
diff --git a/doc/plot_cache/wlfit.png b/doc/plot_cache/wlfit.png
index 3ae14f051..9168de72b 100755
Binary files a/doc/plot_cache/wlfit.png and b/doc/plot_cache/wlfit.png differ
diff --git a/doc/readme.rst b/doc/readme.rst
index 203432ab9..8f58e3b8f 100755
--- a/doc/readme.rst
+++ b/doc/readme.rst
@@ -5,7 +5,7 @@
:target: https://coveralls.io/github/stonerlab/Stoner-PythonCode?branch=master
.. image:: https://app.codacy.com/project/badge/Grade/a9069a1567114a22b25d63fd4c50b228
- :target: https://www.codacy.com/gh/stonerlab/Stoner-PythonCode/dashboard?utm_source=github.com&utm_medium=referral&utm_content=stonerlab/Stoner-PythonCode&utm_campaign=Badge_Grade
+ :target: https://app.codacy.com/gh/stonerlab/Stoner-PythonCode/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade
.. image:: https://badge.fury.io/py/Stoner.svg
:target: https://badge.fury.io/py/Stoner
@@ -63,14 +63,18 @@ After installing the current Anaconda version, open a terminal (Mac/Linux) or An
.. code-block:: sh
- conda install -c phygbu Stoner
+ conda install -c phygbu -c conda-forge Stoner
-If you are not using Anaconda python, then pip should also work:
+If (and only if) you are not using Anaconda python, then pip should also work:
.. code-block:: sh
pip install Stoner
+.. warning::
+ The conda packages are generally much better tested than the pip wheels, so we would recommend using
+ conda where possible.
+
This will install the Stoner package and any missing dependencies into your current Python environment. Since the package is under fairly
constant updates, you might want to follow the development with git. The source code, along with example scripts
and some sample data files can be obtained from the github repository: https://github.com/stonerlab/Stoner-PythonCode
@@ -78,11 +82,11 @@ and some sample data files can be obtained from the github repository: https://g
Overview
========
-The main part of the **Stoner** package provides four basic top-level classes that describe:
- - an individual file of experimental data (**Stoner.Data**),
+The main part of the **Stoner** package provides four top-level classes that describe:
+ - an individual file of experimental data (**Stoner.Data**) - somewhat similar to a DataFrame,
- an individual experimental image (**Stoner.ImageFile**),
- - a list (such as a directory tree on disc) of many experimental files (**Stoner.DataFolder**)
- - a list (such as a directory tree on disc) of many image files (**Stoner.ImageFolder**).
+ - a nested list (such as a directory tree on disc) of many experimental files (**Stoner.DataFolder**)
+ - a nested list (such as a directory tree on disc) of many image files (**Stoner.ImageFolder**).
For our research, a typical single experimental data file is essentially a single 2D table of floating point
numbers with associated metadata, usually saved in some ASCII text format. This seems to cover most experiments
@@ -99,6 +103,15 @@ operations to be chained together in one line.
This is a *data-centric* approach - we have some data and we do various operations on it to get to our result. In
contrasr, traditional functional programming thinks in terms of various functions into which you pass data.
+.. note::
+ This is rather similar to pandas DataFrames and the package provides methods to easily convert to and from
+ DataFrames. Unlike a DataFrame, a **Stoner.Data** object maintains a dictionary of additional metadata
+ attached to the dataset (e.g. of instrument settings, experimental ort environmental; conditions
+ when thedata was taken). To assist with exporting to pandas DataFrames, the package will add a custom
+ attrobute handler to pandas DataFrames **DataFrame.metadata** to hold this additional data.
+
+ Unlike Pandas, the **Stoner** package's default is to operate in-place and also to return the object
+ from method calls to facilitate "chaining" of data methods into short single line pipelines.
Data and Friends
----------------
@@ -188,11 +201,11 @@ At the moment the development version is maily broen....
Build Status
~~~~~~~~~~~~
-Version 0.7 onwards are tested using the Travis-CI services with unit test coverage assessed by Coveralls.
+Version 0.7-0.9 were tested using the Travis-CI services with unit test coverage assessed by Coveralls.
-Version 0.9 is tested with Python 2.7, 3.5, 3.6 using the standard unittest module.
+Version 0.9 was tested with Python 2.7, 3.5, 3.6 using the standard unittest module.
-Version 0.10 is tested using **pytest** with Python 3.6-3.9 using a github action.
+Version 0.10 is tested using **pytest** with Python 3.7-3.11 using a github action.
Citing the Stoner Package
@@ -207,6 +220,7 @@ Stable Versions
New Features in 0.10 include:
+ * Support for Python 3.10 and 3.11
* Refactor Stoner.Core.DataFile to move functionality to mixin classes
* Start implementing PEP484 Type hinting
* Support pathlib for paths
@@ -252,12 +266,11 @@ The ancient stable version is 0.7.2. Features of 0.7.2 include
* DataFolder has an options to skip iterating over empty Data files
* Further improvements to :py:attr:`Stoner.Core.DataFile.setas` handline.
-No further relases will be made to 0.7.x.
+No further relases will be made to 0.7.x - 0.9.x
-0.6, 0.7 should work on Python 2.7 and 3.5
-0.8 is also tested on Python 3.6
+Versions 0.6.x and earlier are now pre-historic!
-.. _online documentation: http://stoner-pythoncode.readthedocs.io/en/latest/
+.. _online documentation: http://stoner-pythoncode.readthedocs.io/en/stable/
.. _github repository: http://www.github.com/stonerlab/Stoner-PythonCode/
.. _Dr Gavin Burnell: http://www.stoner.leeds.ac.uk/people/gb
.. _User_Guide: http://stoner-pythoncode.readthedocs.io/en/latest/UserGuide/ugindex.html
diff --git a/doc/samples/3D_plot.py b/doc/samples/3D_plot.py
index 893f2f296..be0f6702f 100755
--- a/doc/samples/3D_plot.py
+++ b/doc/samples/3D_plot.py
@@ -1,4 +1,5 @@
"""3D surface plot example."""
+
# pylint: disable=invalid-name
import numpy as np
import matplotlib.cm
@@ -8,9 +9,7 @@
x, y = np.meshgrid(np.linspace(-2, 2, 100), np.linspace(-2, 2, 100))
x = x.ravel()
y = y.ravel()
-z = np.cos(4 * np.pi * np.sqrt(x**2 + y**2)) * np.exp(
- -np.sqrt(x**2 + y**2)
-)
+z = np.cos(4 * np.pi * np.sqrt(x**2 + y**2)) * np.exp(-np.sqrt(x**2 + y**2))
p = Data()
p = p & x & y & z
diff --git a/doc/samples/Fitting/Arrhenius.py b/doc/samples/Fitting/Arrhenius.py
index fab3119b4..517ba044f 100755
--- a/doc/samples/Fitting/Arrhenius.py
+++ b/doc/samples/Fitting/Arrhenius.py
@@ -1,4 +1,5 @@
"""Example of Arrhenius Fit."""
+
# pylint: disable=invalid-name
from numpy import linspace, ceil, log10, abs as np_abs
from numpy.random import normal
diff --git a/doc/samples/Fitting/BDR.py b/doc/samples/Fitting/BDR.py
index 493b34326..c5929d2b3 100755
--- a/doc/samples/Fitting/BDR.py
+++ b/doc/samples/Fitting/BDR.py
@@ -1,4 +1,5 @@
"""Example of nDimArrhenius Fit."""
+
# pylint: disable=invalid-name
from numpy import linspace, ones_like
from numpy.random import normal
@@ -50,4 +51,3 @@
d.ylabel = "Current"
d.title = "BDR Model test"
-d.tight_layout()
diff --git a/doc/samples/Fitting/FowlerNordheim.py b/doc/samples/Fitting/FowlerNordheim.py
index 29c57f328..1b84aadd6 100755
--- a/doc/samples/Fitting/FowlerNordheim.py
+++ b/doc/samples/Fitting/FowlerNordheim.py
@@ -1,4 +1,5 @@
"""Example of nDimArrhenius Fit."""
+
# pylint: disable=invalid-name
from numpy import linspace, ones_like
from numpy.random import normal
@@ -51,4 +52,3 @@
d.ylabel = "Current"
d.title = "Fowler-Nordheim Model test"
-d.tight_layout()
diff --git a/doc/samples/Fitting/Powerlaw.py b/doc/samples/Fitting/Powerlaw.py
index ef1500dde..4cd5eb523 100755
--- a/doc/samples/Fitting/Powerlaw.py
+++ b/doc/samples/Fitting/Powerlaw.py
@@ -1,4 +1,5 @@
"""Example of PowerLaw Fit."""
+
# pylint: disable=invalid-name
from numpy import linspace
from numpy.random import normal
diff --git a/doc/samples/Fitting/Quadratic.py b/doc/samples/Fitting/Quadratic.py
index 35d5e9794..6ef6481d6 100755
--- a/doc/samples/Fitting/Quadratic.py
+++ b/doc/samples/Fitting/Quadratic.py
@@ -1,4 +1,5 @@
"""Example of Quadratic Fit."""
+
# pylint: disable=invalid-name
from numpy import linspace
from numpy.random import normal
diff --git a/doc/samples/Fitting/Simmons.py b/doc/samples/Fitting/Simmons.py
index 48ea51de1..25680df01 100755
--- a/doc/samples/Fitting/Simmons.py
+++ b/doc/samples/Fitting/Simmons.py
@@ -1,4 +1,5 @@
"""Example of nDimArrhenius Fit."""
+
# pylint: disable=invalid-name
from numpy import linspace, ones_like
from numpy.random import normal
@@ -43,4 +44,3 @@
d.xlabel = "Bias (V)"
d.title = "Simmons Model test"
d.yscale("symlog", linthresh=1e-5)
-d.tight_layout()
diff --git a/doc/samples/Fitting/b_g.py b/doc/samples/Fitting/b_g.py
index 8fa131d26..1d48ebb11 100755
--- a/doc/samples/Fitting/b_g.py
+++ b/doc/samples/Fitting/b_g.py
@@ -1,4 +1,5 @@
"""Test Weak-localisation fitting."""
+
# pylint: disable=invalid-name
from copy import deepcopy
@@ -41,4 +42,3 @@
prefix="BlochGrueneisen",
)
d.title = "Bloch-Grueneisen Fit"
-d.tight_layout()
diff --git a/doc/samples/Fitting/blochlaw.py b/doc/samples/Fitting/blochlaw.py
index 91255e060..35cc79cb9 100755
--- a/doc/samples/Fitting/blochlaw.py
+++ b/doc/samples/Fitting/blochlaw.py
@@ -1,4 +1,5 @@
"""Test langevin fitting."""
+
# pylint: disable=invalid-name
from numpy import linspace, ones_like, where, nan, isnan
from numpy.random import normal
diff --git a/doc/samples/Fitting/f_s.py b/doc/samples/Fitting/f_s.py
index d2760d88e..b3aa9ddf6 100755
--- a/doc/samples/Fitting/f_s.py
+++ b/doc/samples/Fitting/f_s.py
@@ -1,4 +1,5 @@
"""Test Weak-localisation fitting."""
+
# pylint: disable=invalid-name
from numpy import logspace, ones_like, log10
from numpy.random import normal
@@ -43,4 +44,3 @@
prefix="FluchsSondheimer",
)
d.title = "Fluchs-Sondheimer Fit"
-d.tight_layout()
diff --git a/doc/samples/Fitting/kittel.py b/doc/samples/Fitting/kittel.py
index 06a2324d0..5ce9b8fe5 100755
--- a/doc/samples/Fitting/kittel.py
+++ b/doc/samples/Fitting/kittel.py
@@ -1,4 +1,5 @@
"""Test Weak-localisation fitting."""
+
# pylint: disable=invalid-name
from copy import copy
@@ -53,4 +54,3 @@
d.title = "Kittel Fit"
d.fig.gca().xaxis.set_major_formatter(TexEngFormatter())
d.fig.gca().yaxis.set_major_formatter(TexEngFormatter())
-d.tight_layout()
diff --git a/doc/samples/Fitting/langevin.py b/doc/samples/Fitting/langevin.py
index 1f87361f5..1045225a9 100755
--- a/doc/samples/Fitting/langevin.py
+++ b/doc/samples/Fitting/langevin.py
@@ -1,4 +1,5 @@
"""Test langevin fitting."""
+
# pylint: disable=invalid-name
from copy import copy
diff --git a/doc/samples/Fitting/lorentzian.py b/doc/samples/Fitting/lorentzian.py
index f483f3eeb..00cfd142a 100755
--- a/doc/samples/Fitting/lorentzian.py
+++ b/doc/samples/Fitting/lorentzian.py
@@ -1,4 +1,5 @@
"""Test langevin fitting."""
+
# pylint: disable=invalid-name
from copy import copy
diff --git a/doc/samples/Fitting/modArrhenius.py b/doc/samples/Fitting/modArrhenius.py
index 66db19bd3..51f1c426a 100755
--- a/doc/samples/Fitting/modArrhenius.py
+++ b/doc/samples/Fitting/modArrhenius.py
@@ -1,4 +1,5 @@
"""Example of nDimArrhenius Fit."""
+
# pylint: disable=invalid-name
from numpy import linspace
from numpy.random import normal
diff --git a/doc/samples/Fitting/nDimArrhenius.py b/doc/samples/Fitting/nDimArrhenius.py
index 78d4479fc..77cb44275 100755
--- a/doc/samples/Fitting/nDimArrhenius.py
+++ b/doc/samples/Fitting/nDimArrhenius.py
@@ -1,4 +1,5 @@
"""Example of nDimArrhenius Fit."""
+
# pylint: disable=invalid-name
from numpy import linspace
from numpy.random import normal
diff --git a/doc/samples/Fitting/vftEquation.py b/doc/samples/Fitting/vftEquation.py
index 02c76fa81..04f166a49 100755
--- a/doc/samples/Fitting/vftEquation.py
+++ b/doc/samples/Fitting/vftEquation.py
@@ -1,4 +1,5 @@
"""Example of Arrhenius Fit."""
+
# pylint: disable=invalid-name
from numpy import logspace, log10
from numpy.random import normal
diff --git a/doc/samples/Fitting/weak_localisation.py b/doc/samples/Fitting/weak_localisation.py
index fada3ba19..27e06bba3 100755
--- a/doc/samples/Fitting/weak_localisation.py
+++ b/doc/samples/Fitting/weak_localisation.py
@@ -1,4 +1,5 @@
"""Test Weak-localisation fitting."""
+
# pylint: disable=invalid-name
from copy import copy
@@ -39,4 +40,3 @@
prefix="WLfit",
)
d.title = "Weak Localisation Fit"
-d.tight_layout()
diff --git a/doc/samples/Smoothing_Data.py b/doc/samples/Smoothing_Data.py
index d12bed0d3..e528836b6 100755
--- a/doc/samples/Smoothing_Data.py
+++ b/doc/samples/Smoothing_Data.py
@@ -1,4 +1,5 @@
"""Smoothing Data methods example."""
+
# pylint: disable=invalid-name, no-member, not-callable
import matplotlib.pyplot as plt
diff --git a/doc/samples/Vectorfield.py b/doc/samples/Vectorfield.py
index fb4cf1ddf..82cef694a 100755
--- a/doc/samples/Vectorfield.py
+++ b/doc/samples/Vectorfield.py
@@ -1,4 +1,5 @@
"""Create a 2D vector field plot."""
+
# pylint: disable=invalid-name
from os import path
@@ -23,5 +24,3 @@
e.setas = "xy.uvw"
e.plot()
e.title = "3D Vector, 3D Field"
-
-e.tight_layout()
diff --git a/doc/samples/bins.py b/doc/samples/bins.py
index 3767a1bfd..4408509fb 100755
--- a/doc/samples/bins.py
+++ b/doc/samples/bins.py
@@ -1,4 +1,5 @@
"""Re-binning data example."""
+
# pylint: disable=invalid-name
from Stoner import Data
from Stoner.plot.utils import errorfill
@@ -27,4 +28,3 @@
d.xlim = (1, 6)
d.ylim(-100.0, 400)
d.title = "Bin demo" if i == 0 else ""
-d.tight_layout()
diff --git a/doc/samples/channel_math.py b/doc/samples/channel_math.py
index f05223492..7ee01e0de 100755
--- a/doc/samples/channel_math.py
+++ b/doc/samples/channel_math.py
@@ -1,4 +1,5 @@
"""Demonstrate Channel math operations."""
+
# pylint: disable=invalid-name
from numpy import linspace, ones_like, sin, cos, pi
from numpy.random import normal, seed
@@ -7,7 +8,7 @@
from Stoner.plot.utils import errorfill
seed(12345) # Ensure consistent random numbers!
-x = linspace(0, 10 * pi, 101)
+x = linspace(0, 10 * pi, 101) + 1e-9
e = 0.01 * ones_like(x)
y = 0.1 * sin(x) + normal(size=len(x), scale=0.01) + 0.1
e2 = 0.01 * cos(x)
@@ -31,4 +32,10 @@
d.divide(a, b, replace=False)
d.diffsum(a, b, replace=False)
d.setas = "xyeyeyeyeyeyeye"
-d.plot(multiple="panels", plotter=errorfill, color="red", alpha_fill=0.2)
+d.plot(
+ multiple="panels",
+ plotter=errorfill,
+ color="red",
+ alpha_fill=0.2,
+ figsize=(5, 8),
+)
diff --git a/doc/samples/colormap_plot.py b/doc/samples/colormap_plot.py
index 26ff24b83..7b6b12f19 100755
--- a/doc/samples/colormap_plot.py
+++ b/doc/samples/colormap_plot.py
@@ -1,15 +1,15 @@
"""Plot 3d fdata as a colourmap."""
+
# pylint: disable=invalid-name
import numpy as np
+
from Stoner import Data
x, y = np.meshgrid(np.linspace(-2, 2, 100), np.linspace(-2, 2, 100))
x = x.ravel()
y = y.ravel()
-z = np.cos(4 * np.pi * np.sqrt(x**2 + y**2)) * np.exp(
- -np.sqrt(x**2 + y**2)
-)
+z = np.cos(4 * np.pi * np.sqrt(x**2 + y**2)) * np.exp(-np.sqrt(x**2 + y**2))
p = Data(np.column_stack((x, y, z)), column_headers=["X", "Y", "Z"])
p.setas = "xyz"
diff --git a/doc/samples/common_y_plot.py b/doc/samples/common_y_plot.py
index 38383e019..ad32a1723 100755
--- a/doc/samples/common_y_plot.py
+++ b/doc/samples/common_y_plot.py
@@ -1,4 +1,5 @@
"""Plot data on a single y-axis."""
+
# pylint: disable=invalid-name
from Stoner import Data
diff --git a/doc/samples/contour_plot.py b/doc/samples/contour_plot.py
index 57c97002a..fe89ee538 100755
--- a/doc/samples/contour_plot.py
+++ b/doc/samples/contour_plot.py
@@ -1,4 +1,5 @@
"""Plot 3D data on a contour plot."""
+
# pylint: disable=invalid-name
import numpy as np
from Stoner import Data
@@ -6,9 +7,7 @@
x, y = np.meshgrid(np.linspace(-2, 2, 100), np.linspace(-2, 2, 100))
x = x.ravel()
y = y.ravel()
-z = np.cos(4 * np.pi * np.sqrt(x**2 + y**2)) * np.exp(
- -np.sqrt(x**2 + y**2)
-)
+z = np.cos(4 * np.pi * np.sqrt(x**2 + y**2)) * np.exp(-np.sqrt(x**2 + y**2))
p = Data()
p = p & x & y & z
diff --git a/doc/samples/curve_fit_line.py b/doc/samples/curve_fit_line.py
index 1692911c5..96439d8d1 100755
--- a/doc/samples/curve_fit_line.py
+++ b/doc/samples/curve_fit_line.py
@@ -1,4 +1,5 @@
"""USe curve_fit to fit a straight line."""
+
# pylint: disable=invalid-name
from Stoner import Data
diff --git a/doc/samples/curve_fit_plane.py b/doc/samples/curve_fit_plane.py
index 45d369d52..e418c459f 100755
--- a/doc/samples/curve_fit_plane.py
+++ b/doc/samples/curve_fit_plane.py
@@ -1,4 +1,5 @@
"""Use curve_fit to fit a plane to some data."""
+
# pylint: disable=invalid-name
from numpy.random import normal, seed
from numpy import linspace, meshgrid, column_stack, array
@@ -26,13 +27,20 @@ def plane(coord, a, b, c):
)
d.column_headers = ["X", "Y", "Z"]
-d.figure(projection="3d")
-d.plot_xyz(plotter="scatter")
+d.plot_xyz(plotter="scatter", title=None, griddata=False, color="k")
popt, pcov = d.curve_fit(plane, [0, 1], 2, result=True)
-d.setas = "xy.z"
+col = linspace(-10, 10, 128)
+X, Y = meshgrid(col, col)
+Z = plane(array([X, Y]).T, *popt)
+e = Data(
+ column_stack((X.ravel(), Y.ravel(), Z.ravel())),
+ filename="Fitting a Plane",
+ setas="xyz",
+)
+e.column_headers = d.column_headers
-d.plot_xyz(linewidth=0, cmap=cmap.jet)
+e.plot_xyz(linewidth=0, cmap=cmap.jet, alpha=0.5, figure=d.fig)
txt = "$z=c-ax+by$\n"
txt += "\n".join(
@@ -40,4 +48,4 @@ def plane(coord, a, b, c):
)
ax = d.axes[0]
-ax.text(15, 5, -50, txt)
+ax.text(-30, -10, 10, txt)
diff --git a/doc/samples/curvefit_models.py b/doc/samples/curvefit_models.py
index b3c04aaae..0707d82a7 100755
--- a/doc/samples/curvefit_models.py
+++ b/doc/samples/curvefit_models.py
@@ -1,4 +1,5 @@
"""Simple use of lmfit to fit data."""
+
# pylint: disable=invalid-name
from numpy import linspace, random
@@ -33,8 +34,8 @@
)
# Reset labels
d.labels = []
-
# Make nice two panel plot layout
+d.figure(figsize=(7, 5), no_axes=True)
ax = d.subplot2grid((3, 1), (2, 0))
d.setas = "x..y"
d.plot(fmt="g+", label="Fit residuals")
diff --git a/doc/samples/decompose.py b/doc/samples/decompose.py
index 602ce3731..63350ea09 100755
--- a/doc/samples/decompose.py
+++ b/doc/samples/decompose.py
@@ -1,4 +1,5 @@
"""Decompose Into symmetric and antisymmetric parts example."""
+
# pylint: disable=invalid-name
from numpy import linspace, reshape, array
@@ -22,4 +23,3 @@
)
d.ylabel = "Data"
d.title = "Decompose Example"
-d.tight_layout()
diff --git a/doc/samples/differential_evolution_simple.py b/doc/samples/differential_evolution_simple.py
index 1ca8871a4..57394dc3b 100755
--- a/doc/samples/differential_evolution_simple.py
+++ b/doc/samples/differential_evolution_simple.py
@@ -1,4 +1,5 @@
"""Simple use of lmfit to fit data."""
+
# pylint: disable=invalid-name
from numpy import linspace, exp, random
@@ -11,8 +12,6 @@
d = Data(x, y, column_headers=["Time", "Signal"], setas="xy")
-d.plot(fmt="ro") # plot our data
-
func = lambda x, A, B, C: A + B * exp(-x / C)
diff --git a/doc/samples/double_y_plot.py b/doc/samples/double_y_plot.py
index 5fe4da17e..3f4c3fc75 100755
--- a/doc/samples/double_y_plot.py
+++ b/doc/samples/double_y_plot.py
@@ -1,9 +1,9 @@
"""Plot data using two y-axes."""
+
# pylint: disable=invalid-name
from Stoner import Data
p = Data("sample.txt", setas="xyy")
-
p.plot_xy(0, 1, "k-")
p.y2()
p.plot_xy(0, 2, "r-")
diff --git a/doc/samples/extrapolate-demo.py b/doc/samples/extrapolate-demo.py
index f1af8ba5b..c6fffd7f2 100755
--- a/doc/samples/extrapolate-demo.py
+++ b/doc/samples/extrapolate-demo.py
@@ -1,4 +1,5 @@
"""Extrapolate data example."""
+
# pylint: disable=invalid-name
from numpy import linspace, ones_like, column_stack, exp, sqrt
from numpy.random import normal, seed
diff --git a/doc/samples/find_peaks_example.py b/doc/samples/find_peaks_example.py
index 0c8ea30fc..dc1fd7c64 100755
--- a/doc/samples/find_peaks_example.py
+++ b/doc/samples/find_peaks_example.py
@@ -1,4 +1,5 @@
"""Detect peaks in a dataset."""
+
# pylint: disable=invalid-name
from Stoner import Data
from Stoner.analysis.fitting.models.generic import Linear
diff --git a/doc/samples/folder_fit.py b/doc/samples/folder_fit.py
index f88aac5fd..d98c0124e 100755
--- a/doc/samples/folder_fit.py
+++ b/doc/samples/folder_fit.py
@@ -1,4 +1,5 @@
"""Demo of Fitting a directory of files."""
+
# pylint: disable=invalid-name
from os.path import join
from matplotlib.pyplot import figure
diff --git a/doc/samples/image/STXMIMage_Demo.py b/doc/samples/image/STXMIMage_Demo.py
index 08b03951b..a946263c0 100755
--- a/doc/samples/image/STXMIMage_Demo.py
+++ b/doc/samples/image/STXMIMage_Demo.py
@@ -1,4 +1,5 @@
"""Demonstrate STXM Image Processing - G.Burnell Nov. 2017"""
+
# pylint: disable=invalid-name,no-member
from os.path import join, dirname
from types import MethodType
@@ -53,7 +54,7 @@
# Create a profile and plot it
profile = xmcd.profile_line((0, 0), (100, 100))
-profile.figure(figsize=(7, 5), no_axes=True)
+profile.figure(figsize=(7, 6), no_axes=True)
profile.subplot(222)
profile.plot()
profile.title = "XMCD Cross Section"
@@ -120,6 +121,3 @@ def guess(self, data, **kwargs):
profile.subplot(224)
hist.plot(fmt=["b+", "b--", "r-"])
hist.title = "Intensity histogram"
-
-# Tidy up
-plt.tight_layout()
diff --git a/doc/samples/inset_plot.py b/doc/samples/inset_plot.py
index 5e560609d..777b0a9b9 100755
--- a/doc/samples/inset_plot.py
+++ b/doc/samples/inset_plot.py
@@ -1,4 +1,5 @@
"""Add an inset to a plot."""
+
# pylint: disable=invalid-name
from Stoner import Data
diff --git a/doc/samples/joy_division_plot.py b/doc/samples/joy_division_plot.py
index 65f5aadc4..ac008cfdb 100755
--- a/doc/samples/joy_division_plot.py
+++ b/doc/samples/joy_division_plot.py
@@ -11,7 +11,6 @@
class RigakuFolder(DataFolder):
-
"""Quick subclass of DataFolder that knows how to extract multiple files from a single Rigaku file."""
def load_files(self, filename):
diff --git a/doc/samples/lmfit_chi^2_demo.py b/doc/samples/lmfit_chi^2_demo.py
index f3755c224..5d7280abc 100755
--- a/doc/samples/lmfit_chi^2_demo.py
+++ b/doc/samples/lmfit_chi^2_demo.py
@@ -1,4 +1,5 @@
"""Demo of new Stoner.Analysis.AnalyseFile.lmfit."""
+
# pylint: disable=invalid-name
from os.path import join
@@ -12,7 +13,5 @@
model, p0 = cfg_model_from_ini(config, data=d)
fit = d.lmfit(model, p0=p0, result=True, header="Fit", output="data")
-
fit.plot(multiple="panels", capsize=3)
fit.yscale = "log" # Adjust y scale for chi^2
-fit.tight_layout()
diff --git a/doc/samples/lmfit_demo.py b/doc/samples/lmfit_demo.py
index 468c43a22..be699603f 100755
--- a/doc/samples/lmfit_demo.py
+++ b/doc/samples/lmfit_demo.py
@@ -1,4 +1,5 @@
"""Demo of new Stoner.Analysis.AnalyseFile.lmfit."""
+
# pylint: disable=invalid-name
from os.path import join
diff --git a/doc/samples/lmfit_demo_url.py b/doc/samples/lmfit_demo_url.py
index ce3d73684..5da4d4437 100755
--- a/doc/samples/lmfit_demo_url.py
+++ b/doc/samples/lmfit_demo_url.py
@@ -1,4 +1,5 @@
"""Demo of new Stoner.Analysis.AnalyseFile.lmfit."""
+
# pylint: disable=invalid-name
import urllib
import io
diff --git a/doc/samples/lmfit_example.py b/doc/samples/lmfit_example.py
index d6565b0a5..46aad96ab 100755
--- a/doc/samples/lmfit_example.py
+++ b/doc/samples/lmfit_example.py
@@ -1,4 +1,5 @@
"""Example of using lmfit to do a bounded fit."""
+
# pylint: disable=invalid-name
from Stoner import Data
from Stoner.analysis.fitting.models.generic import StretchedExp
diff --git a/doc/samples/lmfit_simple.py b/doc/samples/lmfit_simple.py
index f14f26c8d..c574622b8 100755
--- a/doc/samples/lmfit_simple.py
+++ b/doc/samples/lmfit_simple.py
@@ -1,4 +1,5 @@
"""Simple use of lmfit to fit data."""
+
# pylint: disable=invalid-name
from numpy import linspace, exp, random
diff --git a/doc/samples/matrix_plot.py b/doc/samples/matrix_plot.py
index efa37a81c..fd56d8a8b 100755
--- a/doc/samples/matrix_plot.py
+++ b/doc/samples/matrix_plot.py
@@ -1,13 +1,12 @@
"""Plot data defined on a matrix."""
+
# pylint: disable=invalid-name
import numpy as np
from Stoner import Data
x, y = np.meshgrid(np.linspace(-2, 2, 101), np.linspace(-2, 2, 101))
-z = np.cos(4 * np.pi * np.sqrt(x**2 + y**2)) * np.exp(
- -np.sqrt(x**2 + y**2)
-)
+z = np.cos(4 * np.pi * np.sqrt(x**2 + y**2)) * np.exp(-np.sqrt(x**2 + y**2))
p = Data()
p = p & np.linspace(-2, 2, 101) & z
diff --git a/doc/samples/multiple_panels_plot.py b/doc/samples/multiple_panels_plot.py
index ebd1294a2..f79d3362d 100755
--- a/doc/samples/multiple_panels_plot.py
+++ b/doc/samples/multiple_panels_plot.py
@@ -1,4 +1,5 @@
"""Plot data using multiple sub-plots."""
+
# pylint: disable=invalid-name
from Stoner import Data
@@ -6,4 +7,5 @@
# Quick plot
p.plot(multiple="panels")
# Helps to fix layout !
-p.tight_layout()
+p.set_layout_engine("tight")
+p.draw()
diff --git a/doc/samples/multiple_y2_plot.py b/doc/samples/multiple_y2_plot.py
index 6d22a4e2b..e7516141c 100755
--- a/doc/samples/multiple_y2_plot.py
+++ b/doc/samples/multiple_y2_plot.py
@@ -1,4 +1,5 @@
"""Double y axis plot."""
+
# pylint: disable=invalid-name
from Stoner import Data
diff --git a/doc/samples/odr_demo.py b/doc/samples/odr_demo.py
index b19ed4a72..82682041b 100755
--- a/doc/samples/odr_demo.py
+++ b/doc/samples/odr_demo.py
@@ -1,4 +1,5 @@
"""Demo of new Stoner.Analysis.AnalyseFile.lmfit."""
+
# pylint: disable=invalid-name
from os.path import join
diff --git a/doc/samples/odr_model.py b/doc/samples/odr_model.py
index da2966e59..500429ffb 100755
--- a/doc/samples/odr_model.py
+++ b/doc/samples/odr_model.py
@@ -1,4 +1,5 @@
"""Simple use of lmfit to fit data."""
+
# pylint: disable=invalid-name
from numpy import linspace, exp, random
from scipy.odr import Model as odrModel
diff --git a/doc/samples/odr_simple.py b/doc/samples/odr_simple.py
index c47e2d472..703810b99 100755
--- a/doc/samples/odr_simple.py
+++ b/doc/samples/odr_simple.py
@@ -1,4 +1,5 @@
"""Simple use of lmfit to fit data."""
+
# pylint: disable=invalid-name
from numpy import linspace, exp, random
diff --git a/doc/samples/outlier.py b/doc/samples/outlier.py
index b696fa2f4..289201951 100755
--- a/doc/samples/outlier.py
+++ b/doc/samples/outlier.py
@@ -1,4 +1,5 @@
"""Detect outlying points from a lione."""
+
# pylint: disable=invalid-name
import numpy as np
diff --git a/doc/samples/panel_plot.py b/doc/samples/panel_plot.py
index a7f73e61d..20ba2f032 100755
--- a/doc/samples/panel_plot.py
+++ b/doc/samples/panel_plot.py
@@ -1,4 +1,5 @@
"""Plot multiple y data using separate sub-plots."""
+
# pylint: disable=invalid-name
from Stoner import Data
diff --git a/doc/samples/peaks_example.py b/doc/samples/peaks_example.py
index 3e41b197a..f15447ec3 100755
--- a/doc/samples/peaks_example.py
+++ b/doc/samples/peaks_example.py
@@ -1,4 +1,5 @@
"""Detect peaks in a dataset."""
+
# pylint: disable=invalid-name,unsubscriptable-object
from matplotlib.cm import jet
from numpy import linspace
diff --git a/doc/samples/plotstyles/GBStyle.py b/doc/samples/plotstyles/GBStyle.py
index 84dc7560a..0e57367e1 100755
--- a/doc/samples/plotstyles/GBStyle.py
+++ b/doc/samples/plotstyles/GBStyle.py
@@ -1,4 +1,5 @@
"""Example plot using experimental GBStyle."""
+
# pylint: disable=invalid-name
import os.path as path
diff --git a/doc/samples/plotstyles/JTBStyle.py b/doc/samples/plotstyles/JTBStyle.py
index 597e06f8f..47c824e72 100755
--- a/doc/samples/plotstyles/JTBStyle.py
+++ b/doc/samples/plotstyles/JTBStyle.py
@@ -1,4 +1,5 @@
"""Example plot using Joe Batley's plot style."""
+
# pylint: disable=invalid-name
import os.path as path
diff --git a/doc/samples/plotstyles/PRBStyle.py b/doc/samples/plotstyles/PRBStyle.py
index 60bbe27b1..36b226a01 100755
--- a/doc/samples/plotstyles/PRBStyle.py
+++ b/doc/samples/plotstyles/PRBStyle.py
@@ -1,4 +1,5 @@
"""Example plot using a style similar to Physical Review B."""
+
# pylint: disable=invalid-name
import os.path as path
diff --git a/doc/samples/plotstyles/SeabornStyle.py b/doc/samples/plotstyles/SeabornStyle.py
index ceade5884..60d11385a 100755
--- a/doc/samples/plotstyles/SeabornStyle.py
+++ b/doc/samples/plotstyles/SeabornStyle.py
@@ -1,4 +1,5 @@
"""Example plot style using Seaborn plot styling template."""
+
# pylint: disable=invalid-name
import os.path as path
diff --git a/doc/samples/plotstyles/SketchStyle.py b/doc/samples/plotstyles/SketchStyle.py
index 52c755db6..3dbeac4bd 100755
--- a/doc/samples/plotstyles/SketchStyle.py
+++ b/doc/samples/plotstyles/SketchStyle.py
@@ -1,4 +1,5 @@
"""Example plot in XKCD comic style SketchPlot template."""
+
# pylint: disable=invalid-name
import os.path as path
from Stoner import Data, __home__
diff --git a/doc/samples/plotstyles/customising.py b/doc/samples/plotstyles/customising.py
index 74c1448d0..80db26213 100755
--- a/doc/samples/plotstyles/customising.py
+++ b/doc/samples/plotstyles/customising.py
@@ -1,4 +1,5 @@
"""Example customising a plot using default style."""
+
# pylint: disable=invalid-name
import os.path as path
from cycler import cycler
diff --git a/doc/samples/plotstyles/default.py b/doc/samples/plotstyles/default.py
index 4492b5f26..05411230c 100755
--- a/doc/samples/plotstyles/default.py
+++ b/doc/samples/plotstyles/default.py
@@ -1,4 +1,5 @@
"""Example plot using default style."""
+
# pylint: disable=invalid-name
import os.path as path
diff --git a/doc/samples/scale_curves.py b/doc/samples/scale_curves.py
index ba4a48f81..5400ae60d 100755
--- a/doc/samples/scale_curves.py
+++ b/doc/samples/scale_curves.py
@@ -1,14 +1,17 @@
"""Example of using scale to overlap data."""
+
# pylint: disable=invalid-name, no-member
from numpy import linspace, sin, exp, pi, column_stack
-from numpy.random import normal
+from numpy.random import normal, seed
import matplotlib as mpl
from tabulate import tabulate
from Stoner import Data
+seed(3) # Just fix the random numbers to stop optimizer warnings
mpl.rc("text", usetex=True)
+
x = linspace(0, 10 * pi, 201)
x2 = x * 1.5 + 0.23
y = 10 * exp(-x / (2 * pi)) * sin(x) + normal(size=len(x), scale=0.1)
diff --git a/doc/samples/select_example.py b/doc/samples/select_example.py
index 4c3e2598f..17d6b748f 100755
--- a/doc/samples/select_example.py
+++ b/doc/samples/select_example.py
@@ -1,4 +1,5 @@
"""Example using select method to pick out data."""
+
# pylint: disable=invalid-name
from Stoner import Data
diff --git a/doc/samples/single_plot.py b/doc/samples/single_plot.py
index 0298eaec2..b5ceb8bd0 100755
--- a/doc/samples/single_plot.py
+++ b/doc/samples/single_plot.py
@@ -1,4 +1,5 @@
"""Simple plot in 2 lines."""
+
# pylint: disable=invalid-name
from Stoner import Data
diff --git a/doc/samples/sphere_fit.py b/doc/samples/sphere_fit.py
index 01de9abac..272c5abd7 100755
--- a/doc/samples/sphere_fit.py
+++ b/doc/samples/sphere_fit.py
@@ -1,4 +1,5 @@
"""Fit a sphere with curve_fit."""
+
# pylint: disable=invalid-name, redefined-outer-name
from numpy import (
sin,
@@ -54,6 +55,7 @@ def sphere(coords, a, b, c, r):
d.template.fig_width = 5.2
d.template.fig_height = 5.0 # Square aspect ratio
d.plot_xyz(plotter="scatter", marker=",", griddata=False)
+d.set_box_aspect((1, 1, 1.0)) # Passing through to the current axes
# curve_fit does the hard work
popt, pcov = d.curve_fit(sphere, (0, 1, 2), zeros_like(d.x))
diff --git a/doc/samples/stitch.py b/doc/samples/stitch.py
index 96a2f267c..70c26f8ad 100755
--- a/doc/samples/stitch.py
+++ b/doc/samples/stitch.py
@@ -1,4 +1,5 @@
"""Scale data to stitch it together."""
+
# pylint: disable=invalid-name
import matplotlib.pyplot as plt
diff --git a/doc/samples/stitch_int_overlap.py b/doc/samples/stitch_int_overlap.py
index 7a8f98e11..72766637a 100755
--- a/doc/samples/stitch_int_overlap.py
+++ b/doc/samples/stitch_int_overlap.py
@@ -1,6 +1,7 @@
"""Scale data to stitch it together.
This example demonstrates specifying the overlap as an integer and scaling both curves to match the other."""
+
# pylint: disable=invalid-name
import matplotlib.pyplot as plt
diff --git a/doc/samples/subplot_plot.py b/doc/samples/subplot_plot.py
index a756cae36..d3d762a11 100755
--- a/doc/samples/subplot_plot.py
+++ b/doc/samples/subplot_plot.py
@@ -1,9 +1,8 @@
"""Independent sub plots for multiple y data."""
+
# pylint: disable=invalid-name
from Stoner import Data
p = Data("sample.txt", setas="xyy")
# Quick plot
p.plot(multiple="subplots")
-# Helps to fix layout !
-p.tight_layout()
diff --git a/doc/samples/template.py b/doc/samples/template.py
index 245c18a21..94d01f412 100755
--- a/doc/samples/template.py
+++ b/doc/samples/template.py
@@ -1,4 +1,5 @@
"""Simple plotting with a template."""
+
# pylint: disable=invalid-name, no-member
from cycler import cycler
diff --git a/doc/samples/template2.py b/doc/samples/template2.py
index 7e169c069..445ef8a3e 100755
--- a/doc/samples/template2.py
+++ b/doc/samples/template2.py
@@ -1,4 +1,5 @@
"""Customising a template for plotting."""
+
# pylint: disable=invalid-name
from Stoner import Data
from Stoner.plot.formats import SketchPlot
diff --git a/doc/samples/voxel_plot.py b/doc/samples/voxel_plot.py
index a6a37d8b4..d6e680e48 100755
--- a/doc/samples/voxel_plot.py
+++ b/doc/samples/voxel_plot.py
@@ -1,4 +1,5 @@
"""3D surface plot example."""
+
# pylint: disable=invalid-name
import numpy as np
import matplotlib.cm
@@ -16,4 +17,5 @@
p = Data(x, y, z, u, setas="xyzu", column_headers=["X", "Y", "Z"])
p.plot_voxels(cmap=matplotlib.cm.jet, visible=lambda x, y, z: x - y + z < 2.0)
+p.set_box_aspect((1, 1, 1.0)) # Passing through to the current axes
p.title = "Voxel plot"
diff --git a/scripts/BNL_FileSplitter.py b/scripts/BNL_FileSplitter.py
index dd84660ab..4acbdedd7 100755
--- a/scripts/BNL_FileSplitter.py
+++ b/scripts/BNL_FileSplitter.py
@@ -4,6 +4,7 @@
Will overwrite any files already split but that should be ok for an update.
"""
+
# pylint: disable=invalid-name
import os
import numpy as np
diff --git a/scripts/GenX/stoner.py b/scripts/GenX/stoner.py
index df7fa68d5..9bc2ea99a 100755
--- a/scripts/GenX/stoner.py
+++ b/scripts/GenX/stoner.py
@@ -3,6 +3,7 @@
Hacked up version of the default data loader plugin that uses the Stoner package classes to
read in data files. Supports loading from the Brucker D8
"""
+
# pylint: disable=invalid-name
import numpy as np
import wx
@@ -16,7 +17,6 @@
class Plugin(Template):
-
"""Plugin class from GenX."""
def __init__(self, parent):
@@ -94,7 +94,6 @@ def SettingsDialog(self):
class SettingsDialog(wx.Dialog):
-
"""Plugin Settings dialog class."""
def __init__(self, parent, col_values):
diff --git a/scripts/Kiessig.py b/scripts/Kiessig.py
index 82d1a4310..cdf010f46 100755
--- a/scripts/Kiessig.py
+++ b/scripts/Kiessig.py
@@ -4,6 +4,7 @@
TODO: Implement an error bar on the uncertainity by understanding the significance of the covariance terms
"""
+
# pylint: disable=invalid-name
import sys
from copy import copy
diff --git a/scripts/PCAR-New.py b/scripts/PCAR-New.py
index d923cf6d6..92ad71559 100755
--- a/scripts/PCAR-New.py
+++ b/scripts/PCAR-New.py
@@ -2,6 +2,7 @@
Gavin Burnell g.burnell@leeds.ac.uk
"""
+
# pylint: disable=invalid-name
import configparser as ConfigParser
import pathlib
@@ -13,7 +14,6 @@
class working(Data):
-
"""Utility class to manipulate data and plot it."""
def __init__(self, *args, **kargs):
diff --git a/scripts/VSManalysis_v2.py b/scripts/VSManalysis_v2.py
index f61b87a29..718a6618e 100755
--- a/scripts/VSManalysis_v2.py
+++ b/scripts/VSManalysis_v2.py
@@ -15,6 +15,7 @@
shift algorithm is different to make for better zeroing
"""
+
# pylint: disable=invalid-name, redefined-outer-name
import os
diff --git a/scripts/VSManalysis_v3.py b/scripts/VSManalysis_v3.py
index 5e84b4d1a..314ce34d6 100755
--- a/scripts/VSManalysis_v3.py
+++ b/scripts/VSManalysis_v3.py
@@ -13,7 +13,6 @@
class VSMAnalysis(Data):
-
"""Augment Data with some extra methods."""
def true_m(self):
diff --git a/scripts/XMCD_Reduction.py b/scripts/XMCD_Reduction.py
index fedb15bdc..8eb7a39a8 100755
--- a/scripts/XMCD_Reduction.py
+++ b/scripts/XMCD_Reduction.py
@@ -1,4 +1,5 @@
"""Simple XMCD Data reduction example."""
+
# pylint: disable=invalid-name, redefined-outer-name
import re
import numpy as np
diff --git a/tests/Stoner/Image/test_stack.py b/tests/Stoner/Image/test_stack.py
index a6126c38e..c9c234d84 100755
--- a/tests/Stoner/Image/test_stack.py
+++ b/tests/Stoner/Image/test_stack.py
@@ -41,27 +41,15 @@ def test_ImageStack_align():
assert data.shape == (16, 2), "Slice metadata went a bit funny"
istack2 = selfistack2.clone
- try:
+ with pytest.raises(ValueError):
istack2.align(0, 10, 20, method="imreg_dft")
- except ValueError:
- pass
- else:
- assert False, "stack.align with more than one positional argument failed to raise ValueError"
- try:
+ with pytest.raises(TypeError):
istack2.align(np.zeros(5), method="imreg_dft")
- except TypeError:
- pass
- else:
- assert False, "stack.align with a 1D array failed to raise a TypeError"
- try:
+ with pytest.raises(TypeError):
istack2.align(object, method="imreg_dft")
- except TypeError:
- pass
- else:
- assert False, "stack.align with an object failed to raise a TypeError"
istack2.align(method="imreg_dft")
- assert (istack2[0]["tvec"] == (0.0, 0.0), "stack didn't align to first image")
+ assert istack2[0]["tvec"] == (0.0, 0.0), "stack didn't align to first image"
istack2 = selfistack2.clone
diff --git a/tests/Stoner/analysis/test_filtering.py b/tests/Stoner/analysis/test_filtering.py
index 0eb577157..d76939fd7 100755
--- a/tests/Stoner/analysis/test_filtering.py
+++ b/tests/Stoner/analysis/test_filtering.py
@@ -4,6 +4,9 @@
import pytest
import numpy as np
from Stoner import Data
+import warnings
+
+warnings.filterwarnings("error")
testd = None
np.random.seed(12345)
diff --git a/tests/Stoner/folders/test_Folders.py b/tests/Stoner/folders/test_Folders.py
index ae192a863..7db1e58f1 100755
--- a/tests/Stoner/folders/test_Folders.py
+++ b/tests/Stoner/folders/test_Folders.py
@@ -218,7 +218,7 @@ def test_Properties():
fldr /= "Loaded as"
grps = list(fldr.lsgrp)
skip = 0 if Hyperspy_ok else 1
- assert len(grps) == 27 - skip, f"Length of lsgrp not as expected: {len(grps)} not {27-skip}"
+ assert len(grps) == 26 - skip, f"Length of lsgrp not as expected: {len(grps)} not {27-skip}"
fldr.debug = True
fldr = fldr
assert fldr["XRDFile"][0].debug, "Setting debug on folder failed!"
diff --git a/tests/Stoner/plot/test_plot.py b/tests/Stoner/plot/test_plot.py
index 5fda1e7dc..80774475a 100755
--- a/tests/Stoner/plot/test_plot.py
+++ b/tests/Stoner/plot/test_plot.py
@@ -31,6 +31,8 @@
selfd = Data(path.join(__home__, "..", "sample-data", "New-XRay-Data.dql"))
+warnings.filterwarnings("error")
+
def test_set_no_figs():
global selfd
assert Options.no_figs, "Default setting for no_figs option is incorrect."
@@ -146,7 +148,6 @@ def test_misc_funcs():
selfd.x2()
selfd.setas = ".yx"
selfd.plot()
- selfd.tight_layout()
assert len(selfd.fig_axes) == 2, "Creating a second X axis failed"
plt.close("all")
for i in range(4):
diff --git a/tests/Stoner/test_Util.py b/tests/Stoner/test_Util.py
index c57276341..a05502a78 100755
--- a/tests/Stoner/test_Util.py
+++ b/tests/Stoner/test_Util.py
@@ -12,7 +12,6 @@
pth = path.realpath(path.join(pth, "../../"))
sys.path.insert(0, pth)
-
def is_2tuple(x):
"""Return tru if x is a length two tuple of floats."""
return isinstance(x, tuple) and len(x) == 2 and isinstance(x[0], float) and isinstance(x[1], float)
diff --git a/tests/test-env.yml b/tests/test-env.yml
index bdef1aadc..316d88874 100755
--- a/tests/test-env.yml
+++ b/tests/test-env.yml
@@ -4,31 +4,31 @@ channels:
- phygbu
- conda-forge
dependencies:
- - coveralls >= 3.2.0
- - cycler >= 0.10.0
- - dill >= 0.3.4
- - fabio >= 0.11.0
- - filemagic >= 1.6
- - h5py >= 3.3.0
- - hyperspy >= 1.6.4
- - importlib_resources >= 5.2.0
- - imreg_dft >= 2.0.0
- - image-registration >= 0.2.7
- - lmfit >= 1.0.0
- - matplotlib >= 3.4.3
- - memoization >= 0.3.1
- - multiprocess >= 0.70.12.2
- - nptdms == 1.1.0
- - numpy >= 1.21.2
- - pytest >= 6.2.4
- - pytest-cov >= 2.12.1
- - pytest-forked >= 1.3.0
- - pytest-runner >= 5.3.1
- - pytest-xdist >= 2.3.0
- - python-dateutil >= 2.8.2
- - scikit-image >= 0.18.3
- - scipy >= 1.7.1
- - seaborn >= 0.11.2
- - statsmodels >= 0.13.0
- - tabulate >= 0.8.10
- - urllib3 >= 1.26.7
+ - coveralls
+ - cycler
+ - dill
+ - fabio
+ - filemagic
+ - h5py
+ - hyperspy
+ - importlib_resources
+ - imreg_dft
+ - image-registration
+ - lmfit
+ - matplotlib
+ - memoization
+ - multiprocess
+ - nptdms
+ - numpy
+ - pytest
+ - pytest-cov
+ - pytest-forked
+ - pytest-runner
+ - pytest-xdist
+ - python-dateutil
+ - scikit-image
+ - scipy
+ - seaborn
+ - statsmodels
+ - tabulate
+ - urllib3