TensorFlow Model Analysis 0.26.1

Major Features and Improvements

• N/A

Bug fixes and other changes

• Fix support for exporting the UI from a notebook to a standalone HTML page.
• Depends on apache-beam[gcp]>=2.25,!=2.26,<2.29.
• Depends on numpy>=1.16,<1.20.

Breaking changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.30.0

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Fix bug that FeaturesExtractor incorrectly handles RecordBatches that
  have only the raw input column but no other feature columns.

Breaking Changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.29.0

Major Features and Improvements

• Added support for output aggregation.

Bug fixes and other Changes

• For lift metrics, support negative values in the Fairness Indicator UI bar
  chart.
• Make legacy predict extractor also input/output batched extracts.
• Updated to use new compiled_metrics and compiled_loss APIs for keras
  in-graph metric computations.
• Add support for calling model.evaluate on keras models containing custom
  metrics.
• Add CrossSliceMetricComputation metric type.
• Add Lift metrics under addons/fairness.
• Don't add metric config from config.MetricsSpec to baseline model spec by
  default.
• Fix invalid calculations for metrics derived from tf.keras.losses.
• Fixes following bugs related to CrossSlicingSpec based evaluation results.
  • metrics_plots_and_validations_writer was failing while writing cross
    slice comparison results to metrics file.
  • Fairness widget view was not compatible with cross slicing key type.
• Fix support for loading the UI outside of a notebook.
• Depends on absl-py>=0.9,<0.13.
• Depends on tensorflow-metadata>=0.29.0,<0.30.0.
• Depends on tfx-bsl>=0.29.0,<0.30.0.

Breaking Changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.28.0

Major Features and Improvements

• Add a new base computation for binary confusion matrix (other than based on
  calibration histogram). It also provides a sample of examples for the
  confusion matrix.
• Adding two new metrics - Flip Count and Flip Rate to evaluate Counterfactual
  Fairness.

Bug fixes and other Changes

• Fixed division by zero error for diff metrics.
• Depends on apache-beam[gcp]>=2.28,<3.
• Depends on numpy>=1.16,<1.20.
• Depends on tensorflow-metadata>=0.28.0,<0.29.0.
• Depends on tfx-bsl>=0.28.0,<0.29.0.

Breaking Changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.27.0

Major Features and Improvements

• Created tfma.StandardExtracts with helper methods for common keys.
• Updated StandardMetricInputs to extend from the tfma.StandardExtracts.
• Created set of StandardMetricInputsPreprocessors for filtering extracts.
• Introduced a padding_options config to ModelSpec to configure whether
  and how to pad the prediction and label tensors expected by the model's
  metrics.

Bug fixes and other changes

• Fixed issue with metric computation deduplication logic.
• Depends on apache-beam[gcp]>=2.27,<3.
• Depends on pyarrow>=1,<3.
• Depends on tensorflow>=1.15.2,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,<3.
• Depends on tensorflow-metadata>=0.27.0,<0.28.0.
• Depends on tfx-bsl>=0.27.0,<0.28.0.

Breaking changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.26.0

Major Features and Improvements

• Added support for aggregating feature attributions using special metrics
  that extend from tfma.metrics.AttributionMetric (e.g.
  tfma.metrics.TotalAttributions, tfma.metrics.TotalAbsoluteAttributions).
  To use make use of these metrics a custom extractor that add attributions to
  the tfma.Extracts under the key name tfma.ATTRIBUTIONS_KEY must be
  manually created.
• Added support for feature transformations using TFT and other preprocessing
  functions.
• Add support for rubber stamping (first run without a valid baseline model)
  when validating a model. The change threshold is ignored only when the model
  is rubber stamped, otherwise, an error is thrown.

Bug fixes and other changes

• Fix the bug that Fairness Indicator UI metric list won't refresh if the
  input eval result changed.
• Add support for missing_thresholds failure to validations result.
• Updated to set min/max value for precision/recall plot to 0 and 1.
• Fix issue with MinLabelPosition not being sorted by predictions.
• Updated NDCG to ignore non-positive gains.
• Fix bug where an example could be aggregated more than once in a single
  slice if the same slice key were generated from more than one SlicingSpec.
• Add threshold support for confidence interval type metrics based on its
  unsampled_value.
• Depends on apache-beam[gcp]>=2.25,!=2.26.*,<3.
• Depends on tensorflow>=1.15.2,!=2.0.*,!=2.1.*,!=2.2.*,!=2.4.*,<3.
• Depends on tensorflow-metadata>=0.26.0,<0.27.0.
• Depends on tfx-bsl>=0.26.0,<0.27.0.

Breaking changes

• Changed MultiClassConfusionMatrix threshold check to prediction > threshold
  instead of prediction >= threshold.
• Changed default handling of materialize in default_extractors to False.
• Separated tfma.extractors.BatchedInputExtractor into
  tfma.extractors.FeaturesExtractor, tfma.extractors.LabelsExtractor, and
  tfma.extractors.ExampleWeightsExtractor.

Deprecations

• N/A

TensorFlow Model Analysis 0.25.0

Major Features and Improvements

• Added support for reading and writing metrics, plots and validation results
  using Apache Parquet.

• Updated the FI indicator slicing selection UI.

• Fixed the problem that slices are refreshed when user selected a new
  baseline.

• Add support for slicing on ragged and multidimensional data.

• Load TFMA correctly in JupyterLabs even if Facets has loaded first.

• Added support for aggregating metrics using top k values.

• Added support for padding labels and predictions with -1 to align a batch of
  inputs for use in tf-ranking metrics computations.

• Added support for fractional labels.

• Add metric definitions as tooltips in the Fairness Inidicators metric
  selector UI

• Added support for specifying label_key to use with MinLabelPosition metric.

• From this release TFMA will also be hosting nightly packages on
  https://pypi-nightly.tensorflow.org. To install the nightly package use the
  following command:

  pip install -i https://pypi-nightly.tensorflow.org/simple tensorflow-model-analysis
  

  Note: These nightly packages are unstable and breakages are likely to
  happen. The fix could often take a week or more depending on the complexity
  involved for the wheels to be available on the PyPI cloud service. You can
  always use the stable version of TFMA available on PyPI by running the
  command pip install tensorflow-model-analysis .

Bug fixes and other changes

• Fix incorrect calculation with MinLabelPosition when used with weighted
  examples.
• Fix issue with using NDCG metric without binarization settings.
• Fix incorrect computation when example weight is set to zero.
• Depends on apache-beam[gcp]>=2.25,<3.
• Depends on tensorflow-metadata>=0.25.0,<0.26.0.
• Depends on tfx-bsl>=0.25.0,<0.26.0.

Breaking changes

• AggregationOptions are now independent of BinarizeOptions. In order to
  compute AggregationOptions.macro_average or
  AggregationOptions.weighted_macro_average,
  AggregationOptions.class_weights must now be configured. If
  AggregationOptions.class_weights are provided, any missing keys now
  default to 0.0 instead of 1.0.
• In the UI, aggregation based metrics will now be prefixed with 'micro_',
  'macro_', or 'weighted_macro_' depending on the aggregation type.

Deprecations

• tfma.extractors.FeatureExtractor, tfma.extractors.PredictExtractor,
  tfma.extractors.InputExtractor, and
  tfma.evaluators.MetricsAndPlotsEvaluator are deprecated and may be
  replaced with newer versions in upcoming releases.

TensorFlow Model Analysis 0.24.3

Major Features and Improvements

• N/A

Bug fixes and other changes

• Depends on apache-beam[gcp]>=2.24,<3.
• Depends on tfx-bsl>=0.24.1,0.25.

Breaking changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.24.2

Major Features and Improvements

• N/A

Bug fixes and other changes

• Added an extra requirement group all. As a result, barebone TFMA does not
  require tensorflowjs , prompt-toolkit and ipython any more.
• Added an extra requirement group all that specifies all the extra
  dependencies TFMA needs. Use pip install tensorflow-model-analysis[all] to
  pull in those dependencies.

Breaking changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.24.1

Major Features and Improvements

• N/A

Bug fixes and other changes

• Fix Jupyter lab issue with missing data-base-url.

Breaking changes

• N/A

Deprecations

• N/A