v0.6.0

KerasNLP is adding experimental support for Jax and PyTorch backends on top of the Keras Core library. Read the anouncement, and browse the full library documentation, including how to specify the backend when running your code.

Support for both Jax and PyTorch is still experimental, expect some rough edges and please give us feedback!

Summary

• This release should be equivalent to 0.5.2 with the addition of multi-backend support.
• The following API symbols are currently restricted to the tensorflow backend:
  • keras_nlp.models.DebertaV3*
  • keras_nlp.models.FNet*
  • keras_nlp.metrics
  • keras_nlp.samplers.BeamSampler
  • keras_nlp.samplers.ContrastiveSampler
• Note that there are two ways you can run on top of Tensorflow.
  • If you run your scripts/colab without any changes, KerasNLP will use tf.keras for all layer and modeling implementations. This should be a no-op from previous releases of the library.
  • If you run your scripts/colab with KERAS_BACKEND={jax, torch, tensorflow}, you will be trying the new Keras Core library, using the specified backend. This is a great way to test out the future of the library!
  • Full details on runtime specification is available along with the Keras Core documentation.

What's Changed

• small updates to the release doc by @chenmoneygithub in #1031
• Sampler docstring edit by @abuelnasr0 in #1033
• Fix program crash for id_to_token() method in SentencePieceTokenizer by @abuelnasr0 in #1040
• Update our release process to preview docs before release by @mattdangerw in #1043
• Add Whisper Tokenizer and Audio Feature Extractor by @abheesht17 in #847
• Also strip padding token for opt by @mattdangerw in #1028
• Add regex dep by @mattdangerw in #1044
• Add BartSeq2SeqLM and conditional text generation with BART by @abheesht17 in #974
• Support list/tuple inputs for special tokens in StartEndPacker layer by @abheesht17 in #1045
• Support list/tuple inputs for special tokens in MultiSegmentPacker layer by @abheesht17 in #1046
• Fix a misleading part of our cached MHA docs by @mattdangerw in #1048
• Always pass weight name by kwarg by @mattdangerw in #1053
• Always pass metrics in a list or dict by @mattdangerw in #1054
• Move Defaults to to end of arg docstring and standardise values by @SamuelMarks in #1057
• Fix beam search for BART by @abheesht17 in #1058
• Replace tf.dtype with "dtype" by @mattdangerw in #1059
• Test shapes directly by @mattdangerw in #1064
• Clean up metrics tests by @mattdangerw in #1063
• Remove metrics merge tests by @mattdangerw in #1065
• Fix whisper feature inputs by @mattdangerw in #1069
• Always specify shape when creating variables by @mattdangerw in #1067
• Remove ragged support from position embeddings by @mattdangerw in #1068
• Clean up dtype handling for preprocessing layers by @mattdangerw in #1066
• Add BART finetuned on CNN+DM for summarisation by @abheesht17 in #1060
• Fix saving bug by @mattdangerw in #1073
• Fix t5 forward pass by @mattdangerw in #1082
• Feat/make transformer decoder callable without causal mask by @ferraric in #1083
• Adding GPTNeoXBackbone by @shivance in #1056
• Add a common test case by @mattdangerw in #1095
• Update register_keras_serializable to use saving module by @mattdangerw in #1094
• Don't test tf format by @mattdangerw in #1104
• Add GPTNeoXPreprocessor by @shivance in #1093
• Split layers into layers/modeling & layers/preprocessing by @mattdangerw in #1102
• Fix merge conflict from #1102 by @mattdangerw in #1105
• Add a common base class for generative models by @mattdangerw in #1096
• Add GPTNeoXCausalLMPreprocessor by @shivance in #1106
• Add Whisper Presets by @abheesht17 in #1089
• Refactor RotaryEmbedding and GPTNeoXAttention by @shivance in #1101
• Remove all the secret keys for ci by @mattdangerw in #1126
• Fix publish to pypi action by @mattdangerw in #1127
• Unexport models that are not in the 0.6 release by @mattdangerw in #1125
• Bump the version to 0.6.0 by @mattdangerw in #1128

New Contributors

• @SamuelMarks made their first contribution in #1057
• @ferraric made their first contribution in #1083

Full Changelog: v0.5.2...v0.6.0