Annotations/improve the type annotation for config #1143
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…e type safety and structure
…mproved type safety
…pe order for min_count and min_proportion
…tilize it for improved configuration management
… safety and structure
… LLMEval classes to enhance type safety
…ine SecurityTestsConfig and SafetyTestsConfig TypedDicts for enhanced test configuration
…ormance tests to enhance type safety and configuration structure
…type safety and configuration structure
chakravarthik27
deleted the
annotations/improve-the-type-annotation-for-config
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request includes several changes to the
langtestmodule, focusing on adding type hints, updating method signatures, and enhancing configuration options for various transformation classes. The most important changes include the introduction ofTypedDictfor configuration hints, updates to method signatures to include new types, and improvements to the representation of available tests.Type Hinting and Configuration Enhancements:
langtest/langtest.py: AddedHarnessConfigto the import statement and updated theconfigparameter in the__init__andconfiguremethods to useHarnessConfig. [1] [2] [3]Transformation Classes Updates:
langtest/transform/accuracy.py: AddedTypedDictforTestConfiginBaseAccuracyandLLMEvalclasses to provide configuration hints. [1] [2]langtest/transform/bias.py: IntroducedTypedDictforTestConfiginBaseBiasclass for bias tests configuration.langtest/transform/clinical.py: AddedTypedDictforTestConfiginBaseClincialclass and updatedDemographicBiasalias names. [1] [2]langtest/transform/fairness.py: AddedTypedDictforTestConfiginBaseFairnessand its subclasses to define configuration options for fairness tests. [1] [2] [3] [4] [5] [6] [7]langtest/transform/grammar.py: IntroducedTypedDictforTestConfiginBaseGrammarclass and updatedavailable_testsmethod signature. [1] [2] [3]langtest/transform/legal.py: AddedTypedDictforTestConfiginLegalTestFactoryclass for legal tests configuration. [1] [2]langtest/transform/performance.py: IntroducedTypedDictforTestConfiginBasePerformanceclass and updatedavailable_testsmethod signature. [1] [2] [3]langtest/transform/representation.py: AddedTypedDictforTestConfiginBaseRepresentationand its subclasses to define configuration options for representation tests. [1] [2] [3] [4] [5]