Skip to content

Releases: Privex/python-helpers

2.2.0 - Added export_private/public to KeyManager, add Mocker class plus more

04 Nov 12:52
Compare
Choose a tag to compare

The below list is mostly exhaustive list of all core changes in this version. Some small things may have been missed, but the majority
of changes are written below.

New Features / Additions

  • privex.helpers.crypto.KeyManager
    • Added export_public and export_private allowing you to export the currently loaded public/private key in KeyManager
    • Added export_key which serialises a cryptography PrivateKey / PublicKey - intended for internal use by KeyManager, but
      may be useful to others, so it's available as a public class method.
    • Refactored generate_keypair to use export_key
    • Added identify_algorithm which returns the string algorithm e.g. 'ed25519' for a given cryptography PublicKey / PrivateKey instance
    • Added attribute type_name_map which maps public/private key cryptography types to their string algorithm name
    • Improved exception handling in load_key ensuring most common key decoding errors raise InvalidFormat instead of different package exceptions
  • Added Mocker class to helpers.common - useful for mocking classes, or used as a makeshift replacement for modules/classes that couldn't be imported
  • Added mock_decorator to helpers.decorators - intended to be used with Mocker - it's a decorator which simply... does nothing. purely pass-thru
  • Added HAS_CRYPTO HAS_SETUPPY_COMMON HAS_SETUPPY_BUMP and HAS_SETUPPY_COMMANDS to helpers.plugin allowing for easier detection whether certain
    modules are available or not (e.g. due to a user not having a certain dependency package installed).

Testing

  • Re-wrote parts of the "How to use the unit tests" docs in tests/__init__.py
  • Added PyTest skipping to tests/test_cache.py for redis tests (uses the new Mocker to ensure the tests work without PyTest installed)
  • Added PyTest skipping to tests/test_net.py for dnspython tests (uses Mocker just like test_cache)
  • Created a base class CryptoBaseClass for cryptography tests in test_crypto.py
    • Moved signing/verification into a helper method _sign_verify in this base class
  • Added new tests
    • Split up KeyManager tests into TestKeyManagerLoad, TestKeyManagerGeneration, and TestKeyManagerSignVerifyEncrypt
    • Added unit tests for outputting key pairs to files with KeyManager.output_keypair
    • Added several unit tests for outputting and then loading key pairs from files with KeyManager.load_keyfile
    • Added relatively thorough unit tests for the new Mocker class

Documentation

  • Lots of small changes.
  • Removed :noindex: from a lot of files because it was causing linking to classes/modules in pydoc comments to break.
    This unfortunately means Sphinx throws a lot of warnings, but can't find a way to fix this without breaking class/module linking.
  • Added docs for Mocker and mock_decorator
  • Re-generated toctree files for various modules including crypto and tests

2.1.0 Added human_name, refactored setuppy + more

04 Nov 05:43
Compare
Choose a tag to compare

This release is the official 2.1.0 release (as published to PyPi).

Melded commits from develop branch

Once we merged develop into master, two small fix commits were melded into this 2.1.0 commit, to ensure
the published PyPi 2.1.0 release matches this Git release.

  • fix all on dynamically loaded functions

  • forgot to include extras/tests.txt in dev.txt

New features:

  • Added new function human_name to helpers.common which humanises a function/class name

  • Converted setuppy.py into a python module folder at setuppy/

  • Added new distutils extras:

    • extras/setuppy.txt to cover all setuppy optional requirements
    • extras/docs.txt contains requirements for building the Sphinx documentation
    • extras/dev.txt is a meta-extra which includes all extra requirements for development via full, docs and tests
  • Created privex.helpers.setuppy.bump

    • Most notably, this module contains a function bump_version which uses the package semver to bump a version number.
      The version number is extracted from a python package file configured in settings.VERSION_FILE and after it's updated,
      the file containing the version is automatically updated with the new version number.
  • Created privex.helpers.setuppy.commands which contains setup.py / distutils command classes

    • BumpCommand is a command class which allows you to bump the version of a python package, including updating the file containing the version,
      simply by running a command such as ./setup.py bump --minor

    • ExtrasCommand is a command class which helps with managing your package's extra_require by offering these features:

      • With no arguments, ./setup.py extras would output a list of all requirements listed for each extra in your extras_require
      • With the --save=somefile.txt argument, the requirements can be outputted into a file
      • With the --install argument, the requirements will be installed with pip using the python version which is running setup.py
      • With the --extra=myextra argument, only requirements for that individual extra will be listed/saved/installed
  • Added unit tests for human_name in tests/test_general.py

  • Created tests/test_net.py for network related unit tests

General changes / improvements:

  • Added __all__ to some modules, to reduce un-necessary objects being imported when you do wildcard imports such as from privex.helpers import *

  • Moved EXTRAS_FOLDER into helpers.settings for consistency and easier changing

  • Moved the various requirements.txt / extras related functions from setuppy.py into helpers.setuppy.common

  • Moved network related tests from test_general.py into the new test_net.py

  • The setup.py used for privex-helpers itself now has the commands bump and extras available, which uses the
    new command classes from privex.helpers.setuppy.commands

  • The file privex/helpers/__init__.py now imports privex.loghelper inside of a try/except block, so if for some strange reason
    privex-loghelper isn't installed or is broken, then it will fallback to standard logging.getLogger instead of just breaking

  • Fixed various issues with PyDoc blocks in several files, such as cache.KeyManager and crypto.RedisCache

  • Travis-CI now uses pip install -U '.[dev]' instead of installing the requirements file docs/requirements.txt and package

  • ReadTheDocs now uses pip install -U '.[dev]' instead of installing the requirements file docs/requirements.txt

  • Possibly other small fixes / changes

Documentation related:

  • Listed some of the new setuppy functionality in the overview in README.md

  • Updated the docs/requirements.txt file to include all important requirements

  • Updated the module.rst autosummary template to include attributes (if possible)

  • Updated the class.rst autosummary template so inherited attributes / methods are ignored (generated unit test class docs were often really spammy)

  • Added detailed documentation for the privex.helpers.setuppy module

  • Added an autosummary block for privex.helpers.settings so that it can correctly show the settings in the navigation

  • Fixed up several module/class rst files with :noindex: and other minor fixes to reduce the amount of warnings when building the docs

  • Re-organised the navigation tree for some modules to make it easier to use

v2.0.0 MAJOR - Added KeyManager class, refactored crypto module, documentation changes, packaging changes and more!

25 Oct 22:53
Compare
Choose a tag to compare

There have been many, many changes since version 1.0, so I've decided it's about time that we finally bump our major version to version 2.0.0

While this release isn't really a breaking change from 1.5.0, there have been so many breaking changes introduced in the 1.x.x series, as well as major new features such as a cache abstraction layer and a cryptography module, it's worth having a clean semver slate with 2.0.0

Docs

  • Added documentation for byteify and stringify
  • Added documentation for the crypto module
  • Added documentation for the setuppy module
  • Updated the custom sphinx CSS to allow for a sensible 5th nesting level in the navigation
  • Adjusted the sphinx templates for autosummary/class.rst and autosummary/module.rst
    • Insert toctree directive automatically into generated files, so sub-components automatically have their RST stubs for navigation generated,
      instead of having to manually modify the generate files afterwards.
      • :toctree: is automatically added for module functions+classes
      • :toctree: is automatically added for class methods+attributes
    • The headings "Methods" and "Attributes" are automatically added for classes, for display in the navigation bar.
  • Added sphinx_rtd_theme as an import and in the extension list in conf.py (the README says to do this, so maybe it will prevent issues in the future?)

Packaging

  • Added extras_require to setup.py, allowing "extras" to be specified when installing privex-helpers, instead of
    having to specify optional dependencies directly in your requirements.txt
  • Excluded the tests module from the package

Unit Tests

  • Created TestKeyManager class inside of tests/test_crypto.py which tests the new asymmetric key system
    • Tests generation of RSA, ECDSA and Ed25519 keys by checking the public/private key length, as well as looking for strings in their output format
    • Tests loading of RSA, ECDSA, and Ed25519 keys
    • Tests loading invalid keys correctly raises an exception (otherwise how do we know that the previous loading attempts weren't just being ignored?)
    • Tests signing and verifying messages for RSA, ECDSA and Ed25519 keys
    • Tests encryption and decryption for RSA keys (other algorithms don't support encryption/decryption)

Code

  • crypto module is now a folder python module
    • Moved EncryptHelper into crypto/EncryptHelper.py
    • Created crypto/base.py with helper functions such as is_base64 and auto_b64decode
    • Created crypto/KeyManager.py
      • Generates Asymmetric keys in RSA, ECDSA and Ed25519 format
      • Serialize public/private keys into various formats, default: PEM+PKCS8 for private, OpenSSH for public
      • Load public/private keys in most formats, with auto detection
      • Ability to load a key from a file
      • Automatically interpolate public key when loading a private key
      • Ability to output generated keys to private and public file
      • Signing messages with RSA, ECDSA, and Ed25519
      • Verifying messages with RSA, ECDSA, and Ed25519
      • Encrypting messages/data with RSA
      • Decrypting messages/data with RSA
      • Lots of PyDoc blocks, clearly documenting how everything works
  • New setuppy module with various packaging related helpers, such as parsing requirements files and handling importing other requirements files
  • Added InvalidFormat exception, used by the crypto module
  • common module
    • New byteify function - convert a value into bytes if it isn't already bytes
    • New stringify function - decode bytes into a string, if it isn't already a string

v1.5.0 - Added `crypto` module + docs improvements

23 Oct 21:02
Compare
Choose a tag to compare
  • Added crypto module with new EncryptHelper class, designed to make using encryption
    via cryptography.fernet painless as possible.
  • Added docs for crypto module
  • Set up InterSphinx, allowing for cross referencing several different package docs
  • Added new exceptions EncryptionError and EncryptKeyMissing - related to crypto module
  • Added several unit tests for the crypto module in test_crypto.py

v1.4.0 (BREAKING CHANGES) - Added `inject_items`, tests overhaul, bug fixes and cleanup

19 Oct 06:17
Compare
Choose a tag to compare

Notable Changes

New Features

  • Added common.inject_items - a small function for injecting a list into another list
    at an arbitrary position.

General Improvements/Fixes

  • Fixed net.asn_to_name so that it correctly raises KeyError or returns
    'Unknown ASN' when dnspython raises NoAnswer
  • Added plugin.HAS_DNSPYTHON attribute, for checking whether the dnspython dependent
    functions/classes were loaded or not
  • (POTENTIALLY BREAKING CHANGE) __init__.py no longer loads the django module,
    as it was causing endless issues related to Django not being configured, or
    not being ready for accessing certain modules.

The change to __init__.py loading Django means importing privex.helpers.django functions/classes via from privex.helpers import xxx will no longer work.

Any application which imported django functions/classes via privex.helpers must now use absolute imports, e.g.

from privex.helpers.django import is_database_synchronized

Documentation

  • Fixed small typo in cache.RedisCache pydoc block
  • Re-organised Unit Tests section to make it more readable
  • Re-named the cache module into "Cache Abstraction Layer", as the module page contains
    a lot of hand written documentation about it.
  • Re-organised the code documentation to be top level
  • Added Usage Examples section
  • Fleshed out the index page of the docs

Unit Test updates

  • Extracted tests from tests.py into individual files in tests folder, as tests.py
    was close to 500 lines long and growing...
  • Added unit tests for net.asn_to_name
  • Added unit tests for common.inject_items

Added new `cache` module to privex.helpers, overhauled docs

03 Oct 22:26
Compare
Choose a tag to compare

TL;DR; Created new module privex.helpers.cache for easy framework-agnostic caching, plus cleaned up many page titles in the docs

Caching:

  • The main privex.helpers.cache module exposes a cached attribute (from privex.helpers import cached) which
    acts as a singleton adapter wrapper, so the global cache adapter can easily be switched out without any risk
    of other parts of your code using the "old" adapter.
  • Created CacheAdapter which is an abstract base class designed to set the standard for the cache API
  • Created MemoryCache which is a simple caching layer with expiration support that simply stores cache items in memory
    using a static dict attribute.
  • Created RedisCache which as it sounds, is a cache layer that uses Redis for it's backend. It uses the global Redis from privex.helpers.plugin by default,
    however you can pass a redis.Redis class instance to it's constructor to use a custom instance instead.

Unit testing:

  • Created unit tests for MemoryCache
  • Created unit tests for RedisCache - based on the unit tests for MemoryCache to avoid code duplication

Other Updates:

  • Added two new functions to privex.helpers.plugin
    • configure_redis for updating the global redis settings and automatically replacing the Redis instance
    • reset_redis to close the current Redis connection, then delete and re-instantiate the Redis connector class.
  • Added two new exceptions CacheNotFound and NotConfigured
  • Possibly some other small changes

Documentation

  • Added very detailed documentation for the new privex.helpers.cache module
  • Added documentation for CacheAdapter
  • Added documentation for MemoryCache
  • Added documentation for RedisCache
  • Fixed some missing documentation entries
  • Added slightly customized autosummary templates under docs/source/_templates to stop Sphinx using
    ridiculously long namespaced methods/attributes for the page titles / navbar.
  • Regenerated lots of stub files in docs/source/helpers/ such as common/, exceptions/, net/, and others - fixes the long title issue permanently (see previous bullet point)

v1.3.0 - Added `async_sync`, `run_sync` + More

14 Sep 17:31
Compare
Choose a tag to compare

asyncx.py

  • Added async_sync decorator for easily running async functions/methods from
    non-async functions/methods. Useful for unit tests.
  • Added run_sync function, mainly intended for running async functions from the REPL with ease

common.py

  • Added dec_round for rounding Decimal objects to arbitrary DP
  • Added env_cast, env_int, and env_decimal
  • Added chunked function for slicing a list into multiple equal sized lists
  • Added Dictable abstract class - eases creating Python 3.7 dataclass objects from dict's
    as well as converting them back into dict's

tests.py

  • Added new TestGeneral test case, for unit testing some of the new functions

Documentation

  • Added docs for asyncx module
  • Updated common module to include new functions/classes
  • Fixed docs for tests
  • Various cleanup to reduce warnings from Sphinx

Added `env_bool` function, csv/kval functions now allow separator to be overriden, + more

19 Jul 13:03
Compare
Choose a tag to compare

New functions / code improvements

  • Added env_bool function to common.py for loading an environment var as a boolean
  • Added csvsplit parameter to parse/env_csv and keyval, to allow changing the item separator from ,
  • Added valsplit parameter to parse/env_keyval, to allow customising the separator between key's and value's from :

Documentation

  • Added PyDoc param's / return's for various functions, and fleshed out some others
  • Wrapped various docstring values such as True and 0 with backticks so they display better
  • Various small formatting improvements to existing docstrings
  • Added PrivexBaseCase and env_bool to the docs

Unit Testing

  • Refactored various attributes in test.py into the base class PrivexBaseClass
  • Added example to the PyDoc in tests.py showing how to run tests with pytest
  • Wrote new unit tests:
    • test_kval_custom_clean - Validates that parse_keyval works properly with custom valsplit/csvsplit
    • test_kval_custom_spaced - Validates that parse_keyval works properly with space padded values and custom valsplit/csvsplit
    • test_env_nonexist_bool, test_env_bool_true, test_env_bool_false - Validate that the new env_bool function returns the correct values.

Added rDNS functions, exceptions, and more.

19 Jul 13:04
Compare
Choose a tag to compare
  • Added various rDNS functions to privex.helpers.net such as ip_to_rdns
  • Added unit tests to tests.py for the rDNS functions
  • Created new exceptions.py with various exception classes
  • Fixed setup.py to use BASE_DIR, avoiding issues if setup.py is ran relatively
  • Fixed django.py imports
  • Cleaned up PyDoc comments for modules and some functions
  • Added PyDoc comment to tests.py explaining how to use the unit tests
  • Various other small improvements

Initial release

19 Jul 13:06
Compare
Choose a tag to compare

Added the initial versions of the following helper functions/classes:

  • Common helpers: random_str, empty, is_true, is_false, ErrHelpParser
  • Decorators: retry_on_err
  • Django helpers: handle_error, is_database_synchronized, model_to_dict, to_json
  • Net helpers: asn_to_name, ip_is_v4, ip_is_v6

Also added basic usage information in README.md