Releases: Privex/python-helpers
2.2.0 - Added export_private/public to KeyManager, add Mocker class plus more
The below list is mostly exhaustive list of all core changes in this version. Some small things may have been missed, but the majority
of changes are written below.
New Features / Additions
privex.helpers.crypto.KeyManager
- Added
export_public
andexport_private
allowing you to export the currently loaded public/private key in KeyManager - Added
export_key
which serialises a cryptography PrivateKey / PublicKey - intended for internal use by KeyManager, but
may be useful to others, so it's available as a public class method. - Refactored
generate_keypair
to useexport_key
- Added
identify_algorithm
which returns the string algorithm e.g.'ed25519'
for a given cryptography PublicKey / PrivateKey instance - Added attribute
type_name_map
which maps public/private key cryptography types to their string algorithm name - Improved exception handling in
load_key
ensuring most common key decoding errors raiseInvalidFormat
instead of different package exceptions
- Added
- Added
Mocker
class tohelpers.common
- useful for mocking classes, or used as a makeshift replacement for modules/classes that couldn't be imported - Added
mock_decorator
tohelpers.decorators
- intended to be used withMocker
- it's a decorator which simply... does nothing. purely pass-thru - Added
HAS_CRYPTO
HAS_SETUPPY_COMMON
HAS_SETUPPY_BUMP
andHAS_SETUPPY_COMMANDS
tohelpers.plugin
allowing for easier detection whether certain
modules are available or not (e.g. due to a user not having a certain dependency package installed).
Testing
- Re-wrote parts of the "How to use the unit tests" docs in
tests/__init__.py
- Added PyTest skipping to tests/test_cache.py for redis tests (uses the new
Mocker
to ensure the tests work without PyTest installed) - Added PyTest skipping to tests/test_net.py for dnspython tests (uses
Mocker
just like test_cache) - Created a base class
CryptoBaseClass
for cryptography tests intest_crypto.py
- Moved signing/verification into a helper method
_sign_verify
in this base class
- Moved signing/verification into a helper method
- Added new tests
- Split up KeyManager tests into
TestKeyManagerLoad
,TestKeyManagerGeneration
, andTestKeyManagerSignVerifyEncrypt
- Added unit tests for outputting key pairs to files with
KeyManager.output_keypair
- Added several unit tests for outputting and then loading key pairs from files with
KeyManager.load_keyfile
- Added relatively thorough unit tests for the new
Mocker
class
- Split up KeyManager tests into
Documentation
- Lots of small changes.
- Removed
:noindex:
from a lot of files because it was causing linking to classes/modules in pydoc comments to break.
This unfortunately means Sphinx throws a lot of warnings, but can't find a way to fix this without breaking class/module linking. - Added docs for
Mocker
andmock_decorator
- Re-generated toctree files for various modules including
crypto
andtests
2.1.0 Added human_name, refactored setuppy + more
This release is the official 2.1.0 release (as published to PyPi).
Melded commits from develop branch
Once we merged develop
into master
, two small fix commits were melded into this 2.1.0 commit, to ensure
the published PyPi 2.1.0 release matches this Git release.
-
fix all on dynamically loaded functions
-
forgot to include extras/tests.txt in dev.txt
New features:
-
Added new function
human_name
tohelpers.common
which humanises a function/class name -
Converted
setuppy.py
into a python module folder atsetuppy/
-
Added new distutils extras:
extras/setuppy.txt
to cover all setuppy optional requirementsextras/docs.txt
contains requirements for building the Sphinx documentationextras/dev.txt
is a meta-extra which includes all extra requirements for development viafull
,docs
andtests
-
Created
privex.helpers.setuppy.bump
- Most notably, this module contains a function
bump_version
which uses the packagesemver
to bump a version number.
The version number is extracted from a python package file configured insettings.VERSION_FILE
and after it's updated,
the file containing the version is automatically updated with the new version number.
- Most notably, this module contains a function
-
Created
privex.helpers.setuppy.commands
which contains setup.py / distutils command classes-
BumpCommand
is a command class which allows you to bump the version of a python package, including updating the file containing the version,
simply by running a command such as./setup.py bump --minor
-
ExtrasCommand
is a command class which helps with managing your package'sextra_require
by offering these features:- With no arguments,
./setup.py extras
would output a list of all requirements listed for each extra in yourextras_require
- With the
--save=somefile.txt
argument, the requirements can be outputted into a file - With the
--install
argument, the requirements will be installed withpip
using the python version which is running setup.py - With the
--extra=myextra
argument, only requirements for that individual extra will be listed/saved/installed
- With no arguments,
-
-
Added unit tests for
human_name
intests/test_general.py
-
Created
tests/test_net.py
for network related unit tests
General changes / improvements:
-
Added
__all__
to some modules, to reduce un-necessary objects being imported when you do wildcard imports such asfrom privex.helpers import *
-
Moved
EXTRAS_FOLDER
into helpers.settings for consistency and easier changing -
Moved the various requirements.txt / extras related functions from
setuppy.py
intohelpers.setuppy.common
-
Moved network related tests from
test_general.py
into the newtest_net.py
-
The
setup.py
used for privex-helpers itself now has the commandsbump
andextras
available, which uses the
new command classes fromprivex.helpers.setuppy.commands
-
The file
privex/helpers/__init__.py
now importsprivex.loghelper
inside of a try/except block, so if for some strange reason
privex-loghelper
isn't installed or is broken, then it will fallback to standardlogging.getLogger
instead of just breaking -
Fixed various issues with PyDoc blocks in several files, such as
cache.KeyManager
andcrypto.RedisCache
-
Travis-CI now uses
pip install -U '.[dev]'
instead of installing the requirements filedocs/requirements.txt
and package -
ReadTheDocs now uses
pip install -U '.[dev]'
instead of installing the requirements filedocs/requirements.txt
-
Possibly other small fixes / changes
Documentation related:
-
Listed some of the new
setuppy
functionality in the overview in README.md -
Updated the
docs/requirements.txt
file to include all important requirements -
Updated the
module.rst
autosummary template to include attributes (if possible) -
Updated the
class.rst
autosummary template so inherited attributes / methods are ignored (generated unit test class docs were often really spammy) -
Added detailed documentation for the
privex.helpers.setuppy
module -
Added an autosummary block for
privex.helpers.settings
so that it can correctly show the settings in the navigation -
Fixed up several module/class rst files with
:noindex:
and other minor fixes to reduce the amount of warnings when building the docs -
Re-organised the navigation tree for some modules to make it easier to use
v2.0.0 MAJOR - Added KeyManager class, refactored crypto module, documentation changes, packaging changes and more!
There have been many, many changes since version 1.0, so I've decided it's about time that we finally bump our major version to version 2.0.0
While this release isn't really a breaking change from 1.5.0, there have been so many breaking changes introduced in the 1.x.x series, as well as major new features such as a cache abstraction layer and a cryptography module, it's worth having a clean semver slate with 2.0.0
Docs
- Added documentation for
byteify
andstringify
- Added documentation for the
crypto
module - Added documentation for the
setuppy
module - Updated the custom sphinx CSS to allow for a sensible 5th nesting level in the navigation
- Adjusted the sphinx templates for
autosummary/class.rst
andautosummary/module.rst
- Insert toctree directive automatically into generated files, so sub-components automatically have their RST stubs for navigation generated,
instead of having to manually modify the generate files afterwards.:toctree:
is automatically added for module functions+classes:toctree:
is automatically added for class methods+attributes
- The headings "Methods" and "Attributes" are automatically added for classes, for display in the navigation bar.
- Insert toctree directive automatically into generated files, so sub-components automatically have their RST stubs for navigation generated,
- Added
sphinx_rtd_theme
as an import and in the extension list inconf.py
(the README says to do this, so maybe it will prevent issues in the future?)
Packaging
- Added
extras_require
to setup.py, allowing "extras" to be specified when installing privex-helpers, instead of
having to specify optional dependencies directly in your requirements.txt - Excluded the
tests
module from the package
Unit Tests
- Created
TestKeyManager
class inside oftests/test_crypto.py
which tests the new asymmetric key system- Tests generation of RSA, ECDSA and Ed25519 keys by checking the public/private key length, as well as looking for strings in their output format
- Tests loading of RSA, ECDSA, and Ed25519 keys
- Tests loading invalid keys correctly raises an exception (otherwise how do we know that the previous loading attempts weren't just being ignored?)
- Tests signing and verifying messages for RSA, ECDSA and Ed25519 keys
- Tests encryption and decryption for RSA keys (other algorithms don't support encryption/decryption)
Code
crypto
module is now a folder python module- Moved EncryptHelper into crypto/EncryptHelper.py
- Created crypto/base.py with helper functions such as
is_base64
andauto_b64decode
- Created crypto/KeyManager.py
- Generates Asymmetric keys in RSA, ECDSA and Ed25519 format
- Serialize public/private keys into various formats, default: PEM+PKCS8 for private, OpenSSH for public
- Load public/private keys in most formats, with auto detection
- Ability to load a key from a file
- Automatically interpolate public key when loading a private key
- Ability to output generated keys to private and public file
- Signing messages with RSA, ECDSA, and Ed25519
- Verifying messages with RSA, ECDSA, and Ed25519
- Encrypting messages/data with RSA
- Decrypting messages/data with RSA
- Lots of PyDoc blocks, clearly documenting how everything works
- New
setuppy
module with various packaging related helpers, such as parsing requirements files and handling importing other requirements files - Added
InvalidFormat
exception, used by the crypto module common
module- New
byteify
function - convert a value into bytes if it isn't already bytes - New
stringify
function - decode bytes into a string, if it isn't already a string
- New
v1.5.0 - Added `crypto` module + docs improvements
- Added
crypto
module with newEncryptHelper
class, designed to make using encryption
viacryptography.fernet
painless as possible. - Added docs for
crypto
module - Set up InterSphinx, allowing for cross referencing several different package docs
- Added new exceptions
EncryptionError
andEncryptKeyMissing
- related tocrypto
module - Added several unit tests for the
crypto
module intest_crypto.py
v1.4.0 (BREAKING CHANGES) - Added `inject_items`, tests overhaul, bug fixes and cleanup
Notable Changes
New Features
- Added
common.inject_items
- a small function for injecting a list into another list
at an arbitrary position.
General Improvements/Fixes
- Fixed
net.asn_to_name
so that it correctly raisesKeyError
or returns
'Unknown ASN'
when dnspython raisesNoAnswer
- Added
plugin.HAS_DNSPYTHON
attribute, for checking whether the dnspython dependent
functions/classes were loaded or not - (POTENTIALLY BREAKING CHANGE)
__init__.py
no longer loads thedjango
module,
as it was causing endless issues related to Django not being configured, or
not being ready for accessing certain modules.
The change to __init__.py
loading Django means importing privex.helpers.django
functions/classes via from privex.helpers import xxx
will no longer work.
Any application which imported django functions/classes via privex.helpers
must now use absolute imports, e.g.
from privex.helpers.django import is_database_synchronized
Documentation
- Fixed small typo in
cache.RedisCache
pydoc block - Re-organised Unit Tests section to make it more readable
- Re-named the cache module into "Cache Abstraction Layer", as the module page contains
a lot of hand written documentation about it. - Re-organised the code documentation to be top level
- Added Usage Examples section
- Fleshed out the index page of the docs
Unit Test updates
- Extracted tests from
tests.py
into individual files intests
folder, as tests.py
was close to 500 lines long and growing... - Added unit tests for
net.asn_to_name
- Added unit tests for
common.inject_items
Added new `cache` module to privex.helpers, overhauled docs
TL;DR; Created new module privex.helpers.cache
for easy framework-agnostic caching, plus cleaned up many page titles in the docs
Caching:
- The main
privex.helpers.cache
module exposes acached
attribute (from privex.helpers import cached
) which
acts as a singleton adapter wrapper, so the global cache adapter can easily be switched out without any risk
of other parts of your code using the "old" adapter. - Created
CacheAdapter
which is an abstract base class designed to set the standard for the cache API - Created
MemoryCache
which is a simple caching layer with expiration support that simply stores cache items in memory
using a staticdict
attribute. - Created
RedisCache
which as it sounds, is a cache layer that uses Redis for it's backend. It uses the global Redis fromprivex.helpers.plugin
by default,
however you can pass aredis.Redis
class instance to it's constructor to use a custom instance instead.
Unit testing:
- Created unit tests for
MemoryCache
- Created unit tests for
RedisCache
- based on the unit tests forMemoryCache
to avoid code duplication
Other Updates:
- Added two new functions to
privex.helpers.plugin
configure_redis
for updating the global redis settings and automatically replacing the Redis instancereset_redis
to close the current Redis connection, then delete and re-instantiate the Redis connector class.
- Added two new exceptions
CacheNotFound
andNotConfigured
- Possibly some other small changes
Documentation
- Added very detailed documentation for the new
privex.helpers.cache
module - Added documentation for
CacheAdapter
- Added documentation for
MemoryCache
- Added documentation for
RedisCache
- Fixed some missing documentation entries
- Added slightly customized autosummary templates under
docs/source/_templates
to stop Sphinx using
ridiculously long namespaced methods/attributes for the page titles / navbar. - Regenerated lots of stub files in
docs/source/helpers/
such ascommon/
,exceptions/
,net/
, and others - fixes the long title issue permanently (see previous bullet point)
v1.3.0 - Added `async_sync`, `run_sync` + More
asyncx.py
- Added
async_sync
decorator for easily running async functions/methods from
non-async functions/methods. Useful for unit tests. - Added
run_sync
function, mainly intended for running async functions from the REPL with ease
common.py
- Added
dec_round
for roundingDecimal
objects to arbitrary DP - Added
env_cast
,env_int
, andenv_decimal
- Added
chunked
function for slicing a list into multiple equal sized lists - Added
Dictable
abstract class - eases creating Python 3.7dataclass
objects fromdict
's
as well as converting them back intodict
's
tests.py
- Added new
TestGeneral
test case, for unit testing some of the new functions
Documentation
- Added docs for
asyncx
module - Updated
common
module to include new functions/classes - Fixed docs for
tests
- Various cleanup to reduce warnings from Sphinx
Added `env_bool` function, csv/kval functions now allow separator to be overriden, + more
New functions / code improvements
- Added
env_bool
function tocommon.py
for loading an environment var as a boolean - Added
csvsplit
parameter to parse/env_csv and keyval, to allow changing the item separator from,
- Added
valsplit
parameter to parse/env_keyval, to allow customising the separator between key's and value's from:
Documentation
- Added PyDoc param's / return's for various functions, and fleshed out some others
- Wrapped various docstring values such as
True
and0
with backticks so they display better - Various small formatting improvements to existing docstrings
- Added
PrivexBaseCase
andenv_bool
to the docs
Unit Testing
- Refactored various attributes in
test.py
into the base classPrivexBaseClass
- Added example to the PyDoc in
tests.py
showing how to run tests withpytest
- Wrote new unit tests:
test_kval_custom_clean
- Validates thatparse_keyval
works properly with customvalsplit
/csvsplit
test_kval_custom_spaced
- Validates thatparse_keyval
works properly with space padded values and customvalsplit
/csvsplit
test_env_nonexist_bool
,test_env_bool_true
,test_env_bool_false
- Validate that the newenv_bool
function returns the correct values.
Added rDNS functions, exceptions, and more.
- Added various rDNS functions to
privex.helpers.net
such asip_to_rdns
- Added unit tests to
tests.py
for the rDNS functions - Created new
exceptions.py
with various exception classes - Fixed
setup.py
to use BASE_DIR, avoiding issues if setup.py is ran relatively - Fixed django.py imports
- Cleaned up PyDoc comments for modules and some functions
- Added PyDoc comment to tests.py explaining how to use the unit tests
- Various other small improvements
Initial release
Added the initial versions of the following helper functions/classes:
- Common helpers:
random_str
,empty
,is_true
,is_false
,ErrHelpParser
- Decorators:
retry_on_err
- Django helpers:
handle_error
,is_database_synchronized
,model_to_dict
,to_json
- Net helpers:
asn_to_name
,ip_is_v4
,ip_is_v6
Also added basic usage information in README.md