blowtorch 1.3.0

Release Updates

• Added vLLM support for accelerated attention!
• Better loading algorithms and post processing of outputs.
• Included automatic stopping in generation.
• New example files added

v1.2.2

blowtorch 1.2.2

Release Updates

• Added automated context-length detection to stdout so the user is always aware about the current context length.
• Automated context trimming: If the current aggregated context is too long for the context length the auto_trim=True argument (can be called in client init or config) will ensure the most recent context which does not overflow the allowed length. Otherwise users are prone to run into errors like this.
• Added cut_unfinished argument to client. If enabled all outputs will be truncated to the last fulfilled sentence, unfinished sentences will be cutoff.

v1.2.1

blowtorch 1.2.1

Updates

• adding llama.cpp support for cpu and optionally gpu
• more robust code for parameter handling of various backends
• improved setup with tested depenency versions (should work for all OS)