More LLMs

This release brings two main features:

Improved support for 3rd party LLM providers (Fireworks, Mistral, LiteLLM, etc)
Prompta-provided LLMs to users get started

What's new

screenshot_20240105013118.mp4

Using Mistral as a LLM providers

screenshot_20240105015316.mp4

3rd party LLMs

There are currently lots of LLM providers that offer APIs that are compatible with OpenAI. Prompta should be able to use any of these LLMs easily. Previously the experience was not great. You could use 3rd party LLM APIs but you had to fiddle with settings to use a custom provider and had to undo those settings to get back to OpenAI. How LLM providers can all exist together and you can add as many as you like.

Prompta LLMs

For some time I've wanted to improve the first-time user experience of Prompta by making it possible to immediately chat with the LLM without having to go bring their own key. For all of us existing users this is no big deal, we have OpenAI accounts and we know how to generate API keys. However, this is quite a lot of hassle for some people. I want to be able to introduce friends and family to AI by sending them to Prompta. If they have to sign up for OpenAI at the same time it's a non-starter.

There are lots of competing LLM providers now so I've set up an endpoint to provide LLM access for free to users without having to sign up. Assumedly almost no one will use the app so it won't be cost prohibitive on my end, but I may have to revisit if the API starts getting hammered.

For now this just means that all you need to use Prompta is to open it in a browser tab.

I recognize existing users probably don't care about this, so it's also possible to disable the Prompta LLMs in the settings. This provides the same experience as before—simply providing access to OpenAI via your own key.

What's Changed

Allow arbitrary LLM providers, Prompta-provided LLMs by @iansinnott in #29

Full Changelog: v3.3.0...v4.0.0

Contributors

iansinnott

Assets 2

20 Dec 06:43

github-actions

v3.3.0

4b20fbf

Prompta v3.3.0

See the assets to download and install this version.

Assets 8

20 Dec 06:44

github-actions

v3.2.1

ba7da98

Prompta v3.2.1

See the assets to download and install this version.

Assets 8

17 Dec 09:59

github-actions

v3.2.0

a37f591

Prompta v3.2.0

Sync v2

Sync has been revamped so it actually works. The initial implementation used the vlcn p2p module, but it had issues when all clients weren't online at the same time, which is most of the time. Instead, there's now a sync server which remains up and handles connections from all clients. You can run your own sync server or use the default one.