Initial Commit

Add instructions in readme.md

Add a command to check version

Cleanup

Add base LLAIM exception

cleanup some dirs

Add base data aggregator

Add base aggregator exception

Add ruff to requirements

Refactor and Rename the Aggregator to ETL

Add method for source creation

* Also add method for source and destination definitions list

Airbyte etl method fixes

Add utils for importing

* Refactor base etl class

Add langchain loader

Add assets directory

Add support for custom embeddings in langhub etl

Uprev click version

Add basic tests

Remove unused imports

Rename LangHubEtl to LangLoaderEtl

Add util function to extract init_attrs from a class

refactor extraction util

Fix typo

Rename testcases

Fix line lenghts

Add llamahub etl loader

Delete a folder

Initial setup for models

Rename model class

Add a decorator func to start a http server

Add docker-compose setup for weaviate

Update Weaviate readme

Add Langchain retriever

Refactor Airbyte ETL

Pass the method instead of its signature

Add loading using llamahub loaders

Add a test for llamahub loader etl

Fix imports

Create a http server class with FastAPI instead of a decorator function

Refactor Base model class and inherit HttpServer class

Sort imports

Allow overriding of few http server configs

Add OpenAI Gpt 3.5 turbo model

Add helper functions for running and listing models

Add cli commands to list and run a model as http server

Add open_ai model for testing

Revamp the config loader structure in modules

Add vector similarity search in retrieval

Restructure the dynamic loading of modules

Make the config keys to be constant

Wire the retrieval chain

Fix imports in all modules

Complete the entire flow of retrieval

Fix parsing in retrieval

Remove unnecessary print statements

Add langchain retrievalQA chain for retrieval

Rename llaim -> llm stack

Update List of available list of built-in models

Cosmetic updates

* Add more cli aliases
* Update Readme

Update github org url

Update code syntax in Readme

Update classifiers

Rename compulsory_fields -> required_fields

Fix custom exception class typo

Raise exception when unkown prebuilt model is specified in the config file

* Does not apply to custom user models

Fix annotation typo

Fix accessing required_fields attr when its not set

* Format the file

Add support for custom models

Add custom_model config asset

Fix a typo

Remove unused import

Add cli command for etl

Add docs file for airbyte installation

Add a command to download and install airbyte

Fix executing of commands

Update Readme for setting-up Airbyte

Update ISSUE_TEMPLATE.md

Rename compulsory_field(s) -> required_field(s)

Update llm-stack repo

Print the default class_name when not provided

Remove unused imports

Parse config only if provided

Move Configloader in model init

Add support for api key auth for weaviate

Fix typo

Add vectordb based memory for chat models

Fix typo

Parse responses in json

Make vector store optional for running models

Handle when vectorDB is not provided

Refactor gpt3.5 model

Mention required dependencies in requirements

Update git repo

Add weaviate client

Update setup.py

Revert "Update setup.py"

This reverts commit b269b77.

Loosen dependencies

Add support for running gpt3.5 model without retreiver

Remove unused attribute

Fix a typo

Update Readme instructions

Fix a typo

Update readme formatting

Add gpt4all model

Fix typos

Fix syntax

Update config.json in readme instructions

Loosen deps

Add method to init a class in base class of the various components

Add chat_history as an API endpoint

Fix readme

Update LICENSE

Update license in setup.py

Render chat history from llm stack

Uncomment config attr

Default to gpt4all model if no config file is provided

Remove unneeded methods

Add a missing util file

Update Readme.md

Load config before model loading

Add source documentation display and chat input

Create Model.md

Create ETL.md

Create VectorDB.md

Create Retreiver.md

Add text_key for weaviate

Add examples for chatting with a pdf

Add a readme for chat with pdf example

Update Readme.md

add ETL documentation

model docs update

retrieval docs update

add vectordb docs

new readme.md created

remove custom embeddings

Add text_keys validator

Change error message

Update README.md

Update README.md

Update installation dependencies

Update README.md

Include prerequisites in the Readme

Pass encoding

Update Readme.md

Allow customization of openai model

Allow customizing openai model

Do not call super model load method

Add huggingface model support

Loosen deps

Add transformers

Download gpt4all manually

Refactor gpt4all model

Update docs Readme

Update docs

Move documentation section above Usage in main readme file

Capitalize class name in vectordb

Update README.md

Move Readme.md from docs/components to root of the project

Update readme.md

Fix a typo

Update Readme.md

Update Readme.md

Update README.md

Update README.md

Update README.md

Update README.md

Update DEFAULT_MODEL_JSON

Update README.md

Update keywords in setup.py

fix readme

add gpt3.5 json file

Use pathlib

Loosen deps

Add templating engine for installations

Parse Path objects as string

Add template engine requirements

Use absolute path for models

Pass device to HuggingFacePipeline

Restructure the Readme.md

Include torch as a dependencies

Use logger and format and fix imports in some files

Update README.md

Update README.md

Update README.md

Add chromadb to available vectordbs

Alter examples with new json structure

Use huggingface as default embeddings

Add setup enviornment section

Update requirements_dev.txt

update the destination name to vectorDB in etl.json

change the destination to vectorDB

add chromaDB infomation

rename destination

custom config file for chromaDB

remove openai key

Update README.md

rename destination

keeping only relevant code for ChromaDB

remove code snippet

Remove unwanted validation

Add chromadb to requirements

Fix conflicting dependencies

Resolve fastapi dependencies

Add pypdf to requirements

Add python based interface to components

Fix typo in source field generation

Do not stringify response

Update max_tokens for gpt4all model and remove jsonification of the result

Fix Typo

Update README.md

Update README.md

Update README.md

Update README.md

GITBOOK-3: Nikhil's Aug 15 changes

GITBOOK-5: Add Supported Models/LLMs

GITBOOK-2: Data Extraction and Loaders

Add citation

Fix syntax

Update README.md

GITBOOK-11: Update default data types page

GITBOOK-7: Vectordb

GITBOOK-15: change request with no subject merged in GitBook

GITBOOK-17: change request with no subject merged in GitBook

GITBOOK-16: Restructuring component documentation

GITBOOK-18: change request with no subject merged in GitBook

GITBOOK-20: change request with no subject merged in GitBook

GITBOOK-19: sam's Aug 16 changes

GITBOOK-22: change request with no subject merged in GitBook

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

GITBOOK-24: Shreehari's Aug 16 changes

GITBOOK-25: change request with no subject merged in GitBook

Update README.md

Update README.md

GITBOOK-21: change request with no subject merged in GitBook

GITBOOK-26: change request with no subject merged in GitBook

Update README.md

GITBOOK-27: change request with no subject merged in GitBook

GITBOOK-28: change request with no subject merged in GitBook

Update README.md

GITBOOK-29: change request with no subject merged in GitBook

GITBOOK-30: Nikhil's Aug 16 changes

Add workflow image

Update README.md

Remove obsolete docs directory

Handle chromadb sqlite3 compatibility issues

Update workflow image

Update README.md

Fix documentation url paths

Update README.md

Update README.md

Update README.md

Update head section of README.md

Rename llm_stack to genai_stack

Rename figlet

Change Workflow image

Add chat with webpage

Co-authored-by: ShreehariVaasishta <shreehari9481@gmail.com>
Co-authored-by: Samjoel3101 <samjoel3101@gmail.com>
Co-authored-by: nikhilchintawar <chintawarnikhil88@gmail.com>
Co-authored-by: chanukyapatnaik <chanukya.patnaik@gmail.com>
Co-authored-by: lucifertrj <jaintarun.abd17@gmail.com>

.editorconfig

@@ -0,0 +1,21 @@
+# http://editorconfig.org
+
+root = true
+
+[*]
+indent_style = space
+indent_size = 4
+trim_trailing_whitespace = true
+insert_final_newline = true
+charset = utf-8
+end_of_line = lf
+
+[*.bat]
+indent_style = tab
+end_of_line = crlf
+
+[LICENSE]
+insert_final_newline = false
+
+[Makefile]
+indent_style = tab

.github/ISSUE_TEMPLATE.md

@@ -0,0 +1,15 @@
+* GenAI Stack version:
+* Python version:
+* Operating System:
+
+### Description
+
+Describe what you were trying to get done.
+Tell us what happened, what went wrong, and what you expected to happen.
+
+### What I Did
+
+```
+Paste the command(s) you ran and the output.
+If there was a crash, please include the traceback here.
+```

.gitignore

@@ -0,0 +1,112 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+env/
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# dotenv
+.env
+
+# virtualenv
+.venv
+venv/
+ENV/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+
+# IDE settings
+.vscode/
+.idea/
+sandbox/private/
+
+# ruff
+.ruff_cache/
+
+app.conf

AUTHORS.rst

@@ -0,0 +1,13 @@
+=======
+Credits
+=======
+
+Development Lead
+----------------
+
+* AIM by DPhi <support@aiplanet.com>
+
+Contributors
+------------
+
+None yet. Why not be the first?

CITATION.cff

@@ -0,0 +1,19 @@
+cff-version: 1.2.0
+title: 'AI Planet'
+message: >-
+  If you use this software, please cite it using this
+  metadata.
+type: software
+authors:
+  - given-names: Shreehari
+    family-names: Vaasistha
+    email: shreehari@aiplanet.com
+  - given-names: Sam
+    family-names: Joel
+    email: sam@aiplanet.com
+  - given-names: Tarun
+    family-names: Jain
+    email: Tarun@aiplanet.com
+repository-code: 'https://github.com/aiplanethub/genai-stack'
+url: 'https://aiplanet.com/'
+license: Apache-2.0

CONTRIBUTING.rst

@@ -0,0 +1,128 @@
+.. highlight:: shell
+
+============
+Contributing
+============
+
+Contributions are welcome, and they are greatly appreciated! Every little bit
+helps, and credit will always be given.
+
+You can contribute in many ways:
+
+Types of Contributions
+----------------------
+
+Report Bugs
+~~~~~~~~~~~
+
+Report bugs at https://github.com/aiplanethub/genai-stack/issues.
+
+If you are reporting a bug, please include:
+
+* Your operating system name and version.
+* Any details about your local setup that might be helpful in troubleshooting.
+* Detailed steps to reproduce the bug.
+
+Fix Bugs
+~~~~~~~~
+
+Look through the GitHub issues for bugs. Anything tagged with "bug" and "help
+wanted" is open to whoever wants to implement it.
+
+Implement Features
+~~~~~~~~~~~~~~~~~~
+
+Look through the GitHub issues for features. Anything tagged with "enhancement"
+and "help wanted" is open to whoever wants to implement it.
+
+Write Documentation
+~~~~~~~~~~~~~~~~~~~
+
+GenAI Stack could always use more documentation, whether as part of the
+official GenAI Stack docs, in docstrings, or even on the web in blog posts,
+articles, and such.
+
+Submit Feedback
+~~~~~~~~~~~~~~~
+
+The best way to send feedback is to file an issue at https://github.com/aiplanethub/genai-stack/issues.
+
+If you are proposing a feature:
+
+* Explain in detail how it would work.
+* Keep the scope as narrow as possible, to make it easier to implement.
+* Remember that this is a volunteer-driven project, and that contributions
+  are welcome :)
+
+Get Started!
+------------
+
+Ready to contribute? Here's how to set up `genai_stack` for local development.
+
+1. Fork the `genai_stack` repo on GitHub.
+2. Clone your fork locally::
+
+    $ git clone git@github.com:your_name_here/genai_stack.git
+
+3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development::
+
+    $ mkvirtualenv genai_stack
+    $ cd genai_stack/
+    $ python setup.py develop
+
+4. Create a branch for local development::
+
+    $ git checkout -b name-of-your-bugfix-or-feature
+
+   Now you can make your changes locally.
+
+5. When you're done making changes, check that your changes pass flake8 and the
+   tests, including testing other Python versions with tox::
+
+    $ flake8 genai_stack tests
+    $ python setup.py test or pytest
+    $ tox
+
+   To get flake8 and tox, just pip install them into your virtualenv.
+
+6. Commit your changes and push your branch to GitHub::
+
+    $ git add .
+    $ git commit -m "Your detailed description of your changes."
+    $ git push origin name-of-your-bugfix-or-feature
+
+7. Submit a pull request through the GitHub website.
+
+Pull Request Guidelines
+-----------------------
+
+Before you submit a pull request, check that it meets these guidelines:
+
+1. The pull request should include tests.
+2. If the pull request adds functionality, the docs should be updated. Put
+   your new functionality into a function with a docstring, and add the
+   feature to the list in README.rst.
+3. The pull request should work for Python 3.5, 3.6, 3.7 and 3.8, and for PyPy. Check
+   https://travis-ci.com/dphi-official/genai_stack/pull_requests
+   and make sure that the tests pass for all supported Python versions.
+
+Tips
+----
+
+To run a subset of tests::
+
+
+    $ python -m unittest tests.test_genai_stack
+
+Deploying
+---------
+
+A reminder for the maintainers on how to deploy.
+Make sure all your changes are committed (including an entry in HISTORY.rst).
+Then run::
+
+$ bump2version patch # possible: major / minor / patch
+$ git push
+$ git push --tags
+
+Travis will then deploy to PyPI if tests pass.

HISTORY.rst

@@ -0,0 +1,8 @@
+=======
+History
+=======
+
+0.1.0 (2023-07-06)
+------------------
+
+* First release on PyPI.