Chatbot with Gradio, FastApi Endpoint, Langchain Integration

[april-yyt]

Description of changes:

UseCases

1. Gradio Interface Integration

• A new Gradio Chat Interface has been implemented, allowing interactive conversational engagement with FlexFlow.
• This interface generates responses based on user input through a streamlined and user-friendly web interface.

2. FastAPI Integration

• The FastAPI application has been updated for better performance and usability.
• The application can be run using the command uvicorn fastapi_app:app --reload --port PORT_NUMBER, and is accessible at http://localhost:PORT_NUMBER.
• The server setup includes comprehensive API documentation accessible at http://localhost:PORT_NUMBER/docs.

3. FlexFlow and LangChain Integration

• FlexFlowLLM Class: This class manages the initialization, configuration, and operation of the FlexFlow server.
• FF_LLM_wrapper Class: Acts as a wrapper for FlexFlow, facilitating interaction with the LangChain library.
• Main Execution Flow: The main execution now includes initializing FlexFlow, compiling and starting the server with generation configurations, and implementing a prompt template for response generation using LLMChain.

Usage

1. For Gradio Interface: Run the script with an optional configuration file for custom settings, and interact with the model through the Gradio web interface.
2. For FastAPI Application: Execute the application with Uvicorn, with the command uvicorn fastapi_app:app --reload --port PORT_NUMBER, and use the provided base URL for making API requests or accessing documentation.
3. For FlexFlow and LangChain: Running the script would initialize the FlexFlowLLM with an optional configuration file, compile and start the server, and use the FF_LLM_wrapper with LLMChain for generating responses to predefined questions.

Related Issues:

Linked Issues:

• Issue #

Issues closed by this PR:

• Closes #

---

This change is 

[goliaro]

why do we need to remove this? @april-yyt

[april-yyt]

That's a misdeletion, I will add it back.