Cloud-based solution with AI-driven tagging, content-based search, and integration capabilities with existing tools
- Contributors
- Quick Preview & Links
- Inspiration
- What It Does
- How We Built It
- Challenges We Faced
- Key Accomplishments
- Lessons Learned
- What’s Next
- Try demo
- Demo video
- User Documentation and Guide
- DevPost project
- Presentation Deck
- Team slack channel
Empowering NPOs with Accessible, Cost-Effective and Cloud-Based Technology
Our project was inspired by the unique challenges that nonprofit organizations (NPOs) face when managing large volumes of media files. While many cloud-based media management solutions exist, they tend to be costly and overly complex, making them inaccessible to smaller nonprofits or individuals with limited technical expertise. NPOs often have very niche use cases and require a solution tailored to their specific needs. By leveraging open-source models and AI, we aimed to create an affordable, easy-to-use media organization system that addresses these challenges.
Our system streamlines media management for nonprofits by automatically tagging and organizing images using AI. Users can upload media manually or through Dropbox, and easily search collections using content-based queries. Our system uses AI models like CLIP, GPT-4 Mini, and Llama-3.1 to automatically tag and organize images based on visual content, descriptions, keywords and metadata, enabling users to easily search collections.
- Frontend: Built using React.js for an intuitive, interactive user interface.
- Backend: Developed with FastAPI (Python) to handle API requests and facilitate communication between the front and backend.
- Cloud Infrastructure: Hosted on AWS for reliable cloud storage and hosting.
- Database: Postgres was selected for efficient data management and tagging.
- Dropbox Integration: Implemented via the Dropbox API for seamless file handling.
- AI Functionality: Integrated AI models, including:
- Embedding Generation: CLIP for generating image and text embeddings to enable automated tagging.
- Image Captioning: A prompt-driven approach for generating descriptive captions.
- Metadata Extraction: Extracting relevant metadata from images for better organization.
| Purpose | Technologies |
|---|---|
| Frontend | ReactJs(JavaScript) |
| Backend | FastAPI(Python) |
| Cloud | AWS |
| Database | Postgres |
| APIs | Dropbox |
| AI Model | Llama-3.1, CLIP |
- Dropbox Authentication: Resolving issues with secure Dropbox authentication and API compatibility.
- Database Query Optimization: Enhancing Postgres search query performance for fast and accurate media retrieval.
- Model Compatibility: Identifying AI models with compatible embeddings for seamless integration and accurate tagging results.
- Frontend-Backend Integration: Ensuring smooth communication between React.js and FastAPI.
- Significant Time Savings: Achieved a 91.67% reduction in time spent on media management tasks, reducing average time per task from 2.00 hours to 0.17 hours.
- Cost Efficiency: This substantial time reduction translated to an annual savings of approximately $2,383.33 in wages for the nonprofit organization, highlighting the financial benefits of implementing our system.
- AI-Driven Tagging: Successfully integrated AI-driven tagging that automates the media organization process, significantly reducing manual effort and increasing accuracy in tagging.
- Customizable Tagging Options: Implemented manual tagging features that allow users to add personalized tags to their files, providing flexibility and enhancing user experience by combining AI-generated tags with user-defined labels.
- User-Centric Design: Prioritized a simple, intuitive interface to ensure accessibility for non-technical users, particularly benefiting nonprofit organizations with limited technical expertise.
- Achievement of Core Objectives: Met critical project goals, including seamless integration of manual tagging, reliable cloud hosting, and effective Dropbox authentication, establishing a robust foundation for future enhancements.
- Vector Search: Mastered the implementation of vector-based search for improved media retrieval.
- Metadata Structuring: Gained deeper insights into metadata organization for better media tagging and search efficiency.
- Dropbox Integration: Overcame challenges related to secure API authorization for Dropbox.
- Broader Cloud Integration: We plan to extend support to other cloud services like Google Drive and OneDrive, offering more flexibility to users.
- Canva Integration: We plan to integrate with PEEC's Canva account to streamline media editing and design processes directly from the media management platform.
- Digital Rights Management (DRM): Introducing metadata functionality for tracking image ownership and photo credits to ensure proper attribution and usage rights for each media file.
- Advanced Version Control: Implementing version control for media files to track changes, prevent accidental overwrites, and allow restoration of previous file versions if needed.
- Storage Flexibility & Scalability: Enhancing storage management tools and scalability to support growing media libraries, including the ability to archive files and track storage usage.
- Role-Based Access Control: Setting up role-based permissions (e.g., admin vs. view-only) to allow secure file sharing and user access management.
- Mobile Accessibility: Exploring mobile-friendly features to allow PEEC staff to manage and access their media library on the go.