Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AI insights #8384

Draft
wants to merge 13 commits into
base: develop
Choose a base branch
from
Draft

AI insights #8384

wants to merge 13 commits into from

Conversation

gbp
Copy link
Member

@gbp gbp commented Sep 13, 2024

Relevant issue(s)

What does this do?

Why was this needed?

Implementation notes

Screenshots

Notes to reviewer


Have you updated the changelog? If this is not necessary, put square brackets around this: skip changelog

This isn't used as far as we know and also isn't manitained. Due to old
gem requirements if preventing some new gem from being installed.
Allow method to be run without arguments.
@gbp gbp force-pushed the ai-insights branch 6 times, most recently from 0a106d5 to 2dc5c7c Compare September 16, 2024 10:30
This commit introduces a new Workflow system that allows for the
execution of a sequence of jobs on a given resource.

Components include:
- Workflow class: Manages the overall workflow and job execution
- Workflow::Job model: Represents individual jobs within a workflow
- Workflow::Source module: Handles source content and output attachments
- Workflow::Transitions module: Manages job state transitions
- WorkflowJob: Background job for processing workflows

Features:
- Sequential job execution with parent-child relationships
- Job status tracking (pending, queued, processing, failed, completed)
- Automatic progression through workflow steps
- Extensible design for custom job types

This lays the groundwork for implementing specific workflow processes in
the application.
- Introduce Chunk model for storing text fragments and embeddings
- Add Chunkable concern for processing models through chunking workflow
- Implement chunking for InfoRequest, IncomingMessage, and FoiAttachment
  models

This commit lays the groundwork for AI-powered text analysis by adding
a chunking system that breaks down text from various models into smaller,
analyzable pieces.
- Implement ConvertToText, AnonymizeText, and CreateChunks jobs
- Update Workflow class to use new job classes
Run on requests where the requester is using projects and has the
insights feature flag.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant