Skip to content

Commit

Permalink
feat(KB): file-to-embedding worker pool (#32)
Browse files Browse the repository at this point in the history
Because

users want upload the file to build their knowledgebase and the files in
knowledge base need to turn into vectors.

This commit

implement a file-to-embedding work pool.

Note: It is still in progress until preset/pipelines on instill cores
are ready.
  • Loading branch information
Yougigun authored Jul 9, 2024
1 parent 8a52cba commit 5409db3
Show file tree
Hide file tree
Showing 40 changed files with 9,546 additions and 2,235 deletions.
Binary file removed .DS_Store
Binary file not shown.
122 changes: 122 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# From https://github.com/github/gitignore

#### Visual Studio Code
.vscode
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json
*.code-workspace

# Local History for Visual Studio Code
.history/

#### Python
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Sphinx documentation
docs/_build/

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Cython debug symbols
cython_debug/

# Auto-gen files
autogen

# mysql persistent data
mysql-persistence/

# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib

/openapi

# Test binary, built with `go test -c`
*.test

# Output of the go coverage tool, specifically when used with LiteIDE
*.out

# Dependency directories (remove the comment below to include it)
# vendor/

# built binary
artifact-backend

# local protogen-go
protogen-go

# deploy config file
deploy.yaml

# air related files and folders
.air.toml
tmp

.DS_Store

# local files
/config/config_local.yaml
test_.pdf
test_.md
Loading

0 comments on commit 5409db3

Please sign in to comment.