Skip to content

dfunc-serverless/WorkerNode-Python

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

60 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

WorkerNode


The worker node repository defines how workers are interacting with the scheduler component. The communication is supported by Google pubsub, a Google API allows message publication and subscription.

Features

  • Constantly listening to the job scheduler
  • Parallel jobs running in threads
  • Sufficiently fills up allocated resources with function jobs

Dependency

  • Docker
  • Google pubsub

Handling Secrete

  • By using Google API, dFunc has to expose restricted Google API keys to worker nodes. The API keys are shared among all workers and only allows subscription. All Google API keys will be saved inside the .config directory.

Usage

Running Worker node component only requires workers to run the main python function with valid JSON secrets.

Further Work

  • Package worker node into a docker image.
  • Connect with a GUI to allow registered user startup with one click.