-
Notifications
You must be signed in to change notification settings - Fork 5
Configuration
This section explains Kronos configuration.
Scheduler configuration is defined in scheduler.yaml
file and is picked by the Kronos from classpath to initialize and start the scheduler.
The task definition readers are configured in the scheduler.yaml
file under taskReaderConfig
section to read task definitions from a custom source. The user can also implement and configure a custom reader.
KEY | DESCRIPTION | TYPE | DEFAULT | MANDATORY |
---|---|---|---|---|
readerClass | fully qualified class name of the TaskDefinitionReader implementation | string | None | yes |
schedule | A cron schedule to refresh the task definitions from the source | string | None | yes |
config | additional configuration required by the reader during instantiation | objectnode | None | no |
Sample configuration
taskReaderConfig:
# name to be assigned to the reader
filereader:
# fully qualified classname of the TaskDefinitionReader.java implementation
readerClass: com.cognitree.kronos.scheduler.readers.FileTaskDefinitionReader
# additional configuration
config:
source: task-definitions.yaml
# interval in cron to look for changes in the task definition file
schedule: 0 0/1 * 1/1 * ? *
In the above sample, a file reader is configured.
The set of configuration required by scheduler per task type, defines default max execution time and policy to apply on task in case of timeout. The task execution config is configured in the scheduler.yaml
file under taskExecutionConfig
section.
KEY | DESCRIPTION | TYPE | DEFAULT | MANDATORY |
---|---|---|---|---|
maxExecutionTime | max allowed time for the task to finish execution if not explicitly specified by task | string | 1d | no |
timeoutPolicy | policy to apply in case of timeout if not explicitly specified by task | string | None | no |
taskExecutionConfig:
# task type
shellCommand:
# max execution time per task
maxExecutionTime: 10m
# policy to apply on task in case of timeout
timeoutPolicy: notifyOnTimeout
In the above sample, for task type shellCommand
max exection time is set to 10m
and a notifyOnTimeout
policy is configured . The above configuration is applied on all task of type shellCommand
if not explicitly configured at a task level. Check out Task Definition section for more details.
A task store is configured in the scheduler.yaml
file under storeProvider
section and is used to store the current state of the scheduler. The user can also provide a custom implementation of task store.
KEY | DESCRIPTION | TYPE | DEFAULT | MANDATORY |
---|---|---|---|---|
taskStoreClass | fully qualified class name of the TaskStore implementation | string | None | yes |
config | additional configuration required by the task store during instantiation | objectnode | None | no |
taskStoreConfig:
taskStoreClass: com.cognitree.kronos.store.SQLiteTaskStore
config:
connectionURL: jdbc:sqlite:kronos.db
username:
password:
In the above sample, a sqlite task store is configured.
Define policies to apply in case of task timeout. A timeout policy is configured in the scheduler.yaml
file under timeoutPolicyConfig
section. The user can also provide custom implementation of timeout policy.
KEY | DESCRIPTION | TYPE | DEFAULT | MANDATORY |
---|---|---|---|---|
policyClass | fully qualified class name of the TimeoutPolicy implementation | string | None | yes |
config | additional configuration required by the timeout policy during instantiation | objectnode | None | no |
timeoutPolicyConfig:
notifyOnTimeout:
policyClass: com.cognitree.examples.tasks.scheduler.policies.NotifyOnTimeoutPolicy
config:
smtp.host: "smtp.gmail.com"
smtp.port: 587
sender.name:
sender.email:
sender.password:
email.subject: "Task Timeout notification"
recipient.name:
recipient.email:
In the above sample, notifyOnTimeout
is the policy identifier that is to be used while defining the task definition and/or task handler as the timeout policy.
Periodically, tasks older than the specified interval and status in one of the final state are purged from memory. The task purge interval is configured in the scheduler.yaml
file under taskPurgeInterval
section.
KEY | DESCRIPTION | TYPE | DEFAULT | MANDATORY |
---|---|---|---|---|
taskPurgeInterval | purge interval (1d, 1h 15m etc) | string | 1d | no |
taskPurgeInterval: 6h
In the above sample, the task purge interval period is configured as 6 hours.
Executor configuration is defined in executor.yaml
file and is picked by the Kronos from classpath to initialize and start the executor.
For each type of task, a handler needs to be configured to execute the task. A handler is configured in the executor.yaml
file under handlerConfig
section. The user can also implement and configure a custom task handler to handle tasks.
KEY | DESCRIPTION | TYPE | DEFAULT | MANDATORY |
---|---|---|---|---|
handlerClass | fully qualified class name of the TaskHandler implementation | string | None | yes |
config | additional configuration required by the handler during instantiation | objectnode | None | no |
maxParallelTasks | max number of tasks allowed to be executed in parallel | string | max cores/2 | no |
# configure task handlers
taskHandlerConfig:
# type of task to be handled by handler
shellCommand:
# handler implementation used to handle task of type shellCommand
handlerClass: com.cognitree.kronos.executor.handlers.ShellCommandHandler
# max parallel tasks handler is allowed to execute at any point of time
maxParallelTasks: 4
In the above sample, Shell Command Handler is configured to handle tasks of type shellCommand
.
Queue is used by the scheduler and executor to exchange messages. A producer and a consumer for the queue is to be configured in queue.yaml
. The user can provide a custom implementation of queue.
Producer
KEY | DESCRIPTION | TYPE | DEFAULT | MANDATORY |
---|---|---|---|---|
producerClass | fully qualified class name of the Producer implementation | string | None | yes |
config | additional configuration required by the producer during instantiation | objectnode | None | no |
Consumer
KEY | DESCRIPTION | TYPE | DEFAULT | MANDATORY |
---|---|---|---|---|
consumerClass | fully qualified class name of the Consumer implementation | string | None | yes |
config | additional configuration required by the consumer during instantiation | objectnode | None | no |
# configure producer and consumer for the queue
producerConfig:
producerClass: com.cognitree.kronos.queue.producer.InMemoryProducer
consumerConfig:
consumerClass: com.cognitree.kronos.queue.consumer.InMemoryConsumer
pollInterval: 100
# topic to use to publish task status reported by executor and consumed by scheduler
taskStatusQueue: taskstatus
In the above sample, an in-memory queue is configured.