Tasks scheduler
To run, schedule and review tasks, interfaces have been created:
LongRunningTaskExecutor- any executable task. This task can be run from the code.SchedulableTaskExecutor- scheduled task. The task can be scheduled through UI schedulers.AbstractSchedulableStatefulExecutor- statefull scheduled task. Result is logged for every processed item (see next section).
The descendants of these interfaces can be transmitted to the service LongRunningTaskManager to be run. This is how the records related to them needed for their future management in the database IdmLongRunningTask will be created. If the task fails or it is interrupted, again a record in the database is made. On the records of the running / finished tasks, an agenda on front-end has been created where it is possible to see the course of the task and cancel the tasks as needed.
To schedule the task, the Quartz Scheduler has been integrated through the service SchedulerManager. The service will automatically find all the tasks implementing SchedulableTaskExecutor and offers them for scheduling in UI. The task parameters can be specified in the getParameterNames() method, these parameters being passed before running through the init(Map<String, Object> properties) method - there are string values falling through the UI so it depends on the task how it will accept the parameters (the prepared parameter Converter can be used).
The task can be scheduled in two different ways - types of triggers:
- Settled time - will enter a settled launching time If the time was entered in the past, then the task is launched right away.
- Cron - Quartz expression
Every task has a defined instance (server) where it should be run. From one instance, it is possible to schedule tasks on more instances (one front-end for more back-ends over one database). The task running on the selected instance is ensured by the service LongRunningTaskManager which controls actively the tasks to be run prepared by the scheduler (through the database). The instance identifier is defined in application configuration. To run the instance, the tasks which are marked as running in the previous launching of the instance in question are checked, and they are marked as finished (they couldn´t finish).
Cron in profile DEV
Tasks started by CRON in maven profile DEV was started only once for 3600000ms. This configuration is set by property: scheduler.task.queue.process in application-dev.properties. Server restart is needed.
Configuration QUARTZ (datasource, etc.)
Configuration of QUARTZ (scheduler) it is using application properties by individual profiles. In every profile application properties is necessary set path for queart's settings (his own application properties) - scheduler.properties.location.
Example
Example of an implementing task - you will find everything that has been mentioned here.
Stateful task executors
Stateful task executors are an extension of standard long running task executor, which hold a state and processed items log. Such functionality is suitable for tasks, which has to keep historical records of processed items.
The key concepts of stateful executors are queue and log. The queue is a set of records that have been processed, but the information about them is required for next run of the task. Take contract enable task as an example. Once a contract is valid, the task should process it. But it must be processed only once, not every time the task is run and the contract is valid. Therefore until the contract is valid, it is kept in the queue and will not be processed again. The log is a mechanism to keep track of what kind of items were processed by the task and what was the processing result.
Stateful tasks are tightly coupled with Quartz' scheduled tasks, that is the tasks that are run multiple times. Running a stateful task from code as a simple LongRunningExecutor is pointless, because there is no reason to keep neither history nor processed items queue. The relation to Quartz is realized through IdmScheduledTask entity, which holds Quartz job's unique job key. The task queue then references the IdmScheduledTask. Original IdmLongRunningTask entity is the owner of the log items. Log items are related to each task run, which is done by the mentioned entity.
All stateful tasks shall implement the SchedulableStatefulExecutor. Its abstract implementation AbstractSchedulableStatefulExecutor, which can be viewed as a template method for task execution, hold most of the common logic. The intent is to always extend the AbstractSchedulableStatefulExecutor and define the getItemsToProcess and processItem method by concrete task. Therefore the developer should mostly focus only on retrieving items and the business logic, which does the processing.
The item processing result is represented by OperationResult. Successful operations are inserted both into the queue and log, all results go to the log.
After the item processing finishes, the queue is refreshed. The algorithm is simple, as it only removes all items from queue, which were not processed in current run.
Implemented task types
PasswordExpirationWarningTaskExecutor
Sends warning notification before password expires. Days before has to be given as task's parameter (number greater than zero). More task could be configured e.g. for sending warning notification 14,7,3 days before password expires. Default notification topic is configured to email sender.
PasswordExpiredTaskExecutor
Sends warning notification after password expires. Default notification topic is configured to email sender.
AccountProtectionExpirationTaskExecutor
Removes accounts with expired protection interval. Account has to have inProtection flag setted to true.
RetryProvisioningTaskExecutor
Retry failed provisioning operation periodically.
TODO: all other tasks from core module.
LongRunningTask vs. ScheduledTask
ScheduledTask is created on page Scheduled tasks by clicking on add button in the right corner of page. Now we have ScheduledTask which has its own list of items which were processed by this task. After we created this ScheduledTask there is no item on the list. First run of this ScheduledTask is created when we hit run button.
Than it will create LongRunningTask "A" on page All tasks. This LongRunningTask "A" will look at the list of processed items and figures out which items are processed this time with this current LongRunningTask "A". After everything is done, we do not use LongRunningTask "A" anymore, but we still need our Scheduled task because it has our list of processed items. If we creates another LongRunningTask "B", it will look to the list and won't process items which were processed by LongRunningTask "A".
Testing tips
Integration tests are the preferred way of testing stateful executors in combination with stubbing the returned set of items to process by Mockito, using a 'spy' on the executor. This way the developer has full control over the task and can test both processing and items retrieval independently. List through the existing test cases for details.
Future development
- Support for the check of the competition of the running tasks. Now it is on the task implementation to check if it should be run or not. This could be extracted to the general.
- Task running based on finishing another task.
- Immediate task running is carried out through the scheduler - a simple trigger - it could be carried out directly through the service LongRunningTaskManager
- Use of WebSockets for the automatic refresh of the progress bar on front-end
- Editing the scheduled task - the editing is not supported by the scheduler - it needs to be implemented as drop and create + transmit all the triggers
- Display of the scheduled triggers in the detail of the scheduled job