Events - processing of events event architecture configuration

• Event is created with given content

EntityEvent<IdmIdentityDto> event = new IdentityEvent(IdentityEventType.UPDATE, updateIdentity);

• then is given to the processing via EntityEventManager

EventContext<IdmIdentityDto> context = entityEventManager.process(event);

• when event is published and their content is descendant of AbstractDto, then original source is filled to the event - original source contains previously persisted (original) dto and could be used in "check modification" processors. If event creating new dto, then original source is null. Original source could be set externally - then no automatic filling occurs.
• returning context contains results from all reacting processors in defined processors order.
• the processor can label the event as (closed) or (suspended) and therefore skip all the other processors. If the suspended event is published again via EntityEventManager, the processing will continue where it was suspended, if context (with processed results) is preserved. If the processing of the event is (suspended), the called method should return the adequate accepted state.
• when event walk through processors, then event's processed order is incremented - this order is used after event suspending and run again - event processing will continue with processor with the next order. Look out: when event is persisted, she will lost their context ⇒ when event is recreated from persistent state, she will continue with the next order.

Event is published with content and specific event type (e.g. CREATE, PASSWORD). Processors can be register by content type and event type - e.g. process event with IdmIdentityDto content and event type CREATE ⇒ other event contents and other event types will not be processed by this processor.

Event types has to be compared by their string representation, NOT by instance. Concrete event types e.q. IdentityEventType are used for documentation reason only - which domain type supports which event. Event types can be added in different modules with different type, but processor can react across all module (⇒ is registered to string event type representation eventType.name()).

...
EntityEvent<IdmIdentityDto> event = new CoreEvent<IdmIdentityDto>(CoreEventType.CREATE, identity);
if (event.hasType(IdentityEventType.CREATE)) {
 // do something
}
...

Module can publish their own event types. Basic (core) event types

• CREATE - synchronous, new entity is created
• UPDATE - synchronous, entity is updated
• DELETE - synchronous, entity is removed
• EAV_SAVE - synchronous, some entity extended attribute is modified (CUD)
• NOTIFY - asynchronous, notify about entity is changed (CU).

Different (and custom) event types can be used for different entities.

Supported events for individual entities:

• core:
  • IdmAuthorizationPolicyDto - authoriyation policies
    • supports base event types CREATE, UPDATE, DELETE
  • IdmAutomaticRoleAttributeDto - automatic role by attribute
    • supports base event types CREATE, UPDATE, DELETE
  • IdmAutomaticRoleAttributeRuleDto - rule of automatic role by attribute
    • supports base event types CREATE, UPDATE, DELETE
  • IdmIdentityDto - operation with the identity
    • supports base event types CREATE, UPDATE, DELETE, EAV_SAVE, NOTIFY
    • adds event type PASSWORD - changes password
    • adds event type PASSWORD_EXPIRED - password expires
  • IdmRoleDto - operation with the role
    • supports base event types CREATE, UPDATE, DELETE, EAV_SAVE, NOTIFY
  • IdmRoleRequestDto - role requests
    • supports base event types CREATE, UPDATE, DELETE
    • adds event type EXCECUTE - execute role request (i know EXECUTE!, but it's too late …)
  • IdmRoleCatalogueDto - operation with the role catalogue
    • supports base event types CREATE, UPDATE, DELETE, NOTIFY
  • IdmIdentityRoleValidRequestDto - role starts to be valid
    • adds event type IDENTITY_ROLE_VALID.
  • IdmIdentityRoleDto- assigning a role to the user
    • supports base event types CREATE, UPDATE, DELETE, NOTIFY
  • IdmIdentityContractDto - labor-law relation
    • supports base event types CREATE, UPDATE, DELETE, EAV_SAVE, NOTIFY
  • IdmRoleTreeNodeDto - automatic role by tree structure
    • supports base event types CREATE, UPDATE, DELETE
  • PasswordChangeDto - used in password prevalidation only
    • adds event type PASSWORD_PREVALIDATION - evaluates registered password policies - show policies setting before password is changed.
  • IdmContractGuaranteeDto - manually added guarantee to contract
    • supports base event types CREATE, UPDATE, DELETE, NOTIFY
  • IdmTreeTypeDto - tree structure type
    • supports base event types CREATE, UPDATE, DELETE
  • IdmTreeNodeDto - tree structure node
    • supports base event types CREATE, UPDATE, DELETE, EAV_SAVE, NOTIFY
  • IdmPasswordPolicyDto - password policy
    • supports base event types CREATE, UPDATE, DELETE
  • IdmLongRunningTaskDto - long running task
    • adds event type END - long running task ended.
  • IdmEntityEventDto - persisted event
    • supports base event types CREATE, UPDATE, DELETE
    • adds event type EXECUTE - executes persisted event.
  • IdmEntityStateDto - persisted entity / event state
    • supports base event types CREATE, UPDATE, DELETE
• acc:
  • AccAccountDto - Accounts on target system
    • supports base event types CREATE, UPDATE, DELETE
    • adds event type START - starts provisioning for given account.
  • AccIdentityAccountDto - Identity accounts on target system
    • supports base event types CREATE, UPDATE, DELETE
  • SysSystemDto - System in ACC module
    • supports base event types CREATE, UPDATE, DELETE
  • SysSystemMappingDto - Mapping between system and his mapping of provisioning or sync
    • supports base event types CREATE, UPDATE, DELETE
  • SysSchemaAttributeDto - Connector schema on system
    • supports base event types DELETE
  • SysProvisioningOperationDto - execute provisioning operation. Look out, persisting provisioning operation into queue itself doesn't support events. Events are added for executing operations from queue:
    • adds event type CREATE - execute provisioning for CREATE operation
    • adds event type UPDATE - execute provisioning for UPDATE operation
    • adds event type DELETE - execute provisioning for DELETE operation
    • adds event type CANCEL - cancels provisioning operation
  • AbstractSysSyncConfigDto (SysSyncConfigDto, SysSyncContractConfigDto, SysSyncIdentityConfigDto) - synchronization
    • adds event type START - starts synchronization
    • adds event type START_ITEM - starts synchronization of one item (~entity)
    • adds event type CANCEL - cancels synchronization
  • SysSyncItemLogDto - synchronization item
• vs:
  • VsRequestDto - Request for account change in virtual system
    • adds event type EXCECUTE - executes request
• rpt:
  • RptReportDto - generate report
    • adds event type GENERATE - generates request

A page has been created directly in the application on the module page for an overview of all entity types and event types migrating through event processing. All the registered processors including the configuration are listed there:

The default order for listeners (- ⇒ before, + ⇒ after):

• core: 0
• provisioning: 1000
• delete / archivate processed - 5000

• EntityEvent - an event migrating through the processors. The content of the event can be BaseEntity, BaseDto or any serializable content.
• EventContext - holds the context of the processed event - which processors it has been processed by, with what results, if the processing is suspended, closed, etc.
• EventResult - the result of processing of the event by one processor.
• EntityEventProcessor - event processor. Processor has to have unique identifier by module.
• AsyncEntityEventProcessor - asynchronous entity event processor. Processor can control asynchronous processing priority.
• EntityEventManager - ensures publishing of the event to processors.
• EventableDtoService - adds event processing support to service - event processors (for save, delete) have to be provided.

• AbstractEntityEvent - an abstract event migrating through the processors - when adding a proper one can be simply inherited from.
• DefaultEventContext - the default context of the processed event - all abstract and default events and processors use it.
• DefaultEventResult - The default event result processed by one processor - all abstract and default events and processors use it.
• AbstractEntityEventProcessor - abstract event processor - when adding a proper one can be simply inherited from.
• AbstractApprovableEventProcessor - the event processor will send the whole event with dto (or serializable) content to WF for approval. It is necessary to configure the definition of the WF where the event will be sent to.
• AbstractPublishEntityChangeProcessor - publish asynchronous NOTIFY event.
• DefaultEntityEventManager - ensures publishing of the events to processors.
• AbstractEventableDtoService - adds default event processing support to service.

Use this super class, when creating new processor implementation (this class contains some boring parts).

Methods, which have to be implemented:

• getName() - Unique (module scope) configurable object identifier. Its used in configuration key etc.
• process(event) - the main processors method with business logic.
• getOrder() - when will be processor processed. Processors are executed in defined order.

Methods, which could be implemented [optional]:

• supports(event) - Returns true, when processor supports given event. Default implementation takes processor's template entity class and event type given in constructor (or configured by eventTypes property).
• conditional(event) - Returns true, when processor supports given event. Returns true by default. Override this method for adding some condition.
• isClosable() - Returns true, when processor could close event (only documentation purpose now). Returns false by default.

Use this super class, when creating new asynchronous processor implementation.

Methods, which could be implemented [optional]:

• getPriority(event) - Registered async processor can vote about priority of processing for given event. Returns null by default ⇒ processor doesn't vote about priority - preserve original event priority. Use IMMEDIATE to execute whole event synchronously. All registered processors votes about event priority - whole event will be processed with the highest priority.

Transactional processing is controlled before the event publishing itself - the whole processing now takes place in a one transaction and all processors run synchronously by default. In case of an error in any processor, the whole transaction is rolled back, which has some advantages:

• simple adding of validation or referential integrity
• repeating the whole chain

and disadvantages as well:

• having to catch all the exceptions properly to avoid "breaking the chain"
• saving logs and archives in the new transaction (Propagation.REQUIRES_NEW)

CzechIdM 8.0.0 brings new feature - asynchronous event processing. New event type NOTIFY was added, all previous events (CREATE, UPDATE, DELETE, EAV_SAVE etc.) are still synchronous.

Asynchronous NOTIFY event is published for dtos:

• IdmIdentityDto - published, when identity is created or updated (or eav is saved, updated, deleted)
• IdmIdentityContractDto- published, when contract is created or updated (or eav is saved, updated, deleted)
• IdmContractGuaranteeDto- published, when contract guarantee is created or updated (or eav is saved, updated, deleted)
• IdmIdentityRoleDto- published, when identity role is created or updated (or eav is saved, updated, deleted)
• IdmRoleDto- published, when role is created or updated (or eav is saved, updated, deleted)
• IdmTreeNodeDto- published, when tree node is created or updated (or eav is saved, updated, deleted)
• IdmRoleCatalogueDto- published, when identity role catalogue is created or updated (doesn't have eav attributes)

Other entities will be added soon, when new asynchronous entity event processors will be implemented.

NOTIFY event type is processed asynchronously:

• order 500 - by automatic roles in core module,
• order 1000 - by account management in acc module, then provisioning is executed.

When asynchronous event is published, it's persisted into event queue (IdmEntityEvent). Internal scheduled task executes events from queue - all registered processors for event type NOTIFY is processed - the same behavior as standard event processing, processors are called in defined order synchronously by defult. Event (~entity) states (IdmEntityState) are persisted during event is processed (created / running / failed). Successfully processed events are deleted from queue by processor EntityEventDeleteExecutedProcessor. When exception occurs, event stays in queue with appropriate result code. Event agenda is available under APP_ADMIN permission on frontend from audit menu (shortcut tab can be added on all entity details e.g. see identity detail).

Events from queue are processed by event owner id - one event for one owner can be executed in the same time ⇒ we need to preserven event order by created date for one owner. Super owner id (EntityEventManager.EVENTPROPERTYSUPEROWNERID) can be used for setting custom event owner - this property will be resolved for evaluating running events for the same owner concurrency.

Before event is persisted into queue, then event priority is evaluated, priority types:

• IMMEDIATE - immediate ~ synchronously. Event will be executed synchronously.
• HIGH - asynchronously (7 / 10 in one cycle)
• NORMAL - asynchronously (3 / 10 in one cycle)

Events are processed from queue by internal scheduled tasks by priority. Events with HIGH priority will use 7 slots, events with NORMAL priority will use 3 slots ⇒ events will be processed 7 / 3, when internal scheduled task for processing events will be executed.

Priority can be set to event manually or registered processors can vote about event's priority - see AsyncEntityEventProcessor - the highest priority is used.

Execute date can be set to event manually. Event with priority HIGH or NORMAL will be processed after given date. Can be used for events, which could be executed sometimes "in night".

Event can be published by another event ~ event chain (tree) is persisted. For example, when contract is saved, then contract NOTIFY event is published. This event is processed by provisioning processors - but only NOTIFY event with contract's identity is published here only. Provisioning is physically executed in other processor, which processes identity NOTIFY event.

When asynchronous event is published, then event content (and previous ~ original content) and event parameters is persisted into queue. This persisted attributes are used, when event is resurrected from queue and executed. Attributes are available in registered asynchronous processors - evaluate modifications, skipping by event parameter value etc. can be implemented in processors business logic.

When internal scheduled task for executing event from queue is processed, then duplicate events are removed. Duplicate event is event for the same owner with the same event type and properties. Older duplicate events are removed - the newest event is used. Events are processed by priority in bulk, default bulk size is 100 events ⇒ duplicates are removed only in this bulk (not configurable for now, see future development). Bulk size is designed this way, because events are processed by priority - event with HIGH priority should not wait too long for another bulk is begin to process. Remove duplicates should be redesigned from scratch - remove duplicates through whole queue.

Persist event (~entity) state, when event is processed. State can be persisted manually, even without event processing. This state will be shown on entity detail soon (new frontend component).

Notification about registered asynchronous processors is prepared, when asynchronous event is published. Notification is send into topic core:event - uses console log by default and is send to currently logged identity - e.g. identity is saved, but provisioning will be executed asynchronously. Localization for asynchronous processors was added on frontend (see key acc:processor.identity-save-processor).

• Scheduler - configure internal scheduled tasks for processing events from queue.
• Event processing - configure asynchronous event processing
• Processors - configure entity event processors.

• 0 - basic / core functionality - operation save, delete etc.
• 100 - automatic roles computation.
• 1000 - after save - e.g. sends notifications.
• 10000 - publish NOTIFY event about entity is changed.
• -1000 - before delete provisioning (before identity role is deleted).
• Identity:
  • -2000 - validate password in acc module - checks all system password policies and idm default policy ⇒ all policies are evaluated in one request. If acc module is enabled, then core password validation processor can be disabled.
  • -1000 - validate password in core module - checks idm default policy.
  • 100 - persist password
• Contract:
  • 100 - Automatic roles recount while identity contract is saved, updated or deleted / disabled.
  • 200 - Contract exclusion, end and enable.
• LongRunningTask:
  • 100 - execute scheduled long running tasks, which depends on currently ended scheduled task.
• Provisioning:
  • -5000 - check disabled system
  • -1000 - compute attributes for provisioning (read attribute values from target system)
  • -500 - check readonly system
  • 0 - execute provisioning (create / update / delete)
  • 1000 - execute after provisioning actions (e.g. sends notifications)
  • 5000 - archive processed provisioning operation.

Other orders can be found directly in application, see supported event types.

Processors can be configured through Configurable interface by standard application configuration.

##
## approve create automatic role
idm.sec.core.processor.role-tree-node-create-approve-processor.enabled=true
# wf definition 
idm.sec.core.processor.role-tree-node-create-approve-processor.wf=approve-create-automatic-role
##
## approve delete automatic role
idm.sec.core.processor.role-tree-node-delete-approve-processor.enabled=true
# wf definition 
idm.sec.core.processor.role-tree-node-delete-approve-processor.wf=approve-delete-automatic-role

• Check if defined fields on identity was changed. If yes, then send notification.
• As default is used this system template identityMonitoredFieldsChanged.
• Extended attributes is not supported now.
• Order of processor is Integer.Max - 100. We want to send notification on end of chain (after identity is presisted or provisioning are completed).

# Identity changed monitored fields - Check if defined fields on identity was changed. If yes, then send notification.
# Default is disabled
idm.sec.core.processor.identity-monitored-fields-processor.enabled=false
# Monitored fields on change (for Identity, extended attributes are not supported)
idm.sec.core.processor.identity-monitored-fields-processor.monitoredFields=firstName, lastName
# Notification will be send to all identities with this role
idm.sec.core.processor.identity-monitored-fields-processor.recipientsRole=superAdminRole

• Name of processor "role-request-approval-processor".
• This process ensures the approval of the request for change premissions.

# Default is enabled
idm.sec.core.processor.role-request-approval-processor.enabled=true
# Workflow process for change permissions (as default is "approve-identity-change-permissions")
idm.sec.core.processor.role-request-approval-processor.wf=approve-identity-change-permissions

• The approving task will be assigned to all users with role Helpdesk.

# The role can be changed in the application configuration "idm.sec.core.wf.approval.helpdesk.role", the default setting is Helpdesk.
idm.sec.core.wf.approval.helpdesk.role=Helpdesk
# Default is disabled
idm.sec.core.wf.approval.helpdesk.enabled=false

• The approving task will be assigned to all users evaluated as the managers of the applicant. The manager is defined based on the industrial relations of the applicant.

# Default is disabled
idm.sec.core.wf.approval.manager.enabled=false

• The approving task will be assigned to all users with role Usermanager.

# The role can be changed in the application configuration "idm.sec.core.wf.approval.usermanager.role", the default setting is Usermanager.
idm.sec.core.wf.approval.usermanager.role=Usermanager
# Default is disabled
idm.sec.core.wf.approval.usermanager.enabled=false

##
## HR process - enable identity's contract process. The processes is started
## for contracts that are both valid (meaning validFrom and validTill and disabled state) and
## not excluded.
idm.sec.core.processor.identity-contract-enable-processor.enabled=true
# wf definition 
idm.sec.core.processor.identity-contract-enable-processor.wf=hrEnableContract
##
## HR process - end or delete of identity's contract process. The processes is started
## for contracts that are not valid (meaning validFrom and validTill or disabled by state) and deleted. 
## If the processed contract was the last valid contract of the identity, the identity is disabled.
## Additionally all added roles, which were assigned to the ended contract, are removed by the process.
idm.sec.core.processor.identity-contract-end-processor.enabled=true
# wf definition 
idm.sec.core.processor.identity-contract-end-processor.wf=hrEndContract
##
## HR process - identity's contract exclusion. The processes is started for
## contracts that are both valid (meaning validFrom and validTill) and excluded.
## If the processed contract was the last valid contract of the identity, the identity is disabled.
idm.sec.core.processor.identity-contract-exclusion-processor.enabled=true
# wf definition 
idm.sec.core.processor.identity-contract-exclusion-processor.wf=hrContractExclusion

• Provisioning after manually add, update or remove guarantee is controlled by these two processors: ContractGuaranteeSaveProvisioningProcessor and ContractGuaranteeDeleteProvisioningProcessor. Provisioning for update or create is done after success save entity, but provisioning for delete is done before delete entity. Both processors are enabled by default.

## Provisioning identity after add or update IdmContractGuaranteeDto
idm.sec.acc.processor.contract-guarantee-save.enabled=true
##
## Provisioning identity before IdmContractGuaranteeDto will be removed
idm.sec.acc.processor.contract-guarantee-delete.enabled=true

When some long running task ends, then END event is fired. This processor persists task's state.

When some long running task ends, then END event is fired. This processor executes scheduled long running tasks, which depends on currently ended scheduled task.

Processes GENERATE event type with RptReportDto content, order -1000. Generates output data for report by long running task.

## Enable / disable
idm.sec.rpt.processor.report-generate-processor.enabled=true

Processes GENERATE event type with RptReportDto content, order 0. Saves generated report metadata (binary data are stored as attachment).

## Enable / disable
idm.sec.rpt.processor.report-generate-end-processor.enabled=true

Processes GENERATE event type with RptReportDto content, order 1000. Sends notification after report is generated to report creator.

## Enable / disable
idm.sec.rpt.processor.report-generate-end-send-notification-processor.enabled=true

Processes UPDATE event type with IdmIdentityDto content, order 200. When identity starts to be valid and has at least one account on target system, then new password is generated and changed on all identity's accounts ⇒ identity ha the same password in all accounts. Notification is send (see acc:newPasswordAllSystems template) to identity about new password on which accounts.

Identity is starting, when their state is changed from CREATED, NO_CONTRACT, FUTURE_CONTRACT to the VALID state.

## Enable / disable
idm.sec.acc.processor.identity-set-password-processor.enabled=true

• Ecent content: IdmEntityEventDto
• Event type: EXECUTE
• Default order: -1000

Start execution of entity event.

## Enable / disable
idm.sec.core.processor.entity-event-start-processor.enabled=true

• Ecent content: IdmEntityEventDto
• Event type: EXECUTE
• Default order: 0

## Enable / disable
idm.sec.core.processor.entity-event-execute-processor.enabled=true

Execute entity event - resurrects entity event and process her - execute all registered processors.

• Ecent content: IdmEntityEventDto
• Event type: EXECUTE
• Default order: 1000

End execution of entity event - persist state only.

## Enable / disable
idm.sec.core.processor.entity-event-end-processor.enabled=true

• Ecent content: IdmEntityEventDto
• Event type: EXECUTE
• Default order: 5000

Delete successfully executed entity events.

## Enable / disable
idm.sec.core.processor.entity-event-delete-executed-processor.enabled=true

If we want to get hooked after updating the identity, we should implement a processor to the event type IdentityEventType.UPDATE with an order number higher than 0:

@Enabled(ExampleModuleDescriptor.MODULE_ID)
@Component("exampleLogIdentityUpdateSyncProcessor")
@Description("Logs after identity is updated")
public class LogIdentityUpdateSyncProcessor
		extends CoreEventProcessor<IdmIdentityDto> 
		implements IdentityProcessor {
 
	/**
	 * Processor's identifier - has to be unique by module
	 */
	public static final String PROCESSOR_NAME = "log-identity-update-sync-processor";
	private static final org.slf4j.Logger LOG = org.slf4j.LoggerFactory
			.getLogger(LogIdentityUpdateSyncProcessor.class);
 
	public LogIdentityUpdateSyncProcessor() {
		// processing identity UPDATE event only
		super(IdentityEventType.UPDATE);
	}
 
	@Override
	public String getName() {
		// processor's identifier - has to be unique by module
		return PROCESSOR_NAME;
	}
 
	@Override
	public EventResult<IdmIdentityDto> process(EntityEvent<IdmIdentityDto> event) {
		// event content - identity
		IdmIdentityDto updateddIdentity = event.getContent();
		// log
		LOG.info("Identity [{},{}] was updated.", updateddIdentity.getUsername(), updateddIdentity.getId());
		// result
		return new DefaultEventResult<>(event, this);
	}
 
	@Override
	public int getOrder() {
		// right after identity update
		return CoreEvent.DEFAULT_ORDER + 1;
	}
}

If we want to implement the same feature as above but asynchronously, we can process asynchronous IdentityEventType.NOTIFY instead IdentityEventType.UPDATE. When we need to change synchronous processors to asynchronous, we can simply change processed event type and add some condition, when only some original event types has to be processed ⇒ asynchronous NOTIFY event type is published for CREATE, UPDATE and EAV_SAVE event types.

@Enabled(ExampleModuleDescriptor.MODULE_ID)
@Component("exampleLogIdentityUpdateAsyncProcessor")
@Description("Logs after identity is updated")
public class LogIdentityUpdateAsyncProcessor
		extends CoreEventProcessor<IdmIdentityDto> 
		implements IdentityProcessor {
 
	/**
	 * Processor's identifier - has to be unique by module
	 */
	public static final String PROCESSOR_NAME = "log-identity-update-async-processor";
	private static final org.slf4j.Logger LOG = org.slf4j.LoggerFactory
			.getLogger(LogIdentityUpdateAsyncProcessor.class);
 
	public LogIdentityUpdateAsyncProcessor() {
		// processing identity NOTIFY event only
		super(IdentityEventType.NOTIFY);
	}
 
	@Override
	public boolean conditional(EntityEvent<IdmIdentityDto> event) {
		// we want to process original UPDATE event only
		// async NOTIFY event is published for CREATE, UPDATE, EAV_SAVE event types
		return super.conditional(event)
				&& IdentityEventType.UPDATE.name().equals(event.getProperties().get(EntityEventManager.EVENT_PROPERTY_PARENT_EVENT_TYPE));
	}
 
	@Override
	public String getName() {
		// processor's identifier - has to be unique by module
		return PROCESSOR_NAME;
	}
 
	@Override
	public EventResult<IdmIdentityDto> process(EntityEvent<IdmIdentityDto> event) {
		// event content - identity
		IdmIdentityDto updateddIdentity = event.getContent();
		// log
		LOG.info("Identity [{},{}] was updated.", updateddIdentity.getUsername(), updateddIdentity.getId());
		// result
		return new DefaultEventResult<>(event, this);
	}
 
	@Override
	public int getOrder() {
		// notify event has their own process line - we can use default order
		return CoreEvent.DEFAULT_ORDER;
	}
}

• Skip duplicate event in "the bigger window".
• Support to persist confidential event properties.
• Support retry failed events, truncate event queue
• Create FE component with entity state
• Notification about failed events (or report)
• Automatic role event doesn't propagate parent event id ⇒ processed by LRT.