7.4:dev:synchronization

Synchronization

The basic task of the synchronization is to ensure the correct state of the data on the end system and in IdM. Typically users´ accounts are considered as data. The correct state is defined both by the data in IdM (account management) and by the IdM configuration itself - usually by setting provisioning and synchronization on a given system.

Apart from accounts, it will also be possible to synchronize other types of entities (roles, trees, …). However, to make things more simple, synchronization is considered only between an account and an identity in this text.

The usual synchronization process is as follows:

  • Finding the changed accounts on the end system.
  • Iteration of the changed accounts and evaluation of the situation for each account (account in IdM).
  • Action performance for the situation found (e.g. creation of an identity in IdM).
  • Running the subsequent operations (e.g. provisioning updating an account on the end system).

During the synchronization process, the situation in which the account can be found based on the state in IdM is evaluated for every account. Basic synchronization configuration means settings the type of action which should be done in a given situation.

The synchronization situations are fixed and they are the following:

The situation when a corresponding account exists to a given account on the system (AccAccount) in IdM.

In this situation, it is possible to proceed to the following actions:

  • Update the entity - will update the identity linked to an account. The update is done based on mapping linked to the synchronization. At present, mapping enables to update the attributes of the identity itself, and also the attributes saved in the warehouse of advanced attributes as well as in coded warehouse. After saving an entity, standard provisioning will always be initiated.
  • Update account - will initiate standard provisioning. Synchronization initiates an event, it does not do the provisioning itself.
  • Remove link - will remove the link in IdM. meaning it will remove the account (AccAccount) and the links between the identity and the account (AccIdentityAccount). It does not modify the identity itself, does not initiate provisioning.
  • Remove link and related role - will remove the links like in the previous case but will also remove the identity roles which are linked to the corresponding (AccIdentityAccount). In other words, it will remove the roles which assigned a given account to the identity.
  • Ignore - This action does not perform any active operation.

Situation in which there is no link to a given account on the system (account in IdM), but an identity exists.

Since the link does not exist, in this case the identity has been found through a correlation attribute. A correlation attribute can be any attribute from the related synchronization mapping (the correlation attribute is mandatory).

At present, the correlation attribute enables searching according to identity attributes (username, firstName, lastName, email). Searching according to advanced attributes has not been implemented yet!

For example, if you want to find (identify) identities in IdM based on the correspondence of the user name username and the account attribute login, you can use the following correlation attribute:


In this situation, it is possible to proceed to the following actions:

  • Create link - will create a link in IdM, meaning it will create an account (AccAccount) and the link between the identity and the account (AccIdentityAccount). It does not modify the identity itself, does not initiate provisioning.
  • Create link and update account - The link is created in the same way like in the previous case. In addition, the account is updated on the end system, meaning that the event for running the provisioning is initiated.
  • Ignore - This action does not perform any active operation.

Situation in which there is no identity in IdM to a given account on the system.

In this situation, it is possible to proceed to the following actions:

  • Create entity - will create an identity and a link in IdM. The creation will be done according to the mapping set up in synchronization. Creation of an entity will initiate provisioning (account update).
  • Ignore - This action does not perform any active operation.

Situation in which there is no account on the end system to a given account in IdM.

This situation may occur in case the connector supports the operation DELETE. This means that the connector is able to give information on which accounts have been deleted on the end system since the last synchronization. Yet this situation can be typically used in reconciliation when all the accounts in IdM are iterated overnight, verifying if an account exists on the end system - if it doesn´t a preset action is initiated.

Although the response to the DELETE state has been implemented in synchronization, most connectors do not support this operation! The operation will not be available any more once you use own filter

In this situation, it is possible to proceed to the following actions:

  • Create account - Synchronization will only initiate the event for the linked entity IdentityEventType.UPDATE that will initiate provisioning and so the account will be created on the end system.
    • Delete entity - will delete the account in IdM, the link between the account and the identity, and the identity
  • Remove link - will remove the link in IdM, meaning it will remove the account (AccAccount) and the links between the identity and the account (AccIdentityAccount). It does not modify the identity itself, does not initiate provisioning.
  • Remove link and related role - will remove the links like in the previous case but will also remove the identity roles which are linked to the corresponding (AccIdentityAccount). In other words, it will remove the roles which assigned a given account to the identity.
  • Ignore - This action does not perform any active operation.

    Synchronization supports two basic modes of searching the accounts suitable for synchronization. The first mode is the use of synchronization mechanism which is provided directly by the connector. In such case, the method IcConnectorFacade.synchronization is called on the connector after synchronization.

The input of this method is the token defining where the synchronization should continue. In this case, the format of the token is defined directly by the connector (it can be a time mark, the order, …) If the token is empty, then the synchronization will be launched for all the accounts of the system.

The disadvantage of the synchronization in the connector is the impossibility to rule in details which of the accounts you want to synchronize (put together more complicated queries). On the contrary, the advantage is the possibility to use the operation DELETE . I. e. the situation in which the connector is able to report on its own which account has been deleted.

In case of use of the internal synchronization in Database Table Connector (connId), it is necessary to set up a column in its configuration, according to which the search will be made (Change Log Column (Sync)). However, if then you wish to synchronize using the own filter (according to the same column), the filter will not work (nothing will be returned). In such case, it necessary to remove the settings from the configuration!

The second way is theOwn filter. This mode will compose the filter criterion first. This criterion is used to search account on the end system (IcConnectorFacade.search). The synchronization takes places above these results.

This mode can be activated (Use own filter) and set on the Filter folder. The own filter can be simply defined by choosing the attribute (Filter by attribute) by which you want to search, and the corresponding operation (Filtering operation).

To save the final token, it is necessary to determine the account attribute which contains it. To do this, you may use the item Token is contained in this attribute.

In case you need to create a more complicated filter criterion, it is possible to use this script. In the example below, you can find the situation when you want to filter by the chosen attribute and operation (the filter enters as variable into the script), but at the same time, you want to limit the result with one more condition. In this case, the condition is that all the results must have the attribute "lastname" equivalent to the "Doe" value.

import  eu.bcvsolutions.idm.ic.filter.impl.IcFilterBuilder;
import  eu.bcvsolutions.idm.ic.api.IcAttribute;
import  eu.bcvsolutions.idm.ic.impl.IcAttributeImpl;
 
IcAttribute attr = new IcAttributeImpl("lastname", "Doe");
 
return IcFilterBuilder.and(filter, IcFilterBuilder.equalTo(attr));

IcFilterBuilder provides the following operations:

  • IcFilterBuilder.equalTo(…)
  • IcFilterBuilder.contains(…)
  • IcFilterBuilder.startsWith(…)
  • IcFilterBuilder.endsWith(…)
  • IcFilterBuilder.lessThan(…)
  • IcFilterBuilder.greaterThan(…)
  • IcFilterBuilder.or(…)
  • IcFilterBuilder.and(…)
  • IcFilterBuilder.not(…)

The output of this script must be an object of the type IcFilter. If the output is null, then the filter will not be applied and synchronization will be launched over all the accounts.

In default state, synchronization can administer the identities (create, update, delete). Yet if you need a more complex solution, e.g. when you need to establish industrial relations, the standard synchronization mechanisms won´t be sufficient.

The solution is to use the own workflow which will perform the required operation (e.g. the creation of industrial relations mentioned above). The workflow can be set up separately for each situation.

If the synchronization finds out that workflow is set up in a given situation, it will launch it. In such case, the synchronization does not do anything with the given account anymore, i.e. all the active operations are performed by the workflow.

The workflow which should be possible to be used in synchronization, must satisfy several criteria. The first one is the category in which the workflow is included.

The only definitions of workflow that will be offered will be those appearing in the category eu.bcvsolutions.sync.action.

For synchronization, the workflow further expects the following input variables:

- uid (String),
- entityType (enum SystemEntityType),
+ icAttributes (List of IcAttribute),
- syncConfigId (UUID for SysSyncConfig),
- actionType (String)
- situation(String),
+ accountId(UUID),
+ entityId(UUID)

An example of how such workflow may look like is the following demonstration process (syncActionExampl). This process will establish an approval task for the administrator (admin) for every account of which the UID starts with test. All the other accounts will go through standard implementation according to the evaluated situation and the set-up action. The workflow is also showing that only one process resolving all the situations can exist.

The synchronization logs are divided into three parts:

  • SysSyncLog - The main log which is formed after the start of the synchronization. It contains the time of start, time of end, and the log for the whole process. It also contains the information if the synchronization has been launched and if some error occurred.
  • SysSyncActionLog - is linked to the main log, categorizing the individual elements according to the action performed. It also contains the number of the iterations performed for each action. The log may attain the following types of actions:
        CREATE_ENTITY,
	UPDATE_ENTITY, 
	DELETE_ENTITY,
	LINK_AND_UPDATE_ACCOUNT, //create link and produce entity save event
	LINK,
	UNLINK,
	UNLINK_AND_REMOVE_ROLE,
	CREATE_ACCOUNT, // produce only entity save event
	UPDATE_ACCOUNT, // produce only entity save event
	LINKED, // situation (for IGNORE and WF sort)
	MISSING_ENTITY, // situation (for IGNORE and WF sort)
	UNLINKED, // situation (for IGNORE and WF sort)
	MISSING_ACCOUNT, // situation (for IGNORE and WF sort)
	IGNORE;

Each of these types may attain the following states:

 	SUCCESS,
	ERROR,
	WARNING,
	IGNORE,
	WF
  • SysSyncItemLog - is linked to the actions log. It records the course of the synchronization of each element. It contains the identifier and the name of the element.

Synchronization makes use of the standard mechanism of events in IdM, adding three new events:

  • SysSyncConfig.START
  • SysSyncConfig.CANCEL
  • SysSyncItemLog.START_ITEM

and three new processors:

For the scheduled activation of the synchronization, the task SynchronizationSchedulableTaskExecutor has been created which has the uuid as an input parameter of the configuration of the running synchronization.

For now the dynamic form for task scheduling does not support other wording than a simple text field.

The task will read the synchronization configuration according to the entered uuid, and will launch it in the same way like a direct launching from the overview of the configured synchronization configurations on the system detail.

After the launching, it is possible to observe and possibly stop the running synchronization from the overview of running tasks. When the task is finished, the final log can be seen not only in the synchronization agenda on the system, but also in the overview of all tasks.