User handling in the Cool Data Mover

Most Outsystems applications refer to Users in some way or another. Very typically by attributes like CreatedBy and LastModifiedBy. During data migration we must decide how to handle these references and the actual user data.
In this article we will explain how the Cool Data Mover can assist you in the handling of user data.
But first we will start with a short explanation on how users are handled in OutSystems. A good understanding of that helps in deciding how users are handled in the Cool Data Mover.

How users are handled in OutSystems

In OutSystems, users are registered by default in the system entity User. Each user has a username and password which are used as logon credentials. A user is defined within a so-called Tenant (which itself is defined within an espace/module). Even if you don’t use the multi-tenancy option in OutSystems, there still is a default tenant which is called ‘Users’. The idea is that a user has a unique username within the tenant. But there can be users from different tenants with the same username.

The User entity provides a view on the physical table OSSYS_USER. This table contains all users from all tenants. The User entity filters the data and only shows the users from the current tenant. Outsystems also provides an entity called MTUser which does not filter the data and shows all users from all tenants. Including the IT-users (from the standard ServiceCenter tenant) with which developers log in to Service Studio.
The standard users application (available on http:///users) provides screens to maintain users and their authorizations. This application will make sure that within the current tenant no two users can have the same username.

However, there are other ways to create users outside of the standard users application, for instance programmatically via the OutSystems entity action CreateOrUpdate_User. Which makes it possible (although incorrect) to create a user with the same username within the same tenant. This means we would have two the same, indistinguishable, users. This is an oversight of OutSystems of many years that preferably needs to be fixed within the database but that can not be corrected anymore since to many customers already have the incorrect situation.

User migration considerations

When data is migrated/moved from one environment to another, the (references to the) User entity need to be handled. Regardless of the solution, it is important to first determine what the strategy will be. Question to ask yourself are:

• Do we have uniquely identifiable users? Do duplicate users exist in our environments?
  Experience has shown that a substantial part of OutSystems customers have one or more duplicate users. This is not immediately visible, but the Cool Data Model Info React tool can help you identify these users. It is available on the Forge at https://www.outsystems.com/forge/component-overview/18855/cool-data-model-info-react-o11.
• Do we want to move the users along with the application data?
  It might be that you want to migrate all users or just the users that are referenced from the application data, or even only the users that have certain roles assigned to them.
• How do we want users to be migrated to avoid privacy issues?
  Frequently, organisations use an email address as the username. This allows the user to log in using their well-known email address. But since the email address can contain the name of the user/employee/customer it also is a potential privacy issue when migrating user data.

Identifying users

As explained in this article about matching, entities can not be matched/identified across environments by their technical id. That means we need to specify non-identifier match attributes.

In case of the User entity, the advised manner to identify them is shown below. It requires three entities to be included in the package. The match attributes for User are Tenant_Id and Username. This requires the Tenant entity to be included as well. The Tenant entity is identified by its Name and Espace_id. This requires the Espace entity to be included. An Espace is identified by its SS_Key attribute.

The Cool Data Mover will automatically set the preferred match attributes when including one these entities. However, if for some reason you want to use different match attributes, you can change them. If unsure of the consequences, please reach out to us and we’ll be glad to provide you with advice.

One example of this is when you want to anonymize/scramble the name and username and therefore cannot use those attributes as matching attributes. In that case an option would be to use the external_id attribute as a matching attribute. Of course, you will need to make sure that the external_id is populated and contains a value unique to the user.

Moving users separately

In some use cases it is required to migrate users separately. For instance, when we want to migrate an existing OnPrem environment to a new Cloud environment (or vice-versa) and migrate the users first. Or when we want to be sure that all users on Dev are also present on Test.

Users can be migrated separately by simply creating a package containing only the User, Tenant en Espace entities from the ServiceCenter module and marking the User entity as ‘Move’ instead of ‘Lookup’ in the import package.

Of course, the same applies when we only want to migrate the tenants. Just create a package only containing the Tenant and Espace entity and mark the Tenant entity as ‘Move’.

In both scenario’s it might be worth while to be able to specify which users and/or tenants to move. Use one of the three filtering options of the Cool Data Mover to limit the number of records exported. So, apply these filters in the export package. Filtering is described in detail in this article.

Moving referenced users

Frequently, we want to move the users that are referenced from our application data. Just to make sure that we can import the data on the destination side, and that we encounter no referential integrity issues.

One possible ‘solution’ is to NOT migrate the users. This can be done by checking the ‘skip’ checkbox. Of course, this is only possible when the attributes that reference the User entity are either optional or have a ‘Ignore’ delete rule. Those attributes will have a NullIdentifier() value after migration. Your application will need to be able to deal with that.

A more likely scenario is that users DO need to be migrated. For this the Cool Data Mover offer the ‘Move referenced users’ option on the setting tab of the package. When checked it will include the User, Tenant and Espace entity in the package and make sure that during the moves the referenced users are exported and imported. Optionally, the authorizations of those users (either directly or indirectly via a group) can be migrated as well. This will setup some additional entities. To avoid migration of all authorizations (as there might be many) of those users, select the roles for which the related authorisation need to be migrated.

User mapping

In some scenarios you might want to migrate the application data but not all referred (mandatory) users. For this the Cool Data Mover offers the user mapping option which can be set in the Import settings of the package as show below.

The user mapping option provides the ability to map individual users (taken from the export) to the users defined on the import environment.
As an example, suppose we want to move data from Production to Acceptance and take the referenced users along. First, export the data including the referenced users from Production. Secondly, in Acceptance, check the ‘apply user mapping’ option and click the ‘Manage user mapping’ link. This will bring up a screen in which the users exported from Production can be mapped to the users locally defined in Acceptance (see screenshot below).
During import this mapping will be applied making it possible to import data using mapped users instead of the exported ones.