Semarchy xDM developer's guide
This article helps you find some useful content using the knowledge base full search. It is a copy of the official's guide.
Please go to the official Semarchy xDM documentation for more self-investigation on your problem.
Welcome to Semarchy xDM.
This guide contains information about using the product to design and develop an MDM project.
APPENDICES
Appendix A: Component Types and Properties
A component type is a UI component used to display or author a form field or table column.
Component Types Reference
The following table lists the component types available, their supported datatypes and behavior in the browsing and authoring experience.
Component | Supported Datatypes | Browsing Behavior | Authoring Behavior |
Chips | List of Values (Multiple)[1] | Displays the values as a list of chips | Allows selecting multiple values using an auto-complete picker. |
Checkbox | Boolean[1] | Displays a disabled checkbox or switch | Checkbox or switch for authoring. |
Date Picker | Displays the value formatted per the user preferences. | Supports specific date and time picker component. | |
Hyperlink | String, LongText, Binary[1] | Displays a hyperlink to the URL or binary content, opened in a given target. | Allows authoring the URL or uploading binary content. |
ID | ID[1] | Displays the record ID in text format. | Accepts formatted or unformatted ID values. |
Image | String, LongText, Binary | Embeds binary content or URL as an image. | Supports image content upload or URL authoring. |
Markdown | String, LongText | Renders a text containing Markdown content. | Support authoring markdown text content with a rendering preview. |
Menu | String, List of Values[1] | Displays the value (LOV Label) as unformatted text | Allows selecting a value from a drop down menu. |
Object | String, LongText, Binary | Embeds binary content or URL in an iFrame. | Supports binary content upload or URL authoring. |
Reference | Reference (FDN)[1] | Displays the referenced record as a hyperlink text or chip | Allows selecting the reference using an auto-complete picker. |
Text | UUID[1], String[1], LongText[1], Boolean, Decimal[1], ByteInteger[1], Integer[1], LongInt[1], Date, TimeStamp | Displays the value formatted per the user preferences. | Supports multi-line text and editing format for numbers and dates. |
[1]The component type is used by default for this data type.
Common Properties Reference
The following properties are available for fields and/or columns regardless of the component type selected.
| Component | Tab | Property | Value | Default Value | Description |
|---|---|---|---|---|---|
<common> | General | Read-Only[1] | Boolean |
| Defines when the current component should be set to read-only in an authoring form. |
<common> | Label | Helper Text[1] | String | Helper text displayed under the editing component in an authoring form. | |
<common> | Label | Icon[2] | Icon used instead of the label in a form. | ||
<common> | Label | Label Position[2] |
|
| Position of the label in a browsing form. |
<common> | Label | Label Visible[3] | Boolean |
| Defines whether the label is visible. |
<common> | Label | Use Floating Label[1] | Boolean |
| Whether to use the floating label in an authoring form, if the component supports it. |
<common> | Style | Background Color | For tables, background color of the table cell. For forms, background color of the component. | ||
<common> | Style | Column Alignment[3] |
| Depends | For tables only: Column header and content alignment. Default alignment depends on the column content. |
<common> | Style | Header Color[3] | For tables only: Column header text color. | ||
<common> | Style | Icon Color[2] | Icon color when an icon is used. | ||
<common> | Style | Label Color[2] | Text color for the label in a form. | ||
<common> | Style | Label Text Alignment[2] |
|
| Label text alignment in a browsing form. |
<common> | Style | Label Typography[2] |
|
| Typography of the label a form. |
<common> | Style | Text Alignment[2] |
|
| Component’s text alignment in a form. |
<common> | Style | Text Color | Component’s text color in a form or table. | ||
<common> | Style | Text Typography[2] |
|
| Text typography in forms. Tables do not allow typography. |
[1]This property only applies to fields in forms in authoring mode.
[2]This property only applies to fields in forms.
[3]This property only applies to table columns.
Component Specific Properties Reference
The following properties are specific to component types.
| Component | Tab | Property | Value | Default Value | Description |
|---|---|---|---|---|---|
Checkbox | Format | Use Switch[1] | Boolean |
| Displays a switch instead of a regular check-box. |
Chips | Format | Display Avatar | Boolean |
| Displays the avatar in the chip. |
Chips | Format | Display Secondary Text | Boolean |
| Displays the secondary text in the chip, between brackets. |
Chips | Format | Max Width | String | Maximum horizontal size in pixels of each chip in the list. | |
Date Picker | Format | Display Format | String | Date and time formatting pattern. SeeDisplay and Editing Formatsfor more information. | |
Date Picker | Format | Editing Format[1] | String | Date and time data entry pattern. SeeDisplay and Editing Formatsfor more information. | |
Date Picker | Format | Use Time Picker[1] | Boolean |
| Whether the date picker component should also display a time picker, for timestamps only. For dates, the component never displays the time picker. |
Date Picker | Data | Min. Date | Date | Minimum date for the date picker (inclusive) | |
Date Picker | Data | Max. Date | Date | Maximum date for the date picker (inclusive) | |
Hyperlink | Data | Display Text | String | <value> | Text displayed for the hyperlink value. The <display-string>|<url> syntax is also supported in the URL whenLink Sourceis set to url. If no display test is specified the URL is displayed. |
Hyperlink | Data | Link Source |
| Depends. | Defines the type of the value, whether it is a binary content or an URL in a string. Defaults to content for binary value, and to URL otherwise. |
Hyperlink | Format | Link Target |
|
| Navigation target for the hyperlink. |
ID | Format | Display Format | String | Display format pattern that can be used to format the number if ID is a number. SeeDisplay and Editing Formatsfor more information. | |
Image | Data | Fallback Image | Binary or String. SeeImagesfor more information. | Fallback image when the image value is empty. The image type is declared in theFallback Image Source. | |
Image | Data | Fallback Image Source |
|
| Defines the type of the fallback-image expression. |
Image | Data | Source |
| Depends | Defines the type for this image value. Defaults to content the data type is binary, URL otherwise. |
Image | Format | Full Size on Click | Boolean |
| Open the image in full size when it is clicked. |
Image | Format | Height | String |
| Height of the image container in pixels. |
Image | Style | Image Alignment |
|
| Defines how the image is aligned in its container. |
Image | Style | Image Resize Mode |
|
| Defines how the image is resized in its container. |
Markdown | Format | Char Counter[1] | Boolean |
| Defines whether a character counter is displayed under the markdown component when authoring. |
Markdown | Format | Height | String |
| Height of the markdown component in pixels. |
Markdown | Format | Max Width | String | Maximum width of the markdown component, in pixels. If no value is specified, all the horizontal space is used. | |
Menu | Data | List Items[1] | String | Semicolon-separated list of list items, used when theList Typeis set to manual-list. This list contains only codes. The value displayed and selected is stored. | |
Menu | Data | List Type |
| Depends | Source of the values in the menu: Either the underlying list of value, or a list of values provided in theList Itemsproperty. It defaults to |
Menu | Format | Display Format | String | Display format pattern used to format the text displayed for a list of values. SeeDisplay and Editing Formatsfor more information. | |
Menu | Format | List Sort |
|
| Defines how the list is sorted. Only whenList Typeis |
Menu | Format | Display Avatar | Boolean |
| Displays the LOV values' images as avatars for the menu items. |
Object | Data | Source Type |
|
| Defines whether the value contains the content to display or an URL to the content. |
Object | Format | Height | String |
| Height of the object component in pixels. |
Object | Format | Max Width | String | Maximum width of the object component, in pixels. If no value is specified, all the horizontal space is used. | |
Reference | Format | Display Card | Display Card | Display card to use to display this reference as a chip in a form or in the auto-complete component when authoring. | |
Reference | Format | Max Width | String | Maximum horizontal size in pixels when this reference is displayed as a chip. Ignored if theDisplay Typeis hyperlink. | |
Reference | Style | Display Avatar | Boolean |
| Display the avatar in the chip. Ignored if theDisplay Typeis hyperlink. |
Reference | Style | Display Secondary Text | Boolean |
| Display the secondary text in the chip, between brackets. Ignored if theDisplay Typeis hyperlink. |
Reference | Style | Display Type |
|
| Defines how to display the reference, as a hyperlink showing the display card primary text, or a chip. |
Text | Format | Char Counter[1] | Boolean |
| Defines whether a character counter is displayed under the text component when authoring. |
Text | Format | Display Format | String | Display format pattern for numbers and dates. SeeDisplay and Editing Formatsfor more information. | |
Text | Format | Editing Format[1] | String | Data entry format pattern used as a helper when authoring. SeeDisplay and Editing Formatsfor more information. | |
Text | Format | Max Lines[1] | Number |
| Maximum number of lines for a multi-line component when authoring. |
Text | Format | Min Lines[1] | Number |
| Initial number of lines for a multi-line component when authoring. |
Text | Format | Multi-Line[1] | Boolean |
| Defines whether to use a multi-line component when authoring. |
[1]This property only applies for forms in authoring.
[2]This property is not used when authoring.
Display and Editing Formats
Display Format
Dates
For dates, the display format uses theSimple Date Formatpatterns, with the following exceptions:
The following Simple Date Format pattern letters are not supported (E, F, K, W, z, Z).
Texts and IDs
For texts and IDs, the display format uses the following patterns elements for numeric values:
0: A digit that must be displayed, after the decimal separator, the number of zeros indicates the number of decimals.+: '+' or '-' sign,: The thousand separator.: The decimal separator
The thousand and decimal separators used in the pattern (that is,and.) will appear in the application for each user according to the user settings. Although users may choose in their settings different separator characters to suit their locale and regional preferences, the patterns are always defined at design-time with the,and.characters. |
In these examples, the results displayed are for user who has a thousand separator set to,and a decimal separator set to.in his preferences.
| Number | Format | Displayed Result |
|---|---|---|
10000 | 0,0.0000 | 10,000.0000 |
10000.25 | 0,0 | 10,000 |
10000.25 | +0,0 | +10,000 |
-10000 | 0,0.0 | -10,000.0 |
10000.1254 | 0.000 | 10000.125 |
10000.1254 | 0[.]00000 | 10000.12540 |
-10000 | (0,0.0000) | (10,000.0000) |
0.25 | 0.00000 | 0.25000 |
0.25 | 0.0[0000] | 0.25 |
List of Values
For list of values, the display format supports thecodeandlabeltokens, which are replaced by the code and label of the list of value item. For example, if you specify thecode // labelpattern for agenderlist of value, the values will appear asF // Female,M // Male, etc.
Editing Format
Dates
For dates, the editing format property supports the same pattern as the display format, and this format is required strictly when inputing the date.
Texts and IDs
For texts and IDs, the editing format property supports the same pattern as the display format, but this pattern is only used to display the original number value. When the user enters values, only the decimal separator (as specified in the user preferences) and sign are taken into account and other characters are ignored.
List of Values
For list of values in menus, the display format supports thecodeandlabeltokens, which are replaced by the code and label of the list of value items in the selection menu.
Appendix B: Stepper Events Reference
This appendix lists the events triggered into steppers, grouped by User Action and ordered by their execution position. The validations running between those events are also indicated.
User opens a stepper
Stepper - Start, as defined in the stepper.
Step - Enter (Once), as defined in the first step displayed.
Step - Enter (Each), as defined in the first step displayed.
User finishes a stepper by clickingFINISH
Step - Exit (Pre validation - Once), as defined in the current step.
Step - Exit (Pre validation - Each), as defined in the current step.
Validation - Step Exit Validations, as defined in the current step.
Step - Exit (Post validation - Each), as defined in the current step.
Step - Exit (Post validation - Once), as defined in the current step.
Step - Close (Once), as defined in the current step.
Step - Close (Each), as defined in the current step.
Collection Step - Close Child (Once), as defined in the parent collection step. (if any)
Collection Step - Close Child (Each), as defined in the parent collection step. (if any)
Step - Exit (Pre validation - Once), as defined in the parent collection step. (if any)
Step - Exit (Pre validation - Each), as defined in the parent collection step. (if any)
Validation - Step Exit Validations, as defined in the parent collection step. (if any)
Step - Exit (Post validation - Each), as defined in the parent collection step. (if any)
Step - Exit (Post validation - Once), as defined in the parent collection step. (if any)
Stepper - Finish (Pre Validation), as defined in the stepper.
Validation - Stepper Finish Validations, as defined in the stepper.
Stepper - Finish (Post Validation), as defined in the stepper.
User closes a stepper withCLOSEthenDISCARD ALL
Stepper - Close (Discard), as defined in the stepper.
User closes a stepper withCLOSEthenSAVE FOR LATER
Stepper - Close (Saved), as defined in the stepper.
User clicksBACKon any step
Step - Back (Once), as defined in the current step.
Step - Back (Each), as defined in the current step.
Step - Enter (Once), as defined in the previous step.
Step - Enter (Each), as defined in the previous step.
User clicksPREVIOUS,CONTINUEor a step in the header, from any step
Step - Exit (Pre validation - Once), as defined in the current step.
Step - Exit (Pre validation - Each), as defined in the current step.
Validation - Step Exit Validations, as defined in the current step.
Step - Exit (Post validation - Each), as defined in the current step.
Step - Exit (Post validation - Once), as defined in the current step.
Step - Enter (Once), as defined in the previous or next step.
Step - Enter (Each), as defined in the previous or next step.
User clicksCONTINUEon any step
Step - Exit (Pre validation - Once), as defined in the current step.
Step - Exit (Pre validation - Each), as defined in the current step.
Validation - Step Exit Validations, as defined in the current step.
Step - Exit (Post validation - Each), as defined in the current step.
Step - Exit (Post validation - Once), as defined in the current step.
Step - Continue (Once), as defined in the current step.
Step - Continue (Each), as defined in the current step.
Step - Enter (Once), as defined in the next step.
Step - Enter (Each), as defined in the next step.
User clicksCLOSEon a step within a parent collection step
Step - Exit (Pre validation - Once), as defined in the current step.
Step - Exit (Pre validation - Each), as defined in the current step.
Validation - Step Exit Validations, as defined in the current step.
Step - Exit (Post validation - Each), as defined in the current step.
Step - Exit (Post validation - Once), as defined in the current step.
Step - Close (Once), as defined in the current step.
Step - Close (Each), as defined in the current step.
Collection Step - Close Child (Once), as defined in the parent collection step.
Collection Step - Close Child (Each), as defined in the parent collection step.
User clicksADD ANOTHERon a step within a parent collection step
Step - Exit (Pre validation - Once), as defined in the current step.
Step - Exit (Pre validation - Each), as defined in the current step.
Validation - Step Exit Validations, as defined in the current step.
Step - Exit (Post validation - Each), as defined in the current step.
Step - Exit (Post validation - Once), as defined in the current step.
Collection Step - Add Child, as defined in the parent collection step.
Step - Enter (Once), as defined in the first step displayed of the parent collection step.
Step - Enter (Each), as defined in the first step displayed of the parent collection step.
User clicksEDIT NEXTon a step within a parent collection step
Step - Exit (Pre validation - Once), as defined in the current step.
Step - Exit (Pre validation - Each), as defined in the current step.
Validation - Step Exit Validations, as defined in the current step.
Step - Exit (Post validation - Each), as defined in the current step.
Step - Exit (Post validation - Once), as defined in the current step.
Collection Step - Edit Child (Once), as defined in the parent collection step.
Collection Step - Edit Child (Each), as defined in the parent collection step.
Step - Enter (Once), as defined in the first step displayed of the parent collection step.
Step - Enter (Each), as defined in the first step displayed of the parent collection step.
User clicksCREATEon a collection step
Collection Step - Add Child, as defined in the current collection step.
Step - Enter (Once), as defined in the first child step displayed.
Step - Enter (Each), as defined in the first child step displayed.
User clicksEDITon a collection step
Collection Step - Edit Child (Once), as defined in the current collection step.
Collection Step - Edit Child (Each), as defined in the current collection step.
Step - Enter (Once), as defined in the first child step displayed.
Step - Enter (Each), as defined in the first child step displayed.
User clicksREMOVEon a collection step
Collection Step - Remove Child (Once), as defined in the current collection step.
Collection Step - Remove Child (Each), as defined in the current collection step.
User clicksDELETEon a collection step
Collection Step - Remove Child (Once), as defined in the current collection step.
Collection Step - Remove Child (Each), as defined in the current collection step.
User applies aMASS-UPDATEon a collection step
Collection Step - Edit Child (Once), as defined in the current collection step.
Collection Step - Edit Child (Each), as defined in the current collection step.
User clicksCOPYon a collection step
Collection Step - Add Child, as defined in the current collection step.
Collection Step - Copy Child (Once), as defined in the current collection step.
Collection Step - Copy Child (Each), as defined in the current collection step.
Collection Step - Edit Child (Once), as defined in the current collection step.
Collection Step - Edit Child (Each), as defined in the current collection step.
Step - Enter (Once), as defined in the first child step displayed.
Step - Enter (Each), as defined in the first child step displayed.
User applies anIMPORTon a collection step
Collection Step - Import (Once), as defined in the current collection step.
Collection Step - Import (Each), as defined in the current collection step.
Appendix C: Workflow Notifications Variables
The following table lists the variables available in the workflow notifications for tasks and transitions.
| Variable Syntax | Description | Example | Tasks | Transitions |
|---|---|---|---|---|
| ID of the load (workflow instance) stored in the repository | 23442 | Yes | Yes |
| Name of the job executed at the end of the workflow | PRODUCT_ALL | Yes | Yes |
| Name of workflow defined at design-time | CreateProductWorkflow | Yes | Yes |
| Label of workflow instance (either the workflow design-time label or the label entered at runtime) | Create Products Workflow | Yes | Yes |
| Name of the administrator role defined for the workflow | SemarchyAdministrator | Yes | Yes |
| Email of the administrator role, if any | Yes | Yes | |
| Name of current task as defined at design-time | CreateItems | Yes | Yes |
| Label of current task as defined at design-time | Create Items | Yes | Yes |
| Name of the entity managed in the current task | Product | Yes | Yes |
| Label of the entity managed in the current task | Product | Yes | Yes |
| Number of root records for current task. | 2 | Yes | Yes |
| Name of the role the current task is assigned to | SupplyChain | Yes | Yes |
| Email of role the current task is assigned to | Yes | Yes | |
| Name of role who can assign/reassign the current task | SupplyChainRouter | Yes | Yes |
| Email of role who can assign/reassign the current task | Yes | Yes | |
| Name of current task assignee (the performer, when task is running) | Joe Celko | Yes | Yes |
| Email of current task assignee | Yes | Yes | |
| Name of current task assignee before an un-assign or re-assign operation | - | Yes | Yes |
| Email of current task assignee before un-assign or re-assign operation | - | Yes | Yes |
| Max pending duration in minutes for current task | - | Yes | Yes |
| Actual pending duration in minutes for current task | 1929 | Yes | Yes |
| Max running duration in minutes for current task | - | Yes | Yes |
| Actual running duration in minutes for current task | 12 | Yes | Yes |
| Max total duration in minutes for current task | - | Yes | Yes |
| Actual total duration in minutes for current task | 1941 | Yes | Yes |
| First start timestamp of current task | 2016-12-21 21:23:11 PT | Yes | Yes |
| Timestamp at which task was terminated | 2016-12-21 21:23:23 PT | Yes | Yes |
| Number of executions of current task | 1 | Yes | Yes |
| Label of the event that generated current message. Can be one of the following: Task Started, Task Suspended, Task Finished, Task Assigned, Task Unassigned, Task Reassigned, Task Max Pending Duration Reached, Task Max Running Duration Reached, Task Total Duration Reached, Transition | Transition | Yes | Yes |
| Name of the transition | RequestMoreInfo | No | Yes |
| Label of the transition | Request More Information | No | Yes |
| Comments entered by the user during the transition | Please approve. | No | Yes |
| Name of the previous task as defined at design-time | CreateProductData | No | Yes |
| Label of previous task as defined at design-time | Create Product Data | No | Yes |
| Name of the entity for the previous task | Product | No | Yes |
| Label of the entity for the previous task | Product | No | Yes |
| Number of root records for the previous task. | 2 | No | Yes |
| Name of role the previous task is assigned to | Corp Marketing | No | Yes |
| Email of role the previous task is assigned to | - | No | Yes |
| Name of role who can assign/reassign the previous task | Corp Marketing | No | Yes |
| Email of role who can assign/reassign the previous task | - | No | Yes |
| Name of the previous task’s assignee | Bob Marley | No | Yes |
| Email of the previous task assignee (the performer, when task is running) | No | Yes | |
| Max total duration in minutes for the previous task | - | No | Yes |
| Actual total duration in minutes for the previous task | 1941 | No | Yes |
| First start timestamp of previous task | 2016-12-21 21:23:11 PT | No | Yes |
| Timestamp at which previous task was terminated | 2016-12-21 21:23:23 PT | No | Yes |
| Number of executions of previous task | 1 | No | Yes |
| Name of the next task as defined at design-time | CreateProductData | No | Yes |
| Label of next task as defined at design-time | Create Product Data | No | Yes |
| Name of the entity for the next task | Product | No | Yes |
| Label of the entity for the next task | Product | No | Yes |
| Name of role the next task is assigned to | Corp Marketing | No | Yes |
| Email of role the next task is assigned to | - | No | Yes |
| Name of role who can assign/reassign the next task | Corp Marketing | No | Yes |
| Email of role who can assign/reassign the next task | - | No | Yes |
| Name of the next task’s assignee | Bob Marley | No | Yes |
| Email of the next task assignee (the performer, when task is running) | No | Yes | |
| URL allowing to view and interact with the workflow | Yes | Yes | |
| Base URL for the server | Yes | Yes | |
| Base URL for the current application | Yes | Yes | |
| Action button with URLs to interact with the task directly. | <div><a href=…..>View Workflow</a>…</div> | Yes | Yes |
PREFACE
Audience
Document Conventions
This document uses the following formatting conventions:
| Convention | Meaning |
|---|---|
boldface | Boldface type indicates graphical user interface elements associated with an action, or a product specific term or concept. |
italic | Italic type indicates special emphasis or placeholder variable that you need to provide. |
| Monospace type indicates code example, text or commands that you enter. |
Other Semarchy Resources
Obtaining Help
There are many ways to access the Semarchy Technical Support. You can call or email our global Technical Support Center (support@semarchy.com). For more information, seehttp://www.semarchy.com.
Feedback
We welcome your comments and suggestions on the quality and usefulness of this documentation.
Overview
This guide contains information about using the product to design and develop an MDM project.
Using this guide, you will learn how to:
Use the Semarchy Workbench to design and develop an MDM project.
Design the logical model representing your business entities.
Design the certification process for certifying golden data from data published from source systems.
Deploy the logical models and run the certification processes developed in Semarchy Workbench.
INTRODUCTION TO SEMARCHY XDM
Semarchy xDM is designed to support any kind of Enterprise Master Data Management initiative. It brings an extreme flexibility for defining and implementing master data models and releasing them to production. The platform can be used as the target deployment point for all master data of your enterprise or in conjunction with existing data hubs to contribute to data transparency and quality with federated governance processes. Its powerful and intuitive environment covers all use cases for setting up a successful master data governance strategy.
Semarchy xDM is based on a coherent set of features for all Master Data Management projects.
Unified User Interface
A Unique Modeling Framework
Semarchy xDM includes a unique modeling framework that combines both ER and OO modeling. Data architects and business analysts use this environment to define semantically complete models that will serve as references for the enterprise. These models include the description of the business entities as well as the rules associated with them.
The Modeling Framework supports:
Logical Data Modeling: The expression of the logical data model semantics and rules by business analysts. This includes:
The target data model (Entities / Attributes / Relations / List of Values, etc.).
The rules for data quality (validations, referential integrity validation, list of values, etc.).
The data access privileges associated with user roles
Certification Process Logical Design. This includes:
The definition of applications that publish data to the Hub (Publisher)
The rules to enrich and standardize data
The rules to match and to identify groups of similar records
The survivorship rules to produce the reference data (Golden Data)
Applications. This includes:
Creating, organizing and branding user-facing applications
Designing display cards, forms and collections to display the data from the entities
Designing search forms to search this data
Assembling the entities into compound Business Views
Designing steppers for authoring data.
Designing duplicate managers to manually merge and split groups of similar records.
Creating workflows.
Version Management
The innovative Semarchy xDM technology supports an infinite number of metadata versions. The collaborative process between the different governance teams set the rules to close version of models to keep full traceability or to run multiple projects in parallel.
Golden Data Certification
Generated Applications
Semarchy xDM supports Application generation. Applications provide secured and filtered access to the data stored in the hub using business-friendly Web interfaces. Applications also support data management operations and user collaboration.
Golden Data Consumption
The certified data is stored in a relational database, which allows Semarchy xDM to benefit from higher performance and scalability. The data is made available to consumers across multiple channels to allow a non-intrusive integration with the information system:
SQL access to reference data using JDBC or ODBC.
REST access to reference data.
INTRODUCTION TO THE SEMARCHY WORKBENCH
Logging In to the Semarchy Workbench
To access Semarchy Workbench, you need an URL, a user name and password that have been provided by your Semarchy xDM administrator.
To log in to the Semarchy Workbench:
Enter your user name and password.
Logging Out of the Semarchy Workbench
To log out of the Semarchy Workbench:
Opening a Model Edition
A model edition is a version of a model. This edition can be in a closed or open status. Only open editions can be edited.
Opening a model edition connects the workbench to this edition.
To open a model edition:
Working with the Semarchy Workbench
Working with Perspectives
There are several perspectives in Semarchy Workbench:
Overview: this perspective is the landing page in the Workbench. It allows you to access all the other perspectives and the applications.
Model Design: this perspective is used to edit or view a model.
Data Locations: this perspective is used to create data locations and deploy model editions in these locations.
Model Administration: this perspective is used to manage model editions and branches.
Administration Console: this perspective is used to administer Semarchy xDM and monitor run-time activity.
Working with Tree Views
When a perspective opens, a tree view showing the objects you can work with in this view appears.
This view appears on the left hand-side of the screen.
In this tree view you can:
Expand and collapse nodes to access child objects.
Double-click a node to open the object’s editor.
Right-click a node to access all possible operations with this object.
Working with the Outline
You can use the same expand, double-click, right-click actions in the outline as in the tree view.
Working with Editors
An object currently being viewed or edited appears in an editor in the central part of the screen.
You can have multiple editors opened at the same time, each editor appearing with a different tab.
Editor Organization
Editors are organized as follows:
The editor has a local toolbar which is used for editor specific operations. For example, refreshing or saving the content of an editor is performed from the toolbar.
The editor has a breadcrumb that allows navigating up in the hierarchy of objects.
The editor has a sidebar which shows the various sections and child objects attached to an editor. You can click in this sidebar to jump to one of these sections.
Properties in the editors are organized into groups, which can be expanded or collapsed.
Saving an Editor
To save an editor, either:
Closing an Editor
To close an editor, either:
Accelerating Editing with CamelCase
When this option is checked and the object name is entered using CamelCase, the object Label as well as the Physical Name is automatically generated.
Duplicating Objects
The workbench supports duplicating model objects such as forms, collection, search forms, business views, etc.
To duplicate an object:
A copy of the object is created.
It is also possible to duplicate a group of objects. When duplicating a group of objects that reference one another, the references in the copied objects are moved to the copies of the original objects.
For example:
When copying a single business view BV1 that references a collection C1, the copy of the business View (BV2) still references the collection C1.
To duplicate a group of objects:
A copy of the group of objects is created. The links are made, if possible, to the copied objects.
Deleting Objects
The Semarchy xDM model is designed to maintain its consistency.
This dialog indicates which references prevent the object deletion. Remove these references before deleting this object.
The Model Diagram
The Model Diagram Editor shows a graphical representation of a portion of the model or the entire model.
Using this diagram, you can create entities and references in a graphical manner, and organize them as graphical shapes.
The diagram is organized in the following way:
to zoom in and out in the diagram.
to select the elements to show in the diagram (attributes, entities, labels or names, foreign attributes, etc)
The Workflow Diagram
The workflow diagram shows a graphical representation of a workflows.
Using this diagram, you can design tasks and transitions for a workflow.
The diagram is organized in the following way:
to zoom in and out in the diagram.
to select the elements to show in the diagram (name/labels on the tasks and transitions).
to validate the workflow. The validation report shows the errors of the workflow.
Selectallows you to select, move and organize shapes in the diagram. This selection tool allows multiple selection (hold theShiftorCTRLkeys).
In this diagram, you can drag the transition arrows to change the source and target tasks of a transition.
Working with Other Views
Other views (for example: Progress, Validation Report) appear in certain perspectives. These views are perspective-dependent.
Workbench Preferences
User preferences are available to configure the workbench behavior. These preferences are stored in the repository and are specific to each user connecting to the workbench. The preferences are applied regardless of the client computer or browser used by the user to access the workbench.
Setting Preferences
The arrow controls in the upper-right of the right pane enable you to navigate through previously viewed pages. To return to a page after viewing several pages, click the drop-down arrow to display a list of your recently viewed preference pages.
Link Perspective to Active Editor: Select this option to automatically switch to the perspective related to an editor when selecting this editor.
Exporting and Importing User Preferences
Sharing preferences between users is performed using preferences import/export.
To export user preferences:
To import user preferences:
Importing preferences replaces all the current user’s preferences by those stored in the preferences file.
Working with SemQL
SemQL is a language to express declarative rules in Semarchy Workbench.
SemQL Language Characteristics
The SemQL Language has the following characteristics:
The syntax is similar to the SQL language and most SemQL functions map directly to database functions.
SemQL is converted on the fly and executed by the hub database.
SemQL uses Qualified Attribute Names instead of columns names. The code remains implementation-independent.
Qualified Attribute Names
This path not only allows accessing the attributes of the entity, but also allows access to the attributes of the entities related to the current entity.
Examples:
InputAddress.PostalCode: Definition Attribute (PostalCode) of theInputAddressComplex Attribute used in the currentCustomerentity.CostCenter.CostCenterName: CurrentEmployeeentity references theCostCenterentity and this expression returns an employee’s cost center nameCostCenter.ParentCostCenter.CostCenter: Same as above, but the name is the name of the cost center above in the hierarchy. Note thatParentCostCenteris the referenced role name in the reference definition.
SemQL Syntax
Conditionsare expressions returning true or false. They support
AND,OR,NOT,IN,IS NULL,LIKE,REGEXP_LIKE, Comparison operators (=,!=,>,>=,<,<=), etc.
In expressions, conditions and order by clauses, it is possible to use the SemQL functions. The list of SemQL functions is provided in the SemQL Editor
| SemQL is used to create expressions, conditions and order by clauses. SELECT, UPDATE or INSERT queries are not supported, as well as joins, sub-queries, aggregates, in-line views and set operators. |
SemQL Examples
The following condition checks the quality of the InputAddress complex attribute of the customer entity:
InputAddress.Address is not null and ( InputAddress.PostalCode is not null or InputAddress.City is not null)
The following binning expression groups customers by their Country/PostalCode:
InputAddress.Country || InputAddress.PostalCode
The following matching condition matches two customer records by name, address and city name similarity:
SEM_EDIT_DISTANCE_SIMILARITY( Record1.CustomerName, Record2.CustomerName ) > 65 and SEM_EDIT_DISTANCE_SIMILARITY( Record1.InputAddress.Address, Record2.InputAddress.Address ) > 65 and SEM_EDIT_DISTANCE_SIMILARITY( Record1.InputAddress.City, Record2.InputAddress.City ) > 65
The SemQL Editor
The SemQL editor can be called from the Workbench when a SemQL expression, condition or clause needs to be built.
This editor is organized as follows:
Functionsdeclared in SemQL appear in the left bottom panel, grouped in function groups. Double-click a function to add it to the expression. You candeclare your own database functionsin the model so that they appear in the list of functions.
Messagesappear in the right bottom panel, showing parsing errors and warnings.
Descriptionfor the selected function or attribute appear at the bottom of the editor.
Declaring Database Functions and Procedures
You can use customized functions in SemQL implemented in the database hosting the data location. You declare these functions in the model to have them appear in the list of functions. You can also use database procedures declared in the model, as Triggers in Applications.
Both the functions and procedures are declared using the same mechanism.
Note that:
Functions that are not declared can still be used in SemQL, but will not be recognized by the SemQL parser and will cause validation warnings.
Only procedures can be used as triggers, as triggers do not expect a returned value.
To declare a database function or procedure:
Name: Name of the function or procedure. This name must exactly match the name of the function in the database. Note that if this function is part of a package, the function name must be prefixed by the package name.
Schema: The schema containing this function/package.
Categories: Enter or select the function categories of the SemQL Editor into which this function should appear.
Repeat the previous step to declare all the arguments.
Close the editor.
Working with Properties
Literal vs. SemQL Properties
When a field supports literal and SemQL, a value type drop down appears aside the field to select the type of value. (Literal or SemQL).
Colors
CSS Color:
css: <css-color>, where<css-color>is aCSS color value.
Examples:css: rgb(255, 255, 155)css: rgba(255, 255, 155, 0.5)css: #FFFFFFcss: grey
Examplestheme: primarytheme: primary hue-1
Material Design Palette Color:
md:<color>[ <shade>], where<color>is one of named Material Design color and<shade>is one of the hues defined for this color (500, A500…). SeeMaterial Design Colorfor more information.
Examples:md: deep-purplemd: indigo A500
Images
Composing Image URLs It is possible to compose external or image library URLs using SemQL. 'images://myApp/priority_' || cust_priority || '.svg' |
Markdown
Markdownis a markup language with a simple text formatting syntax. Semarchy xDM is able to render Markdown text as HTML in the applications:
Semarchy xDM support the following Markdown flavors:
GitHub Flavored Markdown (GFM), including HTML embedded in markdown text.
Working with Plug-ins
Plug-ins allow extending the capabilities of the Semarchy xDM using Java code and external APIs.
Plug-ins are used to implement enrichers or validations not feasible in SemQL.
The plug-ins have the following characteristics:
Examples of plug-ins:
Enricher: Geocoding using an external API (Google, Yahoo, etc.)
Validation: Pattern Matching, Validating against an external API, etc.
More information:
LOGICAL MODELING
Logical modeling allows defining the business entities that make up the model.
Introduction to Logical Modeling
In Semarchy xDM, modeling is performed at the logical level. You do not design physical objects such as tables in a model, but instead design logical business entities independently from their implementation.
Logical Model Objects
Logical modeling includes creating the following objects:
Display Cards,Forms,Search FormsandCollectionsdefining how the entities are used in an Application. These objects are explained in theWorking with Applicationschapter.
Named Queriesdefining structured integration endpoints, accessed to consume data using a REST API.
Objects Naming Convention
When designing a logical model, it is necessary to enforce a naming convention to guarantee good readability of the model and a clean implementation.
There are several names and labels used in the objects in Semarchy xDM:
The following tips should be used for naming objects:
Define user-friendly Labels and Descriptions. Internal Names are for the model designers, but labels and descriptions are for end users.
The following table lists a typical naming convention you can adapt to your project.
| Object | Name Pattern | Examples |
|---|---|---|
Model | [Model Name] | StoresAndSuppliersMDM, ItemCatalog |
Entity | [Entity Name] | Product, Item, Customer |
Attribute | [Attribute Name] | IsProductAvailable |
Reference | [From Entity]Has[To Entity][Role] | EmployeeHasEmployeeManager |
Reference Role To | [Role Name In Singular] | Manager |
Reference Role From | [Role Name In Plural] | ManagedEmployees |
Complex Type | [ComplexType Name]Type | AddressType |
User-Defined Type | [Type Name]Type | SimpleStringType |
Publisher (Code) | [CODE] | MDM |
Publisher (Name) | [Publisher Name] | MDM Application |
Job | [JOB_NAME][JOB_TYPE] | INITIAL_LOAD_FULL, CERTIFY_PRODUCT_SIMPLE |
Form | [Form Name or Kind]Form | DefaultForm, ProductAuthoringForm, ProductReviewForm, MediaSimpleForm, ItemsMainAttributesForm |
Collection | [Collection Name or Kind]Collection | DefaultCollection, AllAttributesCollection |
List Of Values | [LOV Name]LOV | CurrencyLOV, ProductTypesLOV |
Business View | [Business View Name]([optional Classification]) | Products, ProductsByCategory, EmployeeHierarchy |
Transition Name | [Reference Role From] | ManagedEmployees |
Stepper | Author[Entity Name]s | AuthorProducts, AuthorCustomers |
Stepper Collection Step | [Entity Name]s or [Referencing Role Name] | Products, Customers, RelatedItems |
Stepper Form Step | [Entity Name] | Product, Customer, Item |
Action Set | [Action Set Name Or Purpose]ActionSet | CustomerActionSet, ProductImportExportActionSet, ProductCreateEditActionSet |
Application | [Application Name] | ProductSearch, ProductDesign, ReferenceDataManagement |
Folder | [Folder Name] | ProductsManagement |
Workflow | [Workflow Name] | CreateOrModifyCustomers, InitiateFullProductCreation, RequestProductLabelChange |
Workflow Task | [Task Purpose] | CreateProductBasicInformation, AddTechnicalDetailsForProduct, VerifyProductCreationGuidelines |
Workflow Transition | [Transition Verb/Action] | SendDataForApproval, RejectRequestAndAskForMoreInformation, AcceptAndSendToMarketing |
Model Validation
If you face issues with a model or an application, run a model validation to make sure that the model does not have new issues. |
To validate the model:
If you have unsaved editors, select those to save when prompted.
In the validation report:
| It is recommended to regularly run the validation on the model or on specific entities. Validation may guide you in the process of designing a model. You can perform regular validations to assess how complete the model really is, and you need to pass model validation before deploying or closing a model edition. |
Generating the Model Documentation
When a model is complete or under development, it is possible to generate a documentation set for this model.
This documentation set may contain the following documents:
The documentation is generated in HTML format and supports hyperlink navigation within and between documents.
To generate the model documentation:
The documentation is exported in a zip file containing the selected documents.
| It is possible to export the model documentation only for a valid model. |
Types
Several types can be used in the Semarchy xDM models:
List of Values(LOVs) are a user-defined list of string codes and labels. For example:Gender (M:Male, F:Female),VendorStatus (OK:Active, KO:Inactive, HO:Hold).
All these types, as well as the user-defined types, are reused across the entire model.
Built-in Types
Built-in types are provided out of the box in the platform.
Built-in types include:
Numeric Types:
ByteInteger: 8 bytes signed. Range [–128 to 127]
ShortInteger: 16 bytes signed. Range [–32,768 to –32,767]
Integer: 32 bytes signed. Range [–2^32 to (2^32-1)]
LongInteger: 64 bytes signed. Range [–2^64 to (2^64-1)]
Decimal: Number(Precision,Scale), where Precision in [1-38] and Scale in [–84-127]
Text Types:
String: Length smaller than 4000 characters.
LongText: No size limit – Translated to a CLOB in the database
Date Types:
Date: Date with no time or timezone.
Binary Types:
Binary: Store any type of binary content (image, document, movie, etc.) with no size limit – Translated to a BLOB in the database
Misc. Types:
UUID: 16 bytes Global Unique ID
Boolean: 1 character containing either `1' (true) or `0' (false).
Timestamps Values and Time Zones Conversion
Timestamp values contain the date and time, and are therefore subject to time zones conversions:
Timestamps are stored in the repository and data locations in the time zone of the application server.
Timestamps used in applications:
Users view or author timestamps in the timezone defined in their user profile. The values are automatically converted from/to the application server timezone when read/written to the database.
Import or export is an exception. Users import or export timestamps in the application server timezone, independently from their profile timezone. This allows exporting timestamps in one timezone and reimporting them in a different timezone without altering the values.
Timestamps in integration:
Integration specialists using SQL to consume data from or publish data to the MDM hub should be aware that conversion may happen depending on their driver configuration. It is recommended to have the integration component located in the same timezone as the application server to avoid such conversion.
List of Values
List of Values (LOVs) are a user-defined list of code and label pairs.
They are limited to 1,000 entries and can be imported from a Microsoft Excel Spreadsheet.
Examples:
| Lists of values are limited to 1,000 entries. If a list of value needs to contain more than 1,000 entries, you should consider implementing in the form of an entity instead. |
To create a list of values:
Name: Internal name of the list of values.
Length: Length of the code for the LOV.
Add values to the list using the following process:
In this dialog, enter the following values:
Code: Code of the LOV value. This code is the value stored in an entity attribute.
Label: User-friendly label displayed for a field having this value.
Description: Long description of this value.
Close the editor.
List of values can be entered manually as described above and can be translated.
In addition, you can also import values or translated values from a Microsoft Excel Spreadsheet.
This spreadsheet must contain only one sheet with four columns containing the Code, Label, Image URL and Description values. Note that the first line of the spreadsheet will be ignored in the import process.
To import a list of values from an excel spreadsheet:
Open the editor for the list of value.
Close the editor.
User-Defined Types
User-Defined Types (UDTs) are a user restriction on a built-in type. They can be used as an alias to a built-in type, restricted to a given length/precision.
Examples:
To create a user-defined type:
Name: Internal name of the user-defined type
Built-in Type: Select a type from the list.
Close the editor.
Complex Types
Complex Types are a customized composite type made up of several Definition Attributes using Built-in Type, User-Defined Type or a List of Values.
To create a complex type:
Name: Internal name of the object.
Repeat the following steps to add definition attributes to this complex type:
Name: Internal name of the object.
Type: List of values, built-in or user-defined type of this complex attribute.
Mandatory: Check this box to make this definition attribute mandatory when the complex type is checked for mandatory values.
Close the editor.
A complex type has the following advanced properties that impact its behavior:
Mandatory: When an entity attribute is checked for mandatory values, and this attribute uses a complex type, each of the definition attributes of this complex type with themandatoryoption selected are checked.
Searchable: This option defines whether this attribute is used for searching.
Translated: Options reserved for a future use.
Complex Type Display Name
To create or modify a complex attribute display name:
Separator: String that will separate the selected definition attributes in the display name.
Close the editor.
| Only one display name can be created for a given complex type. |
Entities
Entity Characteristics
Attributes
Entity Type
Each entity has a given entity type. This type expresses the entity capabilities for match/merge and authoring. Entity types are:
Basic: Basic entities do not support match and merge of records. They assume that data comes from a single data source. This entity type is suitable for entities for which data is authored (or imported) exclusively in the hub, or for simple reference data entities.
ID Matched(formerly known as UDPK): ID matched entities support match and merge of records. They assume that data comes from many data sources, and similar records share an identifier common to all sources. Records in entities using ID Matching are matched if they have the same ID and then merged into golden records. This entity type is well suited when there is a true unique identifier for all the applications communicating with the MDM hub.
| The choice of an Entity Type is important. Please take into account the following differentiators when creating an entity. |
Basic Entity
With this type of entity, you must assume that all data comes from a single (de-duplicated) source or is authored exclusively in the MDM hub.
When authoring or loading data in a basic entity, you simply overwrite the existing golden record with the new data, possibly keeping track of the changes. There is no notion of multiple master records merging into a golden record.
This type of entity is particularly suitable for reference data or classification data, which are typically managed only in the hub.
A Matcher can be defined for the entity, for detecting potential duplicates when manually creating records in the hub.
Use Basic Entities for data that only exist in the hub, or for data coming from a single data source into which there are no duplicates.
ID Matched Entity
This ID is stored into a single attribute which will be the golden data Primary Key. If the ID in the information system is composed of several columns, you must concatenate these values into the PK column.
A Matcher can be defined for the entity, for detecting potential duplicates when manually creating records in the hub.
Fuzzy Matched Entity
Fuzzy Matching means that applications in the enterprise have different IDs, and Semarchy xDM needs to generate a unique identifier (Primary Key - PK) for the golden records. This PK can be either a sequence or a Unique ID (UUID).
Similar records may exist various systems, representing the same master data, with different IDs. These similar records must be matched using fuzzy matching methods that compare their content.
ID Generation
The entity type impacts the method used for generating the record IDs:
Basic Entities: The Golden Record Primary Key is the ID sent or authored when creating a record. When creating records, this ID may be:
Manually provided by users
Automatically generated using a Sequence, Universally Unique Identifier generator (UUID) or a SemQL expression.
ID Matched Entities: The Golden Record Primary Key is also the ID that exists in the source systems. When creating source/master records, this ID may be:
Manually provided by users.
Automatically generated using a Sequence, Universally Unique Identifier generator (UUID) or a SemQL expression.
Fuzzy Matched Entities: The Golden Record Primary Key is managed and always generated by the system, using a Sequence or UUID.
When creating source/master records, this source/master ID may be:Manually entered by users
Automatically generated using a Sequence, Universally Unique Identifier generator (UUID) or a SemQL expression.
| When generating IDs automatically using a Sequence in data entry forms for an ID Matched entity, you must take into account records pushed by other publishers (using for example a data integration tool). |
| An ID generated with a SemQL expression is immutable. This ID will not change after the initial record creation even if the value of the attributes used in the expression change. Such an ID is created when a record form is saved for the first time, for example when a record is imported from a file containing no IDs. |
References
Constraints
Data quality rules are created in the design of an entity. These constraints include:
Mandatorycolumns
List of Valuesrange check
Unique Key
Reference Relationships
These constraints are checked on the source records and the consolidated records as part of the certification process. They can also be checked to enforce data quality in data authoring.
Inheritance
Entities can extend other entities (Inheritance). An entity (child) can be based on another entity (parent).
When inheritance is used:
The child entity inherits the following elements: Attributes, Unique Keys, Validations, Enrichers and References.
Matchers and Survivorship Rules are not inherited
It is not possible to modify the entity type. The child inherits from the parent entity type.
It is possible to add elements to the child entity: Attributes, Unique Keys, Validations, Enrichers and References.
The underlying physical tables generated for the child entities and parent entity are the same. They contain a superset of all the attributes in the cluster of entities.
Deleting an inherited attribute on a child entity removes it from the parent entity, and by extension from all the child entities inheriting this attribute from the parent.
Inheritance Limitations & Recommendations Although inheritance looks like a good way to avoid repeating design artifacts on entities that look the same, the interest of inheritance is frequently counterbalanced by certain limitations:
Due to these limitations, using inheritance reduces the capability of a model to evolve on the long run. It is not recommended to use inheritance unless you have a full understanding of these limitations. Experience shows that a simpler approach using a single entity that holds all attributes, along with conditions to run rules or customize UI artifacts, brings the similar benefits and greater flexibility to the model. This approach is recommended in most cases instead of inheritance. |
Certification Rules
In addition to the display characteristics, an entity is designed with certification rules describing how master data is created and certified from the source data published by the source applications.
Entity Records Hard and Soft Delete
This allows two types of deletes on the entity record, configured on each delete operation:
Hard DeletePhysically deletes records from the hub database. Data cannot be recovered.
When an entity support deletion, the propagation of the deletes must be taken into account when configuring the reference relationships to this entity.
| Delete is handled by the jobs generated when deploying the model edition. A change in the delete or delete propagation configuration requires that you re-deploy the model edition. |
Historizing Entity Data
You can historize an entity to keep track of data changes and enable browsing this entity data at any point in time.
| History uses specific table structures and is handled by the jobs generated when deploying the model edition. A change in the history configuration requires that you re-deploy the model edition. |
You can run again data loads in order to force re-creating the history, or copy the current state of the golden data in the history to seed the history with the current data state. |
Creating an Entity
Creating a New Entity
To create an entity:
Name: Internal name of the object.
Plural Label: User-friendly label for this entity when referring to several instances. The value for the plural label is automatically generated from the Label value and can be optionally modified.
If you do not use inheritance, proceed to the next step.
Physical Table Name: This name is used to name the physical table that will be created to store information about this entity. For example, if the physical table isCUSTOMER, then the golden data is stored in aGD_CUSTOMERtable.
Name: Internal name of the primary key attribute.
Label: User-friendly label for this primary key attribute.
Physical Column Name: This name is used to name the physical column that will be created to store values for this attribute.
Sequence: Use this option to automatically generate the Golden ID as a sequential number. You can specify a startup value for the sequence in theStarts Withfield.
UUID: Use this option to automatically generate the Golden ID as a Universally Unique Identifier.
UUID: Use this option to automatically generate the ID as a Universally Unique Identifier.
Manual: Use this option to manually provide the ID.
When an entity is created, it contains no attributes. Simple Attributes and Complex Attributes can be added to the entity now.
Adding a Simple Attribute
To add a simple attribute:
Name: Internal name of the object.
Label: User-friendly label for this object. Note that as theAuto Fillbox is checked, theLabelis automatically filled in. Modifying this label is optional.
Physical Column Name: This name is used to name the physical column that will be created to store values for this attribute.
Type: Select the type of the attribute. This type can be a built-in type, a user-defined type or a list of values.
Mandatory: Check this box to make this attribute mandatory.
Mandatory Validation Scope: This option is available if theMandatoryoption was selected. Select whether this attribute should be checked for null values pre and/or post consolidation. For more information, refer to theCertification Process Designchapter.
LOV Validation Scope: This option is available if the selected type is a list of values. It defines whether the attribute’s value should be checked against the codes listed in the LOV. For more information, refer to theCertification Process Designchapter.
Adding a Complex Attribute
To add a complex attribute:
Name: Internal name of the object.
Physical Prefix: This name is used to prefix the physical column that will be created to store values for this complex attribute. The column name is
<Physical Prefix>_<Definition Attribute Physical Column Name>Complex Type: Select the complex type of the attribute.
LOV Validation Scope: This option is available if at least one of the definition attributes of the complex type is a list of values. It defines whether the definition attribute’s value should be checked against the codes listed in the LOV. For more information, refer to theCertification Process Designchapter.
Working with Attributes
To order the attributes in an entity:
Open the editor for the entity.
To delete attributes from an entity:
Open the editor for the entity.
Altering an Entity
Altering an entity allows modifying the key properties of an entity, including its type, inheritance and primary key attribute.
To alter an entity:
| When altering an entity, it is strongly recommended to validate the model to make sure that the other elements of the model do not conflict with the modified entity definition. You must also re-deploy the model edition in order to make the changes active in the deployment data location. |
Reference Relationships
For example:
They are used for:
Navigatingto records referenced through a relation. For example, navigate from anEmployeerecord to theCostCenterhe belongs to.
Propagating or preventing deletion. For example, to define that aCustomerwith existingContractscannot be deleted, or that when aProductis delete, all itsPartsshould also be deleted.
A reference is expressed in the model in the form of a foreign attribute that is added to the referencing entity.
To create a reference relationship:
Name: Internal name of the object.
Physical Name: Name of the database indexes created for optimizing access via this reference.
Validation Scope: Select whether the referential integrity should be checked pre and/or post consolidation. For more information, refer to theCertification Process Designchapter.
Physical Name: Name of the physical column created for the foreign attribute in the tables representing the referencing entity.
Restrict: Prevent referenced record deletion if a referencing record exists. If a child is found through this relationship while attempting to delete the parent (or an ancestor), the whole delete operation will be aborted or not allowed.
Nullify: Set the reference to null on the referencing record when referenced record is deleted. If a child is found through this relationship while attempting to delete the parent, its relationship to the parent is set to null. This value cannot be selected for mandatory relationships
Cascade: Delete referencing records when referenced record is deleted. If a child is found through this relationship while attempting to delete the parent, an attempt to delete this child and all its children recursively will be made, until all children are finally deleted. If any of the child relationships - at any level are preventing the delete, the record and all its children are not deleted.
Close the editor.
Data Quality Constraints
Data Quality Constraints include all the rules in the model that enforce a certain level of quality on the entities. These rules include:
Mandatory Attributes: An attribute must not have a null value. For example, thePhoneattribute in theCustomerentity must not be null.
LOV Validation: An attribute with an LOV type must have all its values defined in the LOV. For example, theGenderattribute of theCustomerentity is a LOV of typeGenderLOV. It must have its values in the following range: [M:Male, F:Female].
Validations: A formula that must be valid on a given record. For example, aCustomerentity must have either a validEmailor a validAddress.
More information:
Unique Keys
A Unique Key defines a group of attributes that should be unique for an entity.
Name: Internal name of the object.
Label: User-friendly label for this object. Note that as theAuto Fillbox is checked, theLabelis automatically filled in. Modifying this label is optional.
Validation Scope: Select whether the unique key should be checked pre and/or post consolidation. For more information, refer to theCertification Process Designchapter.
Close the editor.
| In the certification process unique keys are checked after the match and consolidation process, on the consolidated (merged) records. Possible unique key violations are not checked on the incoming (source records). |
| Unique keys can be checked in data authoring to verify whether records with the same key already exist. |
Validations
A record-level validation validates the values of a given entity record against a rule. Several validations may exist on a single entity.
There are two types of validation:
SemQL Validation
To create a SemQL validation:
Name: Internal name of the object.
Description: optionally enter a description for the SemQL Validation.
Condition: Enter the SemQL condition that must be true for a valid record. You can use the
Edit Expressionbutton to open theSemQL Editor.
Close the editor.
Plug-in Validation
To create a plug-in validation:
Name: Internal name of the object.
Plug-in ID: Select the plug-in from the list of plug-ins installed in the platform.
Set the values for the parameters:
Repeat the previous steps to set the value for the parameters.
Set the values for the inputs:
Repeat the previous steps to set an expression for the inputs.
Close the editor.
Diagrams
A Diagram is a graphical representation of a portion of the model or the entire model.
Using the Diagram, not only can you make a model more readable, but you can also create entities and references in a graphical manner, and organize them as graphical shapes.
When you double click on a shape from the diagram, you access the actual entity or reference via the shape representing it.
It is possible to remove a shape from the diagram without deleting the entity or reference.
You can have multiple shapes representing the same entity in a diagram. This is typically used for readability reasons. All these shapes point to the same entity.
If you delete an entity or reference, the shapes representing it automatically disappear from the diagrams.
Creating Diagrams
To create a diagram:
Name: Internal name of the object.
Working with Entities and References
In this section, the creation/deletion of entities and references from the diagram is explained.
To create an entity using the diagram:
The entity is created and a shape corresponding to this entity is added to the diagram.
Note that you can also create, edit and delete attributes from the diagram. Select an attribute or entity and use the context menu options.
To create a reference using the diagram:
Select the referencing entity in the diagram. Keep the mouse button pressed, and move the cursor to the referenced entity.
The reference is created and a shape corresponding to this reference is added to the diagram.
To delete a reference or an entity from the diagram:
In the diagram, select the entity or reference you want to delete.
The reference or entity, as well as the shape in the diagram disappear.
| Deleting an entity or reference cannot be undone. |
Working with Shapes
In this section, the creation/deletion of shapes in the diagram without changing the real entity or reference is explained.
To add existing entities to the diagram:
Select the entities to add to the diagram.
You can repeat this operation if you want to add multiple shapes for an entity in the diagram.
To add existing references to the diagram:
It is not possible to manually create a shape for reference in a diagram.
When an entity is added to the diagram, shapes for the references relating this entity to entities already in the diagram are automatically added.
To remove a shape from the diagram:
In the diagram, select the shape representing the entity of reference you want to delete.
The shape disappears from the diagram. The entity or reference is not deleted.
Database Reverse-Engineering
Reverse-engineering can be used to quickly populate a model from table structures stored in a schema.
During this process, entities are created with attributes named after the column names.
| This process connects to a database schema using a JDBC datasource that must be previously configured on the application server for the Semarchy xDM application. Contact the application server administrator to declare this datasource. |
| Reverse-engineering is provided as a one-shot method to quickly populate the initial model from a database schema. The entities created using a reverse-engineering are by default fuzzy matched entities. Foreign keys existing between the tables in the schema are not reverse-engineered as references in the model. Besides, the reverse-engineering process is not incremental. |
To perform a database reverse-engineering:
Select only the entities and attributes you want to reverse-engineer.
Modify the logical and physical names in this page.
Model Variables
| SemQL provides built-in variables that contain information about the load or batch being processed or about the user connected or performing the operations. For example, the user name, its roles, the email or phone number he has entered in his profile. |
Creating Model Variables
| Before creating model variables make sure that the variable value providers, from which you want to retrieve information, are declared for your Semarchy xDM instance. |
To create a model variable:
Name: Internal name of the object.
Variable Value Provider: select the variable value provider that will be queried to retrieve the values for this variable.
Select the
For an LDAP Variable Value Provider, enter the parameters of the LDAP search:
Filter: Criteria to use for selecting elements within the scope. For more information about LDAP filters, see LDAP Search Filter Syntax.
Attribute: The attribute from the returned result to set in the variable value.
Close the editor.
Variable Lookup Queries
| Lookup queries should return a single value (column) and a single result (record). If the query returns multiple results or multiple values, only the first value of the first result is taken into account and set in the variable value. |
| It is not possible to use a variable in the lookup query of another variable. |
Testing Model Variables
After creating a new model variable, it is recommended to test it.
To test a model variable:
Using Model Variables
Model Variables can be used:
In user sessions: They are set when the user accesses an application, using a variable value provider. In this context, variables are used to parameterize the user experience (for example, in filters restricting the user privileges).
In integration jobs: In an integration job, a variable value is usually set using a job parameter. If no job parameter is set, the value is set using the variable value provider. In this context, variables are used to parameterize the job execution (for example, in an enricher’s filter expression to prevent the enricher from processing any record depending on the value).
Model Variable can be used, for example, in the following SemQL expressions:
User’s search filters defined when accessing the data,
Etc.
The connected user will be granted these privileges only for the master data record matching this expression.
To parameterize an enricher’s execution depending on a job parameter value:
| Variable values are not persisted. They are retrieved when the user connects, and disposed at the end of the user session. If the content of the Variable Value Provider (Remote LDAP Directory or Database) is modified, the changes are taken into account only when the user re-connects to Semarchy xDM. |
| When a model variable is used in a job, but no corresponding job parameter is set for the job, then the variable takes its value from the variable value provider, using as the connected user the one that has started the job. |
Named Queries
Named queries define integration endpoints to consume structured information from the MDM hub using a REST API. Each named query is exposed as a REST endpoint, supporting complex structures as query parameters.
Typically: * Values represent attributes of the entity * Multiple Records represent referencing (child) records with their properties. * Single-Records represent referenced (parent) records with their properties.
| A Multiple-Records or Single-Record property can refer to an object that already exist in the named query. Use this capability to create recursive structures. |
Creating a Named Query
To create a named query:
Name: Internal name of the object.
Description: Description of the named query.
Entity: select the entity at the root of the named query.
Adding a Property
To add a property to an object:
Property Type: Select the type of the property.
SemQL Expression: Enter or build the SemQL expression corresponding to a single record, multi-record or value expression, according to the property type.
Name: Provide a name for the property.
Select the Multi Record or Single Record property that you just created.
Creating Query Parameters
A query parameter is used to parameterize the query. It is defined for the query and can be used in the filter conditions, sort expression and the properties' SemQL expressions.
To add a query parameter:
Name: Internal name of the object.
Binding Name: Name used to refer to this variable in the filter conditions, sort and SemQL expressions.
Parameter Data Type: Type of the parameter.
Default Value: Configure a default value for the parameter.
Mandatory: If the parameter has no default value, indicate whether it is mandatory for the query.
ProductID = :QUERY_PARAM_SEARCHED_ID
Using Named Queries
CERTIFICATION PROCESS DESIGN
Introduction to the Certification Process
Source Authoring Record, authored by users in the MDM applications.
When a user authors data in an MDM application, depending on the entity type and the application design, he performs one of the following operations:He overrides golden values resulting from the consolidation of records pushed by publishers. This pattern is allowed only for ID and fuzzy matched entities.
Delete Operations, made by users on golden records from entities with delete enabled.
Matching Decisions, taken by data stewards for fuzzy matched entities, using duplicates managers. Such decisions include confirming, merging or splitting groups of matching records as well as accepting/rejecting suggestions.
The certification process takes these various sources, applies the rules and constraints defined in the model in order to create, update or delete the golden data that business users browse using the MDM applications and that downstream application consume from the hub.
This process is automated and involves several phases, automatically generated from the rules and constraints, which are defined in the model based on the functional knowledge of the entities and the publishers involved.
The following sections describe the details of the certification process for ID, fuzzy matched and basic entities, and the delete process for all entities.
Certification Process for ID and Fuzzy Matched Entities
The certification process involves the following steps:
Note that source authoring records are enriched and validated only for basic entities. For ID and fuzzy matched, source authoring records are not enriched and validated.
Match and Find Duplicates: For fuzzy matched entities, this step matches pairs of records using aMatcherand creates groups of matching records (match groups). For ID matched entities, matching is simply made on the ID value.
Matching Decisionstaken by users on match groups are applied at that point, superseding the matcher’s choices.
Publish Certified Golden Data: This step finally publishes theGolden Recordsfor consumption.
Historize Data: Golden and master data changes are added to their history if historization is enabled.
Source Authoring Recordsare not enriched or validated for ID and fuzzy matched entities as part of the certification process. These records should be enriched and validated as part of the steppers into which users author the data. |
Certification Process for Basic Entities
The certification process involves the following steps:
Publish Certified Golden Data: This step finally publishes theGolden Recordsfor consumption .
Historize Data: Golden data changes are added to their history if historization is enabled.
Note that: |
Deletion Process
Propagate through Cascade: Extends the deletion to the child records directly or indirectly related to the deleted ones with a Cascade configuration for Delete Propagation.
Propagate through Nullify: Nullifies child records related to the deleted ones with a Nullify configuration for the Delete Propagation.
Compute Restrictions: Removes from deletion the records having related child records and a Restrict configuration for the Delete Propagation. If restrictions are found, the entire delete is canceled as a whole.
Publish Deletion: Tracks record deletion, with the record values for soft deletes only, and then removes the records from the golden and master data. When doing a hard delete, this step deletes any trace of the records in every table. The only trace of a hard delete is the ID (without data) of the deleted master and golden records. Deletes are tracked in the history for golden and master records, if historization is configured.
| Is is not necessary to configure in the job all the entities deletion should cascade to. The job generation automatically detects the entities that must be included for deletion based on the entities managed by the job. |
Rules Involved in the Process
The rules involved in the process include:
Enrichers: Sequence of transformations performed on the source and/or consolidated data to make it complete and standardized.
Data Quality Constraints: Checks carried out on the source and/or consolidated data to isolate or flag erroneous rows. These include Referential Integrity, Unique Keys, Mandatory attributes, List of Values, SemQL/Plug-in Validations
Matcher: This rule applies to Fuzzy Matched entities only. It is a set of match rules that bin (group) then match similar records to detect them as duplicates. The resulting duplicate clusters are merged (consolidated) and confirmed depending on their confidence score.
Survivorship Rule: This rule defines, for Fuzzy Matched and ID Matched Entities, how the golden record values are computed. It is composed of:
Integration Jobs
Integration jobs are triggered to integrate data published in batch by data integration/ETL tools, or to process data managed by users in applications.
When pushing data in the hub, a data integration or ETL product performs the following:
Similarly, when a user starts a stepper:
The user performs the data authoring operations in the graphical user interface. All the data manipulations are performed within the transaction. The user can save data in this transaction and resume these data changes later.
Workflows
Workflows allow users to collaborate for data authoring. Each task of a workflow is processed by a user who modifies and reviews data using a stepper.
When a workflow completes, an integration job is triggered to process the changes and choices made by the user.
Workflow Lifecycle
Transaction
A workflow carries along a Load Transaction (equivalent to an external load) which contains the data changes performed by the user. This transaction attached to the workflow. Changes performed in this transaction exist only for the transaction and the workflow until the workflow is submitted or canceled
Publishers
The consolidation rules perform certain choices depending on the publishers, and the publishers are tracked to identify the origin of the data certified by Semarchy xDM.
| Identifying clearly and declaring the publishers is important in the design of the certification process. Make sure to identify the publishers when starting an MDM project. |
| You do not need to define a dedicated publisher for the data authored directly in the generated MDM applications. This data is of the automatically identified by the system as authored by users. |
To create a publisher:
Name: Internal name of the object.
Code: Code of the publisher. This code is used by the certification process pushing data to the hub, to identify records originating from this publisher.
Active: Check this box to make this publisher active. An inactive publisher is simply declared but not used in the consolidation rules.
Close the editor.
Enrichers
Enrichers normalize, standardize and enrich data loaded or authored in the hub. Additional Post-Consolidation enrichers can apply to consolidated data resulting from the match and merge process.
Enrichers have the following characteristics:
Several enrichers can be defined for an entity, and are executed in a sequence. The order in which they are defined in the model will be the order in which they will be executed
Enrichers can be enabled or disabled for integration jobs. Enrichers disabled for the jobs can be used for data authoring.
Enrichers can be configured to run on the source data (pre-consolidation) and/or the consolidated (post-consolidation) data.
Enrichers running pre-consolidation apply to all incoming data. It is possible to define a filter on each enricher. Only the filtered records are modified by the enricher.
Only pre-consolidation enrichers are allowed for basic entities.
There are two types of enrichers:
Plug-in Enrichersuse a Plug-in developed in Java. These enrichers are executed by Semarchy xDM. Transformation that cannot be carried out in within the database (for example that involve calling an external API) can be created using plug-in enrichers.
Creating SemQL Enrichers
A SemQL Enricher enriches several attributes of an entity using attributes from this entity, transformed using SemQL expressions and functions.
To create a SemQL enricher:
Name: Internal name of the object.
Set the enricher expressions:
Repeat the previous steps to set an expression for each attribute to enrich.
Close the editor.
Creating Plug-in Enrichers
A Plug-in Enricher enriches several attributes of an entity using attributes from this entity, transformed using a plug-in developed in Java.
A plug-in enricher takes:
Attributes are mapped on the input to feed the plug-in and on the output to enrich the entity with the resulting data transformed by the plug-in.
To create a plug-in enricher:
Name: Internal name of the object.
Plug-in ID: Select the plug-in from the list of plug-ins installed in the platform.
Optionally click the
Set the values for the parameters:
Repeat the previous steps to set the value for other parameters.
Set the values for the inputs:
Repeat the previous steps to set an expression for other inputs.
Close the editor.
Advanced Plug-in Configuration
The enricher and validation plug-ins provide options for optimizing and configuring their execution.
Max Retries: If the execution of the plug-in fails, it is repeated for this number of times.
Thread Pool Size: This property defines the number of parallel threads used by this plug-in. This option is taken into account if the plug-in used is thread safe and declared as such.
Pre and Post-Consolidation Validation
Several validations can be defined per Entity. Validations check attribute values and reject invalid records. All validations are executed on each record.
Validations can take place pre-consolidation and/or post-consolidation:
Pre-Consolidation: Applies to source data, after the enrichment phase and before the matching phase. All the source records pass through all the pre-consolidation checks and records failing one check areisolatedfrom the certification flow. All the errors for each record are raised.
Post-Consolidation: Applies to the golden data. All the consolidated records pass through all the post-consolidation checks and records failing one check areflaggedas erroneous. They are not removed from the flow. All the errors for each record are raised. Note that post-consolidation validations are not supported for Basic Entities.
Pre vs. Post Validation
Choosing the scope of a validation has impact on the certification process.
The following examples will illustrate the impact of the choice of the pre or post consolidation validation.
Customer data is published from the CRM and Sales applications.
Only the Sales publisher loads revenue data. CRM leaves this attribute null.
The consolidation needs critical information from the CRM application (email, name, address, etc…)
Validation scope impact on data certification:
If CheckNullRevenue is executed pre-consolidation, all data from the CRM is rejected, as revenue is null. No data from the CRM is matched or merged and the golden records are incomplete.
Matching
The matching phase detects the duplicates in order to consolidate them into a single golden record.
Matching works differently for Fuzzy Matched and ID Matched entities.
ID Matched Entitiesperform an exact match on the user-provided ID value, as this ID is a primary key that is unique across all systems.
| Basic entities do not support matching and consolidation. ID Matched entities do not support matching (it is made using the ID), but support record consolidation. |
You can define matchers on Basic and ID Matched Entities. They are used for the sole purpose of detecting duplicates when creating new records. Such matchers interactively warns the user when a new entry matches existing records.
Understanding Match and Merge
Match Rules and Scoring
For example, the following rule (MATCH_RULE_1) defines two businesses that are exactly the same, and may have a score of 100:
Record1.CustomerName = Record2.CustomerName and Record1.InputAddress.Address = Record2.InputAddress.Address and ... Record1.InputAddress.City = Record2.InputAddress.City
The following rule (MATCH_RULE_2) would have for example a score of 85, as it detects businesses that have large number of similarities:
SEM_EDIT_DISTANCE_SIMILARITY( Record1.CustomerName, Record2.CustomerName ) > 85 and SEM_EDIT_DISTANCE_SIMILARITY( Record1.InputAddress.Address, Record2.InputAddress.Address ) > 65 and SEM_EDIT_DISTANCE_SIMILARITY( Record1.InputAddress.City, Record2.InputAddress.City ) > 65
The following rule (MATCH_RULE_3) would have a score of 20, as it detects two businesses with vaguely similar names in the same city:
SEM_EDIT_DISTANCE_SIMILARITY( Record1.CustomerName, Record2.CustomerName ) > 50 Record1.InputAddress.City = Record2.InputAddress.City
When two records match, they receive a match score equal to the highest score of all the rules that matched them (highest confidence).
Matching Groups & Group Confidence Score
| The matching mechanism is transitive, which means that: If A matches B and B matches C, then A, B and C are in the same matching group. |
This score is the average of the match scores in the group. Pairs in the group that have not matched by any rule are considered as having a score of zero.
For example:
A matches B according to MATCH_RULE_1 with a match score of 100
B matches C according to MATCH_RULE_3 with a match score of 20
A did not match C so their match score is 0.
A, B and C are in the same matching group according to the matching transitivity.
Merging Groups into Golden Records
Incoming records are kept as singleton golden records.
Existing groups of records and golden records remain untouched.
Merge Suggestionsare reviewed by data stewards with duplicate management actions, to decide whether or not to merge groups and create golden records.
Confirming Golden Record
As the values change in the source records, the match groups and golden records may change.
Renaming a business from "Micro Soft Incorporated" to "Micro Soft" is likely to have it match and merge with an existing "Microsoft" record, if we fuzzy-match by business name. The original "Micro Soft Incorporated" golden record would then cease to exist, as it would be merged within the "Microsoft" record.
This is typically done by a data steward with a duplicate management action. The steward manually confirms the correct match groups and fixes the incorrect match groups.
Depending on the confidence score computed for a match group, you may also want to automatically confirm the golden record to avoid having the steward reviewing all the data.
It also allows you to set whether singletons (golden records composed of a single master record) should be automatically confirmed.
Not Confirmed: The golden record was never confirmed by the matcher or a user.
Confirmed: The golden record was entirely confirmed by the matcher or a user.
Partially Confirmed: Part of the match group that composes the golden record was confirmed by a user, but some masters in this match group are still not marked as confirmed.
Previously Confirmed: A record that was confirmed but which group has been modified by a user.
| Regardless of the Confirmation status of a record, a data stewards is always able, with a duplicate management action, to manually split and merge duplicates. |
Understanding Match Rules
Each Match Rule runs as a two phase process:
Matching: This phase uses a SemQL condition that compares all the pairs of records (Record1 is compared with Record2) within a bin. When this condition returns true, the pair of records are considered as duplicates.
The Binning Phase
Examples:
Binning is completed using several expressions defined in the matcher. The records for which all binning expressions give the same results belong to the same bin.
To perform binning for Customers with the same Country and Region’s first letter in the GeocodedAddress complex field, we would use:
| Smaller Bins will mean faster processing, but, you must make sure that binning does not exclude possible matches. |
For this specific case, you may consider a different attribute, or use SOUNDEX transformation on the name for binning.
The Matching Phase
The matching phase uses a condition that compares two records.
The following matching condition matches customers having Customer names meeting the two following requirements
Sounding the same in English (SOUNDEX) OR with a similar name (using edit distance) by more than 80%.
Similar City name and address (using edit distance) by more 65%
( SOUNDEX (Record1.CustomerName) = SOUNDEX (Record2.CustomerName)
OR
SEM_EDIT_DISTANCE_SIMILARITY(Record1.CustomerName,Record2.CustomerName)>80 )
and SEM_EDIT_DISTANCE_SIMILARITY(Record1.InputAddress.Address, Record2.InputAddress.Address) > 65
and SEM_EDIT_DISTANCE_SIMILARITY( Record1.InputAddress.City, Record2.InputAddress.City ) > 65Attributes Available for Match Rules
The match rules have access to a certain number of attributes:
Attributes from the entity being matched.
Attributes from related parent entities of the entity being matched.
Attributes from related child entities of the entity being matched.
Matching on Parent Records
This capability is available for all entities. You can access attributes from the parent entities using regular SemQL expressions.
Record1.Customer.Name = Record2.Customer.Name
Matching on Child Records
This capability is available only for fuzzy-matching entities. You must declare in the rule the child entity used for matching.
Match on Child Recordschecked.
Child Recordsset toEmailAddresses
Note also that if any pair of child records matches according to the rule, then the two parent records are matched.
Record1.Address = Record2.Address
Automating Merge and Confirmation
Merge Policy Situations
In the merge policy, you define a confidence score threshold above which the match group is automatically merged into a golden record.
If a match group of new master records with a confidence score of 80 or lower appears, then is not merged automatically. It is proposed for merging to the data steward.
The various situations under which an automated merge may take place are listed and explained below:
Create a golden record from new master records: This is a frequent situation when new master records are loaded into the hub and matched/merged. The initial data loads enter in this situation.
Merge confirmed golden records: This situation occurs when existing master records attached to golden recordsthat have been confirmedare modified, causing the existing golden records to possibly merge all together. In this situation, if the two golden records merge, a confirmed golden record may cease to exist, and the other one may see its values modified.
Merge unconfirmed with confirmed golden records: This situation occurs when existing unconfirmed golden records are about to merge with a golden recordthat has been confirmed. In this situation, unconfirmed golden records may cease to exist, and the confirmed golden record may see its values modified.
Merge golden records previously split by the user: This situation occurs when two groups manually split by a data steward re-matches due a new record matching both groups. In this situation, existing golden records reviewed by the data steward may cease to exist.
| A group may fall into several situations. In that case, its confidence score must exceed all the thresholds for the automated merge to happen. |
Auto-Confirm Situations
There are two situations for automatically confirming merged golden records:
When the match group’s confidence score is above a certain threshold, the resulting golden record can be automatically marked as confirmed.
Singletons, that is golden records composed of a single master record, can be automatically confirmed. Note that singletons that have match suggestions (records matched but with a score not high enough to automatically merge) are not automatically confirmed.
Pattern for Automating Merge & Confirmation
Pattern #1: No Unmonitored Change
In this pattern, the data steward should review all records. All of them are treated with the same importance. No change is made and no record is created without having a user confirming it.
Solution:
Pattern #2: No Stewardship
In this pattern, the hub merges and confirms all content. Fixes will take place on demand on confirmed golden records.
Solution:
Pattern #3: Delayed Stewardship
In this pattern, the hub merges all content and confirms no record. The steward monitors and confirms all records after they are merged.
Solution:
Pattern #4: Merge All then Review Suspicious Matches
In this pattern, the hub merges all content but the steward must review suspicious matches.
Solution:
The steward will be able to review suspicious unconfirmed golden records (those under the 80% confidence score).
Pattern #5: Manually Merge Suspicious Matches
In this pattern, the hub merges and confirms confident matches but the steward must manually merge others.
Solution:
Pattern #6: Prevent Golden Deletion
In this pattern, the hub must prevent golden records from ceasing to exist without the steward approval, but allows other confident merge operations to take place automatically.
Solution:
Creating a Matcher
| Only one matcher can be created for each entity. |
To create a matcher:
Repeat the previous steps to create all your binning expressions.
Use the breadcrumb on top of the editor to return to the matcher. The new match rule appears in the list.
Repeat the previous steps to create all the match rules.
Auto-confirm golden records: Minimum confidence score required for a match group to be automatically confirmed.
Auto-confirm singletons: Select this option to have singletons (un-matched records) automatically confirmed.
Close the editor.
Survivorship
Survivorship indicates which data survives in the golden records.
A survivorship rule applies to a single attribute or to a set of attributes of a given entity.
Grouping Attributes in Survivorship Rules
When a rule has multiple attributes, all these attributes work together for the consolidation and override process:
When a user overrides one of these attributes, all of them are considered overriden. In the user interface, when the override buttons is clicked for one attribute, override is also toggled for all the attributes in the group.
Master ID Survivorship Rule
This rule does not have an override strategy. It only defines how the ID of one of the master records attached to a golden record consolidates into this golden.
Consolidation Rule
The consolidation strategies are listed in the table below. It also shows the equivalent SemQL expression for the strategy.
| Strategy | Description | Expression |
|---|---|---|
Custom Ranking |
|
|
Largest/Smallest Value | Values are sorted using their type-specific sort method (alphabetical for strings, for example). | |
Longest/Shortest Value | The lengths of the values are ordered. | |
Most Frequent Value | The first most frequent non null value. | Specific |
Preferred Publisher | Publishers are manually ordered. The first one returning a value for the field is used. | Specific |
Override Rule
The override strategies are listed in the table below.
| Strategy | Description |
|---|---|
Always Authored in the MDM | The value is always authored in the MDM application. Values coming from other sources (if any) are always ignored. This attribute remains null until a user explicitly changes the value. The attribute is therefore ignored for consolidation. |
No Override | The value is always consolidated according to the consolidation rules. The application does not allow overriding data for this attribute. |
Override - until consolidated value changes | Value override is allowed. Values coming from the publishers consolidate according to the consolidation rules. When an override is made, the value is maintained until the value consolidated from the publishers changes. When this happens, the new consolidated value wins against the authored value. The system reverts to the defined consolidation rules to arbitrate survivorship for the next value changes from the publishers. |
Override - until next user changes | Value override is allowed. Values coming from the publishers consolidate according to the consolidation rules. When an override is made, the value is maintained until a user enters a new value or removes the override. |
Defining a Survivorship Rule
These default rules should be modified to reflect the entity requirements. You can also create new survivorship rules to handle specific attributes.
To create a survivorship rule:
Name: Internal name of the object.
Label: User-friendly label for the object.
Default Rule: Define this new rule as the default rule for the entity.
Select the attributes managed with this rule:
Click
Define the consolidation rule:
Set the parameters depending on the selected strategy:
Custom Ranking:
Preferred Publisher:
Repeat this operation to add the other publishers in the preference order.
Define the override rule:
Close the editor.
Creating Integration Jobs
Creating Jobs
To create a job:
Name: Internal name of the object.
Description: Optionally enter a description for the Job.
Queue Name: Name of the queue that will contain this job.
To edit the processes involved in one task:
Select the process you want to enable for this task.
Use the editor breadcrumb to go back to the Job editor.
Close the editor.
Jobs Parameters
Jobs can be parameterized for optimizing their execution.
To change a job parameter:
Close the editor.
The following table lists the parameters available to customize the jobs.
| Parameter Name | Values | Description |
|---|---|---|
| 0 or 1 | If this parameter is set to 1, error recycling is triggered and rejects from previous job executions are recycled in this job. |
| 0, 1 or JSON object | If this parameter is set to 1, statistics collection is triggered in the MDM hub tables to optimize processing. This option is useful to accelerate the processing of large datasets. The default value for this parameter is 1. For PostgreSQL, you can also provide a JSON object to configure how statistics are collected. A sample JSON object is provided below. Sample configuration |
| 0 or 1 | If this parameter is set to 1, consecutive SemQL enrichers are merged into a single SQL statement when executed. This applies to all entities. |
| 0 or 1 | |
| 0 or 1 | If this parameter is set to 1, consecutive plug-in enrichers are piped, so that the output of an enricher is directly passed in memory to the next one. This applies to all entities. |
| 0 or 1 | |
| Number | When the plugin execution and database are fast, the network can be the bottleneck. A batch update is a group of updates sent together to the database in one batch, rather than sending the updates one by one.This JDBC batch update size is used by plug-in enrichers when writing records to the database. A high value for this parameter reduces the number of network access to write the data but increases the memory load on the server. This parameter applies to plug-ins running individually. For chained plug-ins, it applies to the last plug-in in the chain, which writes enriched data to the database. |
Enricher Aggregation
Enrichers support aggregation for faster processing. When not aggregated, enrichers run one after the other, reading, modifying and then updating the data into the database. With aggregation, you avoid excessive database read/write operations by forcing several enrichers to run in a single operation.
Enricher aggregation follows the rules listed below:
Only successive enrichers of the same type (SemQL or Plug-in) can be aggregated. For example, if you have the following sequence of enrichers SEMQL_1, SEMQL2, PLUGIN_1, PLUGIN_2, SEMQL3, you can aggregate SEMQL_1 with SEMQL2 and PLUGIN_1 with PLUGIN_2.
Plug-in enricher aggregation will stop when:
A plug-in enricher has a filter that uses an attribute updated by a previous enricher in the chain.
A plug-in enricher has an input that contains a complex SemQL expression with attributes updated by a previous enricher in the chain.
Jobs Sequencing and Parallelism
If two jobs can run simultaneously, they should be in different queues. For example, if two jobs address two different areas in the same model, then these jobs can run simultaneously in different queues.
| It is not recommended to configure jobs processing the same entity to run in different queues. Instances of these jobs running simultaneously in two different queues may write conflicting changes to this entity, causing major data inconsistencies. |
Designing Integration Jobs
It is possible to create jobs specific to the data loads performed by the data integration batches, and dedicated jobs for data authoring and duplicate management operations.
Integration Jobs for Data Integration
Data published in batch may target several entities.
It is recommended to define jobs specific to the data loads targeting the hub:
Such jobs should include all the entities loaded by the data load process.
In addition, it is recommended to include the entities referencing the entities loaded by the data load process.
Integration Jobs for Authoring
A stepper (used directly or in a workflow) manipulates data in several entities.
If a stepper (or the workflow using it) is created without a reference to a job, a job is automatically generated and attached to that stepper when the model is deployed. This generated job processes all the entities involved in that stepper.
It is also possible to define a specific job for the stepper:
Such job should process all the entities involved in the stepper.
In addition, if some of these are fuzzy matched entities, the job should process all the entities referencing these entities.
Integration Jobs for Duplicate Management
A duplicate management action handles duplicates detected on a specific entity.
It is possible to define a specific jobs for each duplicate management action:
Such job should process the entity involved in the duplicate management workflow.
In addition, if the entity involved in the duplicate management workflow is a fuzzy matched entity, the job should process all the entities referencing this entity.
WORKING WITH APPLICATIONS
Introduction to Applications
Applications Features
These applications may:
Display relevant information organized in a business-friendly way.
Support data management operations, such as data authoring.
Support user collaboration for data management operations.
A Data Steward would need access to the data and errors to fix them.
Browsing and Searching Data in Applications
Authoring Data in Applications
The data visible in the applications reflect two streams of information :
External source systems that publish their data to the MDM hub.
Users that author (create, edit, import, etc) data in the MDM applications.
Publishing Data to the Hub
Data curated in external source systems (known as the Publishers) is published to the MDM using data integration. This data goes through the certification process to create golden data for the three types of entity (basic, ID and fuzzy matched):
For ID and fuzzy matched entities, the source data from the publishers becomes master data, which is matched and consolidated into golden data.
For basic entities, there is no notion of publisher and source data is directly converted to golden data with no matching and consolidation.
Authoring Data
There are two ways of authoring data in an application:
This method provides a simple "edit and save" approach to authoring.
This method provides a more collaborative way of authoring, where multiple users with different roles can edit subsets of the information and perform review tasks.
The authoring data lifecycle differs depending on the entity type. The next sections explain the various patterns.
Authoring Basic Entities
With basic entities:
Creating a new record creates a golden record via the certification process.
Updating a golden record modifies it and submits the changes to the certification process.
Importing either creates a new record if the imported record ID is not found in the hub, or updates the existing record that is found.
An update performed on an erroneous record takes this record, modifies it and resubmits it to the hub.
| Basic entity records authoring uses direct authoring or workflows, with all the capabilities of steppers. You can manage complex clusters of entity records (customers and contacts) within steppers. |
Authoring ID or Fuzzy Matched Entities
With matched entities, there are two types of golden records:
Matched entities authoring works on golden records or master records.
When authoring golden records:
Updating a golden record or fixing a golden error depends on the record type:
The import and mass update actions on golden records automatically take care of the record type, using the golden record update or override patterns as needed.
| Golden records authoring uses direct authoring or workflows, with all the capabilities of steppers. You can manage complex clusters of entity records (customers and contacts) within steppers. |
When authoring master records:
| Master records authoring is more suitable for data stewards to create or fix records before the publishers in an interim period, that is before the publisher actually pushes those records. It is recommended to use golden record authoring for business users. |
Managing Duplicates in Applications
Master records in fuzzy matched entities are automatically matched using rules defined in the entity matcher.
When these records are incorrectly matched, data stewards can supersede the matcher and manually match and merge records, additionally overriding values in the consolidated golden data. Similarly, when the matcher has suggested merges without actually applying them, stewards can review, approve or reject these suggestions.
Application Components
An application is made up of the following components:
FormsandCollectionsused in business views to one or multiple records.
Action Setsdefining the data management operations (creating new records, editing a record) available on business views.
Duplicate Managersto review, merge or split groups of matching records and edit merged golden records.
Workflowscomposed of tasks for users to collaborate on data management.
Browsing or searching a business view
Creating an Application
To create an application:
Name: Internal name of the object.
Required Role: The application is by default available for all users. To restrict access to this application to a specific role, select it from the drop-down list.
Default Action: Select the action that will be the Home Page of the application. As the application has no other component yet, you have the choice between the Global Search, the Inbox and the Root folder.
Title: The short name of the application that appears in the navigation drawer’s title.
Avatar: An icon representing the application. It appear in the navigation drawer’s title, and is also used to represent this application on the Welcome page.
Logo: Large logo representing the application, for example on the Global Search page.
Cover: Background image of the navigation drawer’s title.
Favicon: Icon displayed in the browser tab.
| It is possible to create multiple applications for a single model and to support different views to this model. For example, a simple application for users browsing the hub through the Business Views and another application supporting advanced data authoring and management operations. |
Creating Application Components
Once the new application is created, designers can start creating the various components of the application:
Reusable components attached to the entities:
Display Cardsto define how entities data appear in compact form.
Formsto define how a single entity record appears.
Collectionsto define how multiple entity records appear.
Search Formsto define how to search and filter the entities records.
Steppersto define how records in entities are authored.
Duplicate Managersto define how duplicate records are manually managed.
Action Setsto define groups of data management actions available for entities.
Business Viewscomposed of related entities, using the forms, collection, display cards and search forms to display and search the data.
Reusable components attached to the model:
Workflowsfor users to collaborate on data management operations.
Application components:
Application Actions and Foldersto organize the possible operations in the applications.
Navigation Drawercontaining shortcuts to applications and folders.
Global Search Configurationfor the global search.
Create Application Components Wizard
To automatically create application components for an entity:
The wizard automatically performs the following operations:
The list uses two lines if the display card has a secondary text, and shows an avatar if the display card has a value for the avatar.
The table uses two lines if the display card has a secondary text, and contains all the attributes defined for the entity.
The grid view is configured if a primary image is configured in the display card. It uses two lines if the display card has a secondary text.
The ID attribute expression
The following user-defined attributes in their order in the entity:
Atomic attributes
Complex atomic attributes: The complex type is therefore decomposed.
FDN attributes (and not FIDs)
The business view is added to the selected application.
Display Cards
The display card defines how an entity record appears in a list. It uses the entity’s attribute values to build the elements of the display card:
Primary Text: This text is displayed as the first line of the display card.
Avatar: This small picture is displayed in a round shape in a list.
The examples below show a display card used to show the same product record in a grid and a list.
In this example:
To create an entity display card:
Name: Internal name of the object.
Close the editor.
| Make sure you define at least one display card for the entities used in applications. A very basic display card uses the most representative attribute of the entity as the primary text. |
Forms
Forms display the details for one record for a given entity.
A form is composed of:
Form Fields, displaying one SemQL expression built from the attributes of the record. For example,Customer Last Nameor theCustomer’s Account Manager Nameare form fields.
Embedded Collections, displaying a list of records related to the current record. For example, theEmail Addressesof the customer.
Creating a New Form
To create a form:
Name: Internal name of the object.
Default Form: Select this option if the form should be used as the default form for this entity.
When creating a form:
A tree table showing the containers, fields and collection composing the form. You can edit the name, label and value for each form component from the tree table.
The toolbar shows buttons to create, organize or delete tabs, sections, containers, form fields and embedded collections.
Designing the Form Layout
Form Layout Concepts
The form layout uses the Flex model, which accelerates the design and automatically adapts the form to the size of the screen at run-time.
At run-time, on smaller devices, the layout direction switches automatically to Vertical, and the elements stack up, avoiding horizontal scrolling.
Forms support three types of containers for layouts:
Sections: They work like visible containers. They have a name and a visible label.
The mockup below illustrates the layout concepts:
In this example:
Section_3(vertical) contains four form fields that stack vertically.
Creating Form Containers
To create a form tab:
In the toolbar, select the
Drag the button to the form tree table. A new tab appears.
Display Properties
Icon: Select an icon from the image library that can be used in addition or instead of the label.
Display Form Tab As: label, icon or icon and label
Visible: This boolean property indicates under condition the tab is visible.
Label Width: Width of the labels in this tab. This with can be expressed in pixels ('200px') but other css units are also supported. For example: '15%' or '100em'.
To create a form section:
In the toolbar, select the
Display Properties
Visible: This boolean property indicates under which condition the section is visible.
Layout DirectionandRelative Width. See theform layoutsection for more information.
Label Width: Width of the labels in this section. This with can be expressed in pixels ('200px') but other css units are also supported. For example: '15%' or '100em'.
To create a form container:
In the toolbar, select the
Drag the button to the form tree table, in a tab or container. A new container appears.
Display Properties
Layout DirectionandRelative Width. See theform layoutsection for more information.
Label Width: Width of the labels in this container. This with can be expressed in pixels ('200px') but other css units are also supported. For example: '15%' or '100em'.
Creating Form Fields
To create a form field:
Configuring Form Fields
A form field has a set of properties to configure its behavior and appearance.
Name and Definition
Use Custom Label: Fields use by default the label and definition of their entity attribute, or a generated label for computed fields. Select this option to define a more specific label and documentation for the field.
Label: Define the label used on the form.
Display Properties
Component Type: Select a component type suitable for your field datatype.
Visible: This boolean property indicates under which condition the field is visible.
Relative Width: See theform layoutsection for more information.
Display Card: If the form field is a reference to a record, provide the display card used to display that reference.
Default Value Data Type: If your value is a SemQL expression, you can explicitly configure its datatype using this property. .In the Properties Tab, configure the other properties.
Common Form Fields Properties
The following properties are commonly used and when designing forms:
Icon, IconColorto replace the label by an icons.
Text Typography, Text Alignment, TextColor,BackgroundColorto style the field.
Display Formatto provide specific formating for text values.
Source Typeto tell the component the nature of the value (for example, whether it is an image binary or a link to the image).
Creating Embedded Collections
To create an embedded collection:
In the toolbar, select the
Drag the button to the form tree table, in a tab or container. A new embedded collection appears.
Name and Definition
Display Properties
Visible: This boolean property indicates under which condition the collection is visible.
Collection: Select the collection used to display the embedded collection.
Allow List, Table, Grid: Select which type of view is allowed for the embedded collection. Note that the selected type must be configured in the selected collection.
Relative Width: See theform layoutsection for more information.
Height: Height taken by the embedded collection, in pixels.
Filter: Filter applied to the data retrieved from the related entity.
Sort Expression: Sort expression applied to the data retrieved from the related entity.
User-Defined Sort: Check this box to allow users to customize the sort.
Limit: Maximum number or records retrieved (after filtering) from the related entity.
Display Record Count: Define whether the number of records should be displayed along the collection title.
Collections
Collections define how multiple records from the same entity appear.
Griduses the display card primary image, primary and secondary texts. It is suitable for records for which the image is the primary way of identifying a record.
Tablecorresponds to a regular table, into which the first column can also show the display card.
The following list view item uses the avatar image and two lines containing the primary and secondary text.
The following grid tile uses the primary image and two lines containing the primary and secondary text.
A collection is typically composed of:
Creating a New Collection
To create a collection:
Name: Internal name of the object.
Default Collection: Select this option if the collection should be used as the default collection for this entity.
Display Card: Select one of the display cards of the entity. SeeDisplay Cardsfor more information.
In the collection editor:
A table showing the table columns of the list. You can edit the name, label and value for each column from this table.
The toolbar shows buttons to create, organize or delete table columns.
To configure the collection:
Configure the properties for each view type:
Table
Allow Table: Select this option for this collection to support the table view type.
Line Nb.: Number of lines of each table row.
Display Card Column: Select this option to use the first table column to show the display card.
Note that unlike the other column, this first column is locked at run-time and cannot be rearranged by users.Display Card Column Label: Label used for the header of the display card column.
Display Card Column Default Width: Width of the display card column by default.
List
Allow List: Select this option for this collection to support the list view type.
Line Nb.: Number of lines of each list item.
Show Avatar: Select this option to display the avatar in the list.
Show Dividers: Select this option to display dividers between list items.
Grid
Allow Grid: Select this option for this collection to support the grid view type.
Line Nb.: Number of lines used for the text in the tiles.
Tile Height: Height of each tile in pixels, or the aspect ratio instead (e.g.: 16:9, 4:3, 1:1).
Number of Columns: Number of horizontal tiles in the grid.
Text Position: Position of the text in the tiles.
Creating Table Columns
If theAllow Tableoption is selected for the collection, then it can be displayed as a table, and you can add additional columns for this view type.
A table column has a givenValue, typically a SemQL expression.
To create a table column:
In theCollectiontab of theCollectioneditor, drag an attribute from the availableAttributeslist into the table. The new column is added to the table, with aNameand aValueequal to attribute name.
You can also drag theIn the tree table, double-click theNamecell to edit the table column name.
Repeat the operation for theLabel.
Click theValuecell, and then click the…(SemQL Editor) button to edit the table column value.
| You can select (by pressing theCTRLkey) and then drag and drop multiple attributes from the attribute list. |
Configuring Table Columns
A table column has a set of properties to configure its behavior and appearance.
The main property is the column’sComponent Type. The component type is the UI component used to display the value in the table. Component types includeText,Image,Checkbox,Date Picker, etc.
You configure the column using properties common to all types (for example, theHeader Color), and others are specific to the selected component type (for example, theImage Alignment).
In theCollectiontab of theCollectioneditor, select the table column to configure.
In thePropertiesview, set the following properties:
Name and Definition
Name,Label,Description: Define the name this field and the label used on the form.
Display Properties
Component Type: Select a component type suitable for your column datatype.
Component types are listed and described inAppendix A: Component Types and PropertiesDefault Width: Default width of the column in pixels. Users can manually resize columns and reset them to this width.
Visible by Default: This boolean property indicates whether this column should be displayed initially in the table, or should only be in the list of available columns.
Display Card: If the table column is a reference to a record, provide here the display card used to display that reference.
Default Value Data Type: If your value is a SemQL expression, you can explicitly configure its datatype using this property. .In the Properties Tab, configure the other properties.
All properties are listed inAppendix A: Component Types and Properties.
Common Table Columns Properties
The following properties are commonly used and when designing forms:
HeaderColor,Column Alignmentto style the header and align the entire column.
Text Typography, Text Alignment, TextColor,BackgroundColorto style the text is each cell.
Display Formatto provide specific formatting for text values.
Max Widthto size components.
Display Typeto choose the appearance of the component when it supports several.
Source Typeto tell the component the nature of the value (for example, whether it is an image binary or a link to the image).
| All available properties are listed inAppendix A: Component Types and Properties. |
Search Forms
ASearch Formexposes severalSearch Parameters. When the user submits a search form, a search query is issued, using these parameters as bind values.
Search Formsare used to filter data in the business views or to search for records in the global search.
A search form exposes severalSearch Parametersto look for records in an entity. When the user submits the search form, aSearch Queryis issued, using these parameters as bind values.
To create a search form:
Right-click theSearch Formsnode under an entity and selectAdd Search Form…. TheCreate New Search Formwizard opens.
In theCreate New Search Formwizard, enter the following values:
Name: Internal name of the object.
Label: User-friendly label in this search form.
Description: Description of this search form.
ClickFinishto close the wizard. TheSearch Formeditor opens.
Define aSearch Tiptext that will be displayed in the search form.
Add a search parameter:
In theSearch Parameterstable, click theAdd Search Parameterbutton. TheCreate New Search Parameterwizard opens.
In theCreate New Search Parameterwizard, enter the following values:
Name: Internal name of the object.
Binding: You use this binding string in the search form SemQL query condition to refer to this parameter. Note that as the Auto Fill box is checked, the Binding is automatically filled in. Modifying the binding is optional.
Label: User-friendly label for this search parameter. Note that as the Auto Fill box is checked, the Label is automatically filled in. Modifying this label is optional.
UseType,Length,ScaleandPrecisionto define logical data type of this parameter.
ClickFinishto close the wizard. The parameter appears in theSearch Parameterstable.
To edit a search parameter: Select thePropertiesview and then select the search parameter in the table. ThePropertiesview shows all its properties, including theName and Definitionas well as theSearch Parameters Display Properties.
To remove a search parameter, select the search parameter in the search parameters table, click theDeletebutton and then confirm the deletion.
Use thePropertiesview to configure the search parameter.
Repeat the previous step to create all your search parameters.
Reorder search parameters in the table using theMove UpandMove Downbuttons.
Create aEdit Expressionbutton to open theSemQL Editor. Note that the SemQL editor displays the search form parameters bindings in theVariablessection.
| New search forms designed for an entity are not immediately available in the business views or applications using that entity. You must define aSearch Configurationin thebusiness viewsto enable or disable both the built-in search types and your own search forms. |
Search Parameters Display Properties
TheDisplay Typeof a search parameter defines the component used to display this parameter in the search form. A Display Type is optionally configured through display properties.
The display type available for a search parameter depends on its logical data Type. For example, theDate Pickerdisplay type is available only for aDateTimesearch parameter.
The display types are listed below with their properties:
Text Field: Simple text field. This type is available for all logical data types.
Checkbox: Simple checkbox. This type is available for the Boolean logical data type.
Date Picker: Date (and time, for a timestamp datatype) selector. This display type is available for the Date and Timestamp logical types.
Drop Down List: Selectable list of elements. This display type is typically used to select a search parameter value from a list of values. It is configured using the following options:
Display Format: Defines whether the label and/or code of the list of values should be displayed.
Sort Order: Defines if the list of values should be sorted by the code or label.
Value Picker: Auto-complete component used to select filtered values from a given entity of the model. It is configured using the following options:
Lookup Entity: Entity from which the values are selected.
Filter Expression: SemQL expression used to filter the entity data. This expression can use other search parameters' values.
Bound Select Expression: SemQL expression defining the value returned by a value picker selection.
Primary Text Expression: SemQL expression defining the value displayed in the auto-complete items' first line.
Secondary Text Expression: SemQL expression defining the value displayed in the auto-complete items' second line.
Use Distinct: Check this option to only display distinct values in the auto-complete component.
Display Count: Check this option to display the count of records for each distinct value in the auto-complete. This option is available ifUse Distinctis selected
Sort By Count: IfDisplay Countis selected, this option allows sorting by ascending or descending record count.
Default Sort Expression: IfUse Distinctis not selected, this SemQL expression defines the sort order of the records in the auto-complete.
SemQL Condition for Search
TheSemQL Conditiondefined for the search form can use attributes from the search entity as well as attributes from related (parent or child) entities. Refer to a search parameter with itsBindingprefixed by a colon:.
For example, if a search form on theCustomersentity has one parameter labeledSearched Name(binding:SEARCHED_NAME), the following SemQL condition filters customers by their name.
CustomerName like '%' || :SEARCHED_NAME || '%'
If we add to this form a second parameter labeledHas Influencer Contact(Binding:HAS_INFLUENCER_CONTACT), the following SemQL condition filters customers by their name and possibly, depending on the fact that they have one or more contacts withIsInfluencer = 1
CustomerName like '%' || :SEARCHED_NAME || '%' and ( :HAS_INFLUENCER_CONTACT = 0 or any Contacts have (IsInfluencer = 1) )
| For a detailed description of the SemQL language with examples, refer to theSemarchy xDM SemQL Reference Guide. |
Steppers
When a user starts a authoring action (creating, editing, importing, etc.) on one or more records, he uses aStepper.
Introduction to Steppers
A stepper defines the cluster of related records that is manipulated when authoring, as well as the wizard-like sequence of steps that drive the user through the authoring operation.
For example, aCreateContactstepper defines that:
Authoring Contacts consists in authoringContactsand their attachedAddresses. So the cluster of records contains several contacts and for each of them several addresses records.
Creating a contact goes through 3 steps:1.General Info,2.Addressesand3.Comments. The second step (2.Addresses) is itself composed of two sub-steps:1.Address Dataand2.Phone Numbers.
Collection and Form Steps
A stepper contains two types of steps:
Collection Stepsshow a set of records. The user may select one or more records and edit these records.
Form Stepsshow attributes of one record. A form step displays attributes using the layout and contents defined in aform section or tab. Multiple form steps are sequenced under a collection step, guiding users through the steps to create or modify a record.
| When using a stepper to author master data on behalf of a publisher (SeeAuthoring ID or Fuzzy Matched Entitiesfor more information), only the first level of the stepper is used, and second level collections steps are ignored. |
Triggers and Validations
As part of the data authoring flow, a steppers supports automated data transformations and data validations.
Automated transformations take place in the form of calls toEnrichersdefined in the entities, or toProceduresdeclared in the model.
Data validations check that the data complies with data quality rules defined in the model, such as mandatory attributes, unique keys, SemQL validations, etc. When a data validation fails, it is raised to the user and may prevent that user from proceeding in the stepper.
Creating a New Stepper
To create a stepper:
Right-click theSteppersnode under an entity and selectAdd Stepper. TheCreate New Stepperwizard opens.
In theCreate New Stepperwizard, enter the following values:
Name: Internal name of the object.
Label: User-friendly label in this stepper form.
On Finish Job: Normally this should be left empty. Semarchy xDM automatically generates a job for you. But in some advanced cases you may need the ability to specify a particular job to execute. In these cases you may select the job to execute when the user completes the stepper and submits the data.
Collection: Select the collection used to display the entity records at the root level of the stepper.
ClickNext.
TheCreate Stepsstep of the wizard displays the forms defined for the entity, expanded to the section level.Select the tabs or sections from which you want to create steps:
Each selected form tab will appear as a form step showing all its child sections and form fields.
Each selected form section will appear as a form step showing all its form fields.
ClickFinishto close the wizard.
TheSteppereditor opens. A root collection step is created in theStepstree table, with child steps for the selected tabs and sections. Now you may configure the existing steps or add new steps.
Configuring Steps
Once the root level of the stepper has been defined, you can:
Add new collection steps to author child records.
Add new form step under existing form steps.
Configure the collection steps
Configure the stepper flow.
Adding a Collection Step
You can create a collection step to author records referencing the record currently handled by the stepper through a reference relationship.
For example, if a collection step managesCompanyrecords, you can create a child collection step to manageContactrecords, asContactreferencesCompanythrough aContactBelongToCompanyrelationship.
To create a collection step:
Identify the relationship you are interested in. The current collection step corresponds to the parent entity. Decide which child entity you want to add to the stepper and which reference joins the parent entity to the child entity.
In theSteppereditor, scroll down to theStepstree table.
Select the collection step under which you want to create a collection step.
In this table’s toolbar, click the
TheCreate New Collection Stepwizard opens.In theCreate Collection Stepwizard, enter the following values:
Name: Internal name of the object.
Label: User-friendly label in this collection step.
Reference: Reference to follow to identify the child records to author.
Collection: Select the collection used to display the child records.
ClickNext.
TheCreate Stepsstep of the wizard displays the forms defined for the child entity, expanded to the section level.Select the tabs or sections from which you want to create steps:
Each selected form tab will appear as a form step showing all its child section and form fields.
Each selected form section will appear as a form step showing all its form fields.
ClickFinishto close the wizard.
A collection step is created, with child steps for the selected tabs and sections.Select this collection step and then use theMove UpandMove Downbuttons in the toolbar to position the step.
Adding Form Steps
You can create additional form steps under collection steps.
To add form steps:
In theSteppereditor, scroll down to theStepstree table.
Select the collection step under which you want to create a form step.
In this table’s toolbar, click the
TheCreate New Form Stepwizard opens.In theCreate Form Stepwizard, the tabs or sections from which you want to create steps:
ClickFinishto close the wizard.
New child steps are created for the selected tabs and sections.Select these steps and then use theMove UpandMove Downbuttons in the toolbar to position them.
Configuring Collection Steps
The collection steps show records being authored. Its aspect and content is configurable through certain properties.
To configure a collection step:
In theSteppereditor, scroll down to theStepstree table.
Select the collection you want to edit.
In thePropertiesview, set the following properties:
Name and Definition
Customized Display Card: Select a display card used to represent the authored record in a child step of this collection step. Typically this is a Name and ID to easily identify the record.
Collection Data. Note that the root collection step is a specific collection step for which these properties cannot be configured.
Reference: Reference to follow to identify the child records to author.
Filter: This SemQL condition filters the records to checkout when performing an edit operation that automatically adds to the stepper data child records.
For example, if the edit operation using this stepper should only add child records whoseStatusattribute is not set toObsolete, use the following SemQL expression:Status <> 'Obsolete'
Display
Collection: Collection used to display the records in the collection step.
Default Display Type: Default view type used for this collection.
Customized Sort: Select this option to sort the records in the collection according to the Sort Expression.
Sort Expression: Sort expression applied to the data in the collection.
User-Defined Sort: Check this box to allow users to customize the sort.
Allow Table,Allow ListandAllow Grid: Select those of the views available for the collection. These must be supported by the selected Collection.
Custom Collection Label: Label (plural) used as the title of the collection. It defaults to the Plural Role Label of the Referencing Entity for the selected reference.
Configuring the Stepper Flow
The stepper flow defines:
The visibility of steps depending on the authoring situation (whether you are creating or editing a record)
Whether a step is read-only or allows authoring.
The availability or options in each situation.
Collection Steps:
The following properties of the collection steps allow designers to configure the flow of the stepper.
InName and Definition
Enable Finish: Adds a finish button on the step to finish the sequence of step it belongs to. This is useful when all subsequent steps are optional. See the Tips section on the next page for additional advice.
On the root collection step, this allows the user to submit the data in the stepper.Enable on Parent Create: Makes this step visible when the parent record is being created, or, for the root collection step, if the stepper was started to create a record.
Enable on Parent Edit: Makes this step visible when the parent record is being edited, or, for the root collection step, if the stepper was started to edit a record.
Read-Only: Makes this step read-only, preventing any data change operation.
Linear: Forces the user to go through the child steps in their order. If this option is unchecked, users can jump to any of the child steps.
Delete Type: Define the type of delete to apply when a user selects to delete a record edited in the stepper. SeeEntity Records Hard and Soft Deletefor more information.
InOn Parent Create
Skip to Child Creation on Parent Create: Skip the collection step and jump directly to the first visible child step when the parent record is being created, or, for the root collection step, if the stepper was started to create a record.
Enable Create,Enable Edit, etc: Activate the checked action in the stepper when the parent record is being created, or, for the root collection step, if the stepper was started to create a record. Note thatEnable Removeonly removes created records from this stepper, but does not delete golden data from the hub.
InOn Parent Edit
Skip to Child Editing on Parent Create: Skip the collection step and jump directly to the first visible child step when the parent record is being edited, or, for the root collection step, if the stepper was started to edit a record.
Enable Create,Enable Edit, etc: Activate the checked action in the stepper when the parent record is being edited, or, for the root collection step, if the stepper was started to edit a record. Note thatEnable Removeonly removes records from this stepper, leaving them untouched in the hub. TheEnable Deleteoption allows deleting golden data from the hub.
Form Steps
The following properties of the form steps allow designers to configure the flow of the stepper.
InName and Definition
Enable Finish: Adds a finish button on the step to finish the sequence of step it belongs to.
Enable on Parent Create: Makes this step visible when it is used in a record creation operation.
Enable on Parent Edit: Makes this step visible when it is used in a record editing operation.
Read-Only: Makes this step read-only.
Enable Master Value Picking: This option applies to form steps for ID or fuzzy matched entities. It enables users overriding a consolidated value to select the override from the master records in addition to being able to enter their own values.
Tips for Configuring the Stepper Flow The following guidelines help configuring the flow of a stepper:
|
Configuring Triggers and Validations
In a stepper, it is possible to automate data changes using triggers and enrichers. It is also possible to run data validations and raise issues to the user.
Data changes or validations run when certain events take place in the stepper. For example, when the user enters a stepper or a step, modifies a field value or leaves a step.
Configuring Triggers and Enrichers
Automated data transformations include:
Stepper Triggersthat execute database procedure calls when the stepper starts or finishes.
Step Triggersthat execute enrichers or database procedure calls when the user enters, leaves or performs certain operations in the step.
Form Step Enrichersthat run enrichers when the user opens a form step or modifies data in this form step.
Triggersrefer either to Enrichers or Database Procedures.
Stepper Triggers
A stepper trigger can execute a procedure declared in the model. Enrichers cannot be attached to the stepper as procedures can. This is because enrichers always affect an individual record and must therefore be attached to the relevant step. Procedures may be attached to steps or to the stepper itself.
To create a stepper trigger:
In theSteppereditor, scroll down to theTriggerstable.
In the table toolbar, select the
In theStepper Triggerwizard step, enter the following values:
Name: Internal name of the object.
Procedure: Select a Database function defined as a procedure in the model.
Event: Select theeventthat should cause this procedure to run.
ClickNext.
TheConfigure Procedurestep of the wizard displays the arguments of the selected procedure.For each argument, click the
ClickFinishto close the wizard. A stepper trigger is created
Select this stepper trigger and then use theMove UpandMove Downbuttons in the toolbar to order the list of triggers. Triggers, for a given event, are executed in the list order.
Step Triggers
A step trigger can execute either a procedure or an enricher on specific step events.
To create a step trigger:
In theSteppereditor, scroll down to theStepstree table.
Select the step you want to modify.
In thePropertiesview, select theStep Triggersfinger tab.
This tab shows a table with the list of triggers for the step.In the table toolbar, select the
In theStep Triggerwizard step, enter the following values:
Name: Internal name of the object.
Procedure/Enricher: Select a database function declared as a procedure or an enricher.
Event: Select theeventthat should cause this procedure or enricher to run.
If you create a Step Procedure Trigger:
ClickNext. TheConfigure Procedurestep of the wizard displays the arguments of the selected procedure.
For each argument, click the
ClickFinishto close the wizard. A step trigger is created
Select this step trigger and then use theMove UpandMove Downbuttons in the toolbar to order the list of triggers. Triggers, for a given event, are executed in the list order.
Form Step Enrichers
A form step enricher executes an enricher when the user enters a form or modifies the value of a form field.
To configure step enrichers:
In theSteppereditor, scroll down to theStepstree table.
Select the form step you want to modify.
In thePropertiesview, select theForm Step Enricherfinger tab.
This tab shows a table with the list of enrichers defined for the fields that appear for this form step.Click the
Check the appropriate option to run enrichers:
On Form Open: When the user opens the form. Note that the enricher is triggered every time the user opens the form, including when resuming the stepper or when returning from another step.
On Data Change: When the user changes the value of an attribute that is an input for the enricher.
| Note that data change is detected when the user leaves the field, when a selection is made in the field (for example, for menus) or after a certain time when the user has performed a change but remains in the field. |
Stepper Events
This section lists the various events that may cause triggers to start for steps and steppers.
The detailed list of events and their corresponding user actions are listed inAppendix B: Stepper Events Reference
Events for Steppers
A user may exit a stepper using different actions:
Finish: The user completes the stepper and submits the records for certification.
Close > Discard: The user completes the stepper and discards all its content. The stepper is canceled and cannot be resumed.
Close > Saved: The user completes the stepper and saves its content for later. He can resume this stepper later.
The following events occur once when a user enters or exits a stepper.
Stepper - Start: Run when the stepper starts
Stepper - Finish (Pre Validation): Run once, when the user finishes the stepper, before the stepper validations
Stepper - Finish (Post Validation): Run once, when the user finishes the stepper, after the stepper validations
Stepper - Close (Discard): Run once, when the user closes the stepper and discards data changes.
Stepper - Close (Saved): Run once, when the user closes the stepper and saves draft data
Events for Steps
A user may exit a step using different actions:
The user clicks theContinueorBackbuttons.
The user clicks (for a non-linear stepper) a step in the header to jump to that step.
The user clicks theClosebutton to go one level up in the stepper.
The events behavior differ for collection steps and form steps:
Event on Steps apply to both collection steps and form steps.
Events specifically indicated for Collection Step do not apply to form steps.
Events used in form steps run once for the record managed in the form step.
Events used in collection steps have two flavors, indicated with(Each)or(Once):
(Each): This type of event runs once for each record in the collection step. It gives you access to the attribute of these records.
(Once): This type of event runs once for all record in the collection step. It gives you access to the attributes of theparent record of these records. From this parent, you can use SemQL’s
any|allsyntax.
| For a detail of the sequence of events triggered for each user action, seeAppendix B: Stepper Events Reference. |
The following events occur depending on user actions for form and collection steps:
Step - Enter (Once): Run once, when a step is entered
Step - Enter (Each): Run for each record, when a step is entered
Collection Step - Add Child: Run whenever a child record is added to this collection (directly or through the ADD ANOTHER button of a child step)
Collection Step - Copy Child (Once): Run once, when a child record is copied in this collection. Note that you can use in such trigger the
CopiedFrombuilt-in attribute which contains the ID of the source record that was copied.Collection Step - Copy Child (Each): Run for each copied record, when a copy child record action is performed in this collection. Note that you can use in such trigger the
CopiedFrombuilt-in attribute which contains the ID of the source record that was copied.Collection Step - Edit Child (Once): Run once, when a child record is edited on this collection (directly or through the EDIT NEXT button of a child step)
Collection Step - Edit Child (Each): Run for each record, when a child record is edited on this collection (directly or through the EDIT NEXT button of a child step)
Collection Step - Import (Once): Run once when a user completes an import
Collection Step - Import (Each): Run for each imported record, when a user completes an import
Collection Step - Remove Child (Once): Run once, when child records are removed from this collection
Collection Step - Remove Child (Each): Run for each record, when child records are removed from this collection
Step - Continue (Once): Run once, when a user clicks the CONTINUE button
Step - Continue (Each): Run for each record, when a user clicks the CONTINUE button
Step - Back (Once): Run once, when a user clicks the BACK button
Step - Back (Each): Run for each record, when a user clicks the BACK button
Step - Close (Once): Run once, when a user clicks the CLOSE button on a step (whether to get back to the parent collection or to close the stepper)
Step - Close (Each): Run for each record, when a user clicks the CLOSE button on a step (whether to get back to the parent collection or to close the stepper)
Step - Exit (Pre validation - Once): Run once, when a user exits a step, before data validation
Step - Exit (Pre validation - Each): Run for each record, when a user exits a step, before data validation
Step - Exit (Post validation - Once): Run once, when a user exits a step, after data validation
Step - Exit (Post validation - Each): Run for each record, when a user exits a step, after data validation
Collection Step - Close Child (Once): Run whenever a child step of this collection is closed
Collection Step - Close Child (Each): Run whenever a child step of this collection is closed
Configuring Validations
Validation types include:
Stepper Validationscheck data in any entity modified in the stepper in order to warn or block the user. They run when the stepper finishes.
Step Validationscheck data in the entity modified in the step in order to warn or block the user. They run when the step completes.
Form Validationscheck data in the entity modified in the step in order to warn or block the user. They run when the user opens or modifies data in the form step.
Validationsrefer to thedata quality rulesdefined for the entities, including:
FOREIGNto check a that a referenced value exists.
MANDATORYto check that a mandatory attribute or reference is not left null.
CHECKto run a SemQL or Plug-in Validation.
| A special validation type calledDETECT_DUPSis available for Stepper and Step Validations. When the user is creating a new record, this validation type run the matching rule defined for the entity and shows possible duplicates, allowing the user to replace the new record in the stepper by an existing duplicate. |
Stepper Validations
A stepper validation runs a validation when the stepper completes and the user tries to apply the data changes.
If a validation fails, it is raised to the user, and may possibly prevent him from applying the data changes.
To configure stepper validations:
In theSteppereditor, scroll down to theValidationstable. This table shows the list of validations defined for the entities managed in the stepper.
Click the
Select the validation you want to run or select multiple validations while keeping the
<CTRL>key pressed.Right-click and then select the behavior of this validation:
Skip: Do not run this validation
Warn: Run this validation and raise a failure as an issue to the user. The user may choose to ignore the warning.
Block: Run this validation and raise a failure as a blocking issue to the user. As long as blocking issues exist, the user cannot proceed.
Step Validations
A step validation runs a validation when the user leaves a step.
If a validation fails, it is raised to the user, and may possibly prevent him from leaving the current step.
To configure step validations:
In theSteppereditor, scroll down to theStepstree table.
Select the step you want to modify.
In thePropertiesview, select theValidationsfinger tab.
This tab shows a table with the list of validations available for the entity managed in the step.Click the
Select the validation you want to run or select multiple validations while keeping the
<CTRL>key pressed.Right-click and then select the behavior of this validation:
Skip: Do not run this validation
Warn: Run this validation and raise a failure as an issue to the user.
Block: Run this validation and raise a failure as a blocking issue to the user. As long as blocking issues exist, the user cannot proceed.
Form Validations
A form validation runs a validation when the user enter or modifies a form. Such a validation does not block the user, but raises a message under the field(s) that causes the validation failure. Combine the use of a form validation (which shows useful information but is non-blocking) with a step validation (blocking) if the user should be forced to fix an issue before proceeding.
To configure form validations:
In theSteppereditor, scroll down to theStepstree table.
Select the form step you want to modify.
In thePropertiesview, select theForm Validationsfinger tab.
This tab shows a table with the list of validations available for the entity managed in the step.Click the
Check the appropriate option to run validations:
On Form Open: When the user opens the form.
On Data Change: When the user changes the value of an attribute that is an input for the validation.
Configuring References Management
Reference Selection
If a form step contains Reference fields, then it is possible to configure the selection of these references.
This selection is performed using one of the following components:
An auto-complete field that lists selectable records filtered by a search expression as you type in.
A reference selection dialog that shows a collection of selectable records. This second option is better if you need to have a detailed view of the records to select.
To configure reference selection:
In theSteppereditor, scroll down to theStepstree table.
Select the form step containing the references.
In thePropertiesview, select theReferences Selectionfinger tab.
This tab shows a table with the list of references managed in the step.Click the
Configure the references:
Reference Filter Type: SelectNoneif you do not want the reference to be selectable.Auto-Completedisplays an auto-complete component to select the reference.Collectiondisplays the list of selectable records in a reference selection dialog.
Picker Filter: This filter restricts the possible selection for both the auto-complete and reference selection dialog. Such a filter is used for example when selecting the Customer referenced by a Contact, to filter only those of the Customers located in the same country as the Contact.
Sort Expression: This sort expression orders the data in the selectable records.
If you choose theCollectiontype, you must configure the following options:
Picker Collection View: Select the collection used to show the selectable records in the reference selection dialog.
Search Configurations: Click this cell to select and order search methods that appear in an optional search step preceding the collection of selectable records. If no search method is selected asAvailable, then this seach step does not appear before the collection.
User-Defined Sort: Check this box to allow users to customize the sort in the reference selection dialog.
Many to Many Creation
If a child collection step points to a child entity that represent a many to many relation, then it is possible to configure selecting the record at the other side of the many to many relation to create the child records in the many to many entity.
For example, aProductStepperstepper has a child collection step to createProductMarkets. ProductMarket is an entity representing the many to many relation betweenProductsandMarkets. When the user createsProductMarkets, he wants in fact to select Markets, and have ProductMarkets created. This configuration aims at giving this capability.
To configure many to many creation:
In theSteppereditor, scroll down to theStepstree table.
Select the collection step that manages an entity representing the many to many relation.
In thePropertiesview, select theMany to Many Relationfinger tab.
Select theMany to Many Relationcheckbox, and then configure the Many to Many Creation options:
Duplicate Managers
ADuplicate Managerdefines the user interface into which data stewards review, merge, split groups of matching records, and override the values of the golden consolidated records.
A duplicate manager is used when the user triggers a duplicate management action with a selection of records. Actions include:
Review and Confirm Duplicatesto confirm groups of fuzzy matching records after reviewing these groups into details.
Merge or Split Duplicatesto reorganize groups of fuzzy matching records.
Review Duplicates Suggestionsto accept, reject or reorganized suggestions made on groups of fuzzy matching records.
In a duplicate manager:
Actions on matching groups and suggestions are performed using two alternate views:
A Graph view, into which master records, golden records, and suggestions appear as graph nodes. This view helps data stewards understanding groups/suggestions as well as the match rules that relate master records. The content of the nodes is defined by aDisplay Card.
A Table view, into which master records, golden records, and suggestions appear as rows in a tree table. This view helps data stewards understanding groups/suggestions, even for very large groups and suggestions, but relations between master records (match rules) are not exposed. The appearance of the table is defined by aTable View Collection.
The user may select and add records to the duplicate management operation. He selects these records from aCollection, optionally sorted and filtered according to search configurations.
The user can visualize the details of a record as defined in a selectedForm Tab.
The same form tab is also used in theExplain Recordview that shows the master records values that consolidate to compose a golden record.
The form tab is also used as an authoring form to override the values of the consolidated golden record.
| A duplicate manager provides features that are available to the user depending on the configuration of the actions in theaction setthat use this duplicates manager. |
To create a duplicate manager:
Right-click theDuplicate Managersnode under an entity and selectAdd Duplicate Manager. TheCreate New Duplicate Managerwizard opens.
In theCreate New Duplicate Managerwizard, enter the following values:
Name: Internal name of the object.
Label: User-friendly label for this object.
On Finish Job: Normally this should be left empty. Semarchy xDM automatically generates a job for you. In some advanced cases, you may need the ability to specify a particular job to execute. In these cases, you may select the job to execute when the user completes the duplicate manager and submits the changes.
Collection: Select the collection used to select records to add to the duplicate manager.
Display Card: Select the display card used to represent the records in the graph.
Form Tab: Select the form tab used to show or edit values of a record.
ClickFinishto close the wizard. TheDuplicate Managereditor opens.
Configure how value overrides take place for golden records:
Enable Master Value Picking: This option enables users overriding a consolidated value to select the override from the master records in addition to being able to enter their own values.
By default, only the Graph view is enabled for managing duplicates. If you want to enable the Table view, select aTable View Collectionin theDisplay Section. Note that the selected collection should have theAllow Tableand theDisplay Card Columnoptions selected.
Configure in theGolden Record Selectionsection how records are selected and added to the duplicate manager:
Collection: Select the collection used to show the list of records.
Search Configuration: Click theEditbutton to select and order search methods available to filter the list of records. The user will be able to filter the records using the selected search methods before seeing them in the collection.
SelectCustomize Sortand enter a SemQLSort Expressionto sort the records in the collection. You can also enableUser-Defined Sortfor this collection.
Select the available display type for the collection using theAllow Table,Allow ListandAllow Gridproperties, and select one of these as theDefault Display Type.
Workflows
Workflows enable business users to manage the data stored in the MDM hub using an application.
| You can accelerate the design of applications byduplicating existing workflows. |
Creating a New Workflow
To create a workflow:
Right-click theWorkflowsnode and selectAdd Workflow…. TheCreate New Workflowwizard opens.
In theCreate New Workflowwizard, check theAuto Filloption and then enter the following values:
Name: Internal name of the workflow.
Label: User-friendly label for this workflow. Note that as theAuto Fillbox is checked, theLabelis automatically filled in. Modifying this label is optional.
Administrator Role: Role required to administer this workflow. Users with this role can perform any operation on this workflow.
Job: Select the job to execute when the workflow completes with aSubmitoperation. Do not select any job to have a job automatically generated for this workflow based on the manipulated data.
ClickNext. In the next page, configure the first task of the workflow:
Name: Internal name of the task.
Label: User-friendly label for this task. Note that as theAuto Fillbox is checked, theLabelis automatically filled in. Modifying this label is optional.
Assigned to Role: Select the role to which this task is automatically assigned.
Entity: Select the entity which data is manipulated in the first task.
Authoring Stepper: Select a stepper from that entity that is used for the first authoring task.
ClickFinishto close the wizard. TheWorkfloweditor opens.
The Workflow Editor
The workflow editor displays the workflow as a diagram. This diagram is used to configure, add tasks and transitions to the workflow.
This workflow contains by default the following elements:
AStart Event(circular shape) that represents the startup point for this workflow. All the start tasks are linked from this event.
AnEnd Event(circular shape) that represents the completion point for this workflow. This event is preceded by two built-in tasks calledSubmit DataandCancel Datathat represent thesubmitandcanceloperations that finish a workflow.
The firstTaskthat is linked from theStart Eventand that links to theSubmit DataandCancel Databuilt-in tasks.
| It is possible to link the start event to your tasks, but your tasks cannot link directly to the end event. They must transition to the built-inSubmit DataandCancel Datatasks that finish the workflow. |
Configuring the Workflow
The workflow can be configured from thePropertiesview. Click the background of the workflow diagram to open the workflow properties.
All workflow have the following properties sections:
Name and Definition: TheName,Label,Description,Administrator RoleandJoboptions in this section are configured when creating the workflow and can be changed here. The additionalEditable Propertiesallows you to define whether workflow properties (the label, for example) can be modified after startup.
Tasks: The list of tasks of the workflow.
Transitions: The list of transitions of the workflow.
Adding a Task
To add a task from the diagram:
In the Workflow diagram, select theAdd Workflow Tasktool in thePalette.
Click the diagram. TheCreate New Taskwizard appears.
In theCreate New Taskwizard, check theAuto Filloption and then enter the following values:
Name: Internal name of the task.
Label: User-friendly label for this task. Note that as theAuto Fillbox is checked, theLabelis automatically filled in. Modifying this label is optional.
Assigned to Role: Select the role to which this task is automatically assigned. It is the role required for the performer of this task.
Select the authoring method for this workflow task, that is oneEntityand oneAuthoring Stepperfrom that entity. This authoring stepper is used when the task is started.
ClickFinishto create the task.
The new task appears in the diagram, with a transition automatically created to the Cancel built-in task. If it is the first task added to this workflow, it is linked from the start event.
Configuring a Task
A task selected in the workflow is configured from thePropertiesview.
You can configure the following properties:
In theName and Definitionsection:
TheName,Label,Description,Assigned to Role,EntityandAuthoring Stepperproperties, configured when created the task, can be modified.
Assigner Roledefines the role required for a user to be able to assign to a user a task pending with theAssigned to Role. A user who has the workflowAdministrator Roleis also able to perform the same operation.
In theTask Managementsection:
Allow Unassignenables the task assignee to release the task to the task’sAssign To Role.
Allow Reassignenables the task assignee to assign the task to a different user who has theAssign To Role.
Max Pending Duration (min)define the time after which a notification is sent when the task is left assigned to the role and not to a user.
Max Running Duration (min)define the time after which a notification is sent when the task is left assigned to a user.
Max Total Duration (min)define the time after which a notification is sent when the workflow remains on the task.
In theEmail Notificationssection, define the email notifications sent on task triggers.
Configuring Email Notifications in Tasks
When an event occurs in a task, it can trigger an email notification sent to a list of recipients.
To configure an email notification in a task:
Select theEmail Notificationssection of the workflow task properties. This table section shows the list of notifications for this task.
Click the
Click theNamecell for this notification and edit this notification’s name.
Click theEdit Expressionbutton to edit the email subject. The subject supportsvariablesthat return contextual information about the workflow.
Click theEdit Expressionbutton to edit the email body. the body supportsvariablesas well as plain text or theMarkdownsyntax for rich text.
Click theTriggerscell to configure the event that should trigger this notification.
A notification may be triggered when the task starts, finishes, is assigned/unassigned/reassigned, or when the max durations defined whenconfiguring the taskare reached.Click theRecipientcell to configure the recipients for this notification.
Recipients are contextual to the current task/workflow, or to aSelect Taskin the workflow. You can also define a list of emails in theOther Recipients.
Configuring Triggers, Enrichers and Validations in Tasks
A task of a workflow provides interactive feedback to the user according to the stepper selected for the task.
The triggers, enrichers and validations defined in the steps are executed, but the stepper-level Triggers and Validations, running at the end of the stepper, arenotexecuted. They are replaced in workflows by theTriggers and Validationsdefined for the transitions.
Configuring the Submit and Cancel Tasks
The Submit and Cancel built-in tasks are specific corresponding to the workflow’s possible ends.
Canceldoes not require configuration other than a transition pointing to it.
Submitsupports only configuring email notifications with one single type of trigger calledJob Completes. This trigger runs the notification when the submit job of the workflow completes.
Adding a Transition
A transition links to tasks in the diagram.
To add a transition from the diagram:
In the Workflow diagram, select theAdd Transitiontool in thePalette.
Select a task the diagram or theStartevent. Keep the mouse button pressed, and move the cursor to the next task in the workflow, or the built-inSubmitorCanceltasks.
Release the mouse button.
The transition is created and a link appears between the two elements in the diagram.
| Transitions have adirection. If a transition goes from Task_A to Task_B, it only means that you can move in the workflow from Task_A to Task_B. If you want to move from Task_B to Task_A, then you must create another transition in the other direction. |
| It is possible to create multiple transitions between two tasks, even in the same direction. For example, you can have two transitions from Task_A to Task_B, each transition having a different configuration. |
Configuring a Transition
A transition selected in the workflow is configured from thePropertiesview.
You can configure the following properties:
In theName and Definitionsection:
TheName,LabelandDescriptionfor the transition.
TheIcondisplayed along with the Label when selecting this transition.
TheRequired Roledefines the role required for a user to view this transition.
Ask for Commentsprompts for comments when the task completes.
In theStepper Configurationsection:
Stepper Modedefines how the stepper of the next task starts:
Create Single Recordopens the stepper in a mode that allows the user to create only one record and then finish.
Edit Single Recordopens the stepper in a mode that allows the user to review/author only one record (the first in the list) and then finish.
Multi-Recordsopens the stepper in a mode that allows the user to manipulate multiple records, according to the other properties, including theMulti-Record Start Action.
Multi-Record Start Actiondefines how a stepper started in a Multi-Records stepper mode behaves
Edit First Recordopens the stepper on the first record, allowing to move to the next records for review/authoring.
Create New Recordopens the stepper for the creation of a new record, allowing afterward to review/author the other records.
Importopens the stepper on the new records import wizard.
Import Updateopens the stepper on the new records import wizard. These records are imported to update existing records.
Display Root Collectionopens the stepper on list of records manipulated in the workflow.
Enable Create,Enable Edit, etc: Select the actions to allow in the stepper.
In theAssigneessection:
Candidate Assigneesdefines the list of candidate assignees (users) for the next task. This list appears when a user completes the previous task via this transition.
No Candidate: No candidate users. The Assigned To role defined for the next task must be included to the list.
Members of Next Task’s Role: All the members of the Assigned To role defined for the next task.
Workflow Performers: All the users who have collaborated to this workflow.
Previous Task Performer: The user who has performed the previous task in the workflow.
All Performers of Selected Task: All the users who have collaborated to theSelected Task.
Last Performer of Selected Task: The user who has last worked on theSelected Task.
User Name from a Variable: User whose name is returned by theAssignee Name Variable.
Selected TaskandAssignee Name Variablemust be set depending on theCandidate Assigneesconfiguration.
Include Assigned to Roleadds to the list of users theAssign To Roledefined for the next task. This option must be selected isCandidate Assigneesis set to No Candidate.
The transition from the Start Event (circular shape) to the startup task of the workflow cannot be configured. The first task’s stepper automatically starts with a mode that depends on the action from the action set that was triggered and the record selection that was made. Transitions to the End Event (circular shape) cannot be edited. |
Configuring Email Notifications in Transitions
When a transition is executed, it can trigger an email notification sent to a list of recipients.
To configure an email notification for a transition:
Select theEmail Notificationssection of the workflow transition properties. This table section shows the list of notifications for this transition.
Click the
Click theNamecell for this notification and edit this notification’s name.
Click theEdit Expressionbutton to edit the email subject. The subject supportsvariablesthat return contextual information about the workflow.
Click theEdit Expressionbutton to edit the email body. the body supportsvariablesas well as plain text or theMarkdownsyntax for rich text.
Click theRecipientcell to configure the recipients for this notification.
Recipients are contextual to the transition’s previous and next tasks, to the workflow, or to aSelect Taskin the workflow. You can also define a list of emails in theOther Recipients.
Configuring Triggers and Validations in Transitions
A transition of can perform data transformations and enforce data quality checks. These checks warn the user of possible data issues and can optionally block the transition.
The following sections are available in thePropertiesview for configuring data transformations and data quality checks on transitions:
TheTriggerssection contains a list of procedures executed before or after the validations when the transition executes.
TheValidationssection contains the list of validations applicable to the entities managed in the task. Each validation is configured toWarnthe user,Blockthe transition, or you can simplySkipit.
Configuring Triggers in Transitions
A transition trigger can execute a procedure declared in the model.
To create a transition trigger:
In theTransitionproperties, select theTriggerssection.
In the table toolbar, select the
In theWorkflow Transition Procedurewizard step, enter the following values:
Name: Internal name of the object.
Procedure: Select a database function defined as a procedure in the model.
Trigger Type: Select whether the trigger should run before or after the validations defined for this transition.
ClickNext.
TheConfigure Procedurestep of the wizard displays the arguments of the selected procedure.For each argument, click the
ClickFinishto close the wizard. A trigger is created
Select this trigger and then use theMove UpandMove Downbuttons in the toolbar to order the list of triggers. Triggers, for a given Trigger Type, are executed in the list order.
Configuring Validations in Transitions
A validation runs a data quality check when the transition executes.
If a validation fails, it is raised to the user, and may possibly prevent him from proceeding.
To configure transition validations:
In theTransitionproperties, select theValidationssection. This table shows the list of validations defined for the entities managed in the workflow.
Click the
Select the validation you want to run or select multiple validations while keeping the
<CTRL>key pressed.Right-click and then select the behavior of this validation:
Skip: Do not run this validation
Warn: Run this validation and raise a failure as an issue to the user. The user may choose to ignore the warning and proceed.
Block: Run this validation and raise a failure as a blocking issue to the user. As long as blocking issues exist, the user cannot proceed.
Action Sets
Action sets are groups of actions that appear together in a business view menu to modify one or more records.
Possible actions, depending on the entity type, include:
Createnew records.
Editselected records.
Importrecords to create new records or update existing records.
Copyselected records.
Mass-Updateselected records or all records.
Confirm Duplicatesto directly confirm groups of fuzzy matching records.
Review and Confirm Duplicatesto confirm groups of fuzzy matching records after reviewing these groups into details.
Merge or Split Duplicatesto reorganize groups of fuzzy matching records.
Review Duplicates Suggestionsto accept, reject or reorganized suggestions made on groups of fuzzy matching records.
Deleteselected records or all records.
Exportselected records or all records to Excel or CSV format.
Browse Graphfor a given record to graphically navigate the relations from and to this record.
Explain Recordfor a fuzzy matched record to graphically view the matches and consolidated values.
Actions that create or modify records (Create, Edit, Import, Copy and Mass-Update) require aStepperor aWorkflowto perform the operation. Similarly, actions that manage duplicates (Merge or Split Duplicates, Review Duplicates Suggestions) require aDuplicate Managerand are available only for Fuzzy Matched Entities.
Creating Action Sets
To create an action set:
Right-click theAction Setsnode under an entity and selectAdd Action set. TheCreate Action Setwizard opens.
In theCreate New Action Setwizard, enter the following values:
Name: Internal name of the object.
Label: User-friendly label for this action that appears in the action menu.
Select theCreate Default Actionto have the wizard seed default actions in the action set.
ClickFinishto close the wizard.
TheAction Seteditor opens, optionally with default actions created.
To add new actions:
In theAction Seteditor, scroll down to theActionstable.
Click the
TheCreate New Actionwizard opens.In the wizard:
Select aStepperor aWorkflowif you want to add data authoring actions. Note that stepper and workflow selections are mutually exclusive.
Select aDuplicate Managerif you want to add duplicate management actions.
Select the actions (Create,Edit,Import, etc). you want to add. Note that authoring actions are locked until you select a stepper that isvalid for these actions, and certain duplicate management actions are unavailable unless you select a Duplicate Manager.
ClickFinishto close the wizard.
The selected actions are added to the action set.Use theMove UpandMove Downbuttons to reorder the actions in the action set. The items in the action menu will appear that order.
Valid Steppers for Actions
Entities and steppers support actions under certain conditions:
Create: Stepper’s Root collection step must have one child step enabled on parent create.
Edit,Mass-Update: Stepper’s root collection step must have one child step enabled on parent edit.
Import: Stepper’s root collection step must enable child import on parent create.
Copy: The entity ID generation must be UUID or Sequence and the stepper’s root collection step must enable child copy on parent edit.
Delete: The entity must have theDelete Enabledoption selected.
Configuring Actions
To configure actions:
In theAction Seteditor, scroll down to theActionstable.
Select an action in the table.
In thePropertiesview, configure the action properties:
Name and Definition
Name,Label, andDescription.
Required Role: Optionally select a role required to perform this action.
Icon: Icon representing the action in the menu.
In theAction Configurationtab theCondition(available for Delete, Mass-Update Edit, Copy and duplicate management actions): the SemQL condition that must be met by all select records for this action to be available.
If configuring a data authoring action, in theAction Configurationtab, set the following properties:
Stepper/Workflow(only for authoring actions): Stepper or workflow used to perform the action.
Support Multiple Creation(available forCreate): Select this option to support cyclic record creation.
Created Record Type(available forCreate): Select the type of records that should be created. SeeAuthoring Data in Applicationsfor more information about master and golden data creation.
Golden Recordscreates new golden records that only exist in the MDM and are unrelated to source publishers.
Master Records(only for ID and fuzzy matched entities) creates master records on behalf of publishers. These records require that you provide the publisher, and they are matched and merged. They may be updated later on by records pushed by publishers with the same source ID.
Imported Record Type(available forImport): Select the type of records that should be handled by the import. SeeAuthoring Data in Applicationsfor more information about golden and master data import.
Golden Recordsimports new golden records or changes to existing golden records that only exist in the MDM. If importing golden data on a record consolidated from publishers, such an import is considered an overrides.
Master Records(only for ID and fuzzy matched entities) imports new or change existing master records on behalf of publishers. These records require that you provide the publisher, and they are matched and merged. They may be updated later on by records pushed by publishers with the same source ID.
Default Publisher(only forImportandCreate): This option is available for ID of fuzzy matched entities, when importing or creating master records. This is the publisher on behalf of which the master records are imported or created by default.
Allow Other Publishers: (only forImportandCreate): This option is available for ID of fuzzy matched entities, when importing or creating master records. It allows the user to select the publisher on behalf of which the records are created, or load the publisher from a column when importing.
Support Multiple Selection(available forEdit,CopyorDelete): Select this option to support this action when multiple records are selected.
Available For Golden Data(available forMass-UpdateandEdit): Select this option to support this action when browsing golden data. SeeAuthoring Data in Applicationsfor more information about golden data authoring.
Available for Master Data(available forMass-UpdateandEdit): Select this option to support this action when browsing master data. SeeAuthoring Data in Applicationsfor more information about master data authoring.
Available for Erroneous Data(available forMass-UpdateandEdit): Select this option to support this action when browsing errors. SeeAuthoring Data in Applicationsfor more information about erroneous data authoring.
Restrict Import To Update(available forImport): Select this option to restrict import to updating existing records. No new records will be created.
If configuring a duplicate management action, in theAction Configurationtab, set the following properties:
Duplicate Manager(available for duplicate management actions): Duplicate Manager used to perform this action.
Enable Edit(available forReview and Confirm Duplicates,Merge or Split DuplicatesandReview Duplicates Suggestions): Select this option to allow editing the values merged into the golden records while managing the duplicates.
Enable Confirm All(available forConfirm Duplicates): Select this option to allow the action with no records selected, which corresponds to confirming all the records.
Enable Review All(available forReview and Confirm DuplicatesandReview Duplicates Suggestions): Select this option to allow the action with no records selected, which corresponds to reviewing and confirming all the records.
Mandate Confirm All(available forReview and Confirm Duplicates): Select this option to force the user to confirm all the duplicates he has to review.
Mandate Resolve All Suggestion(available forReview Duplicates Suggestions): Select this option to force the user to confirm all the suggestions he has to review.
Prompt Golden ID(available forReview Duplicates Suggestions): Select this option to prompt the user for the surviving Golden ID when he merges two golden records while processing duplicates suggestion.
On Finish Job(available forConfirm Duplicates): Select the job to run when the duplicates confirmation is submitted. Note that if you leave this property empty, Semarchy xDM automatically generates a job for you.
If configuring a delete action, in theAction Configurationtab, set the following properties:
Enable Delete All Records(available forDelete): Select this option to allow the action with no records selected, which corresponds to deleting all the records.
Delete Type(available forDelete): Define the type of delete applied with this action. SeeEntity Records Hard and Soft Deletefor more information.
On Finish Job(available forDelete): Select the job to run when record deletion is submitted. Note that if you leave this property empty, Semarchy xDM automatically generates a job for you.
If configuring an explain record action, in theAction Configurationtab, set the following properties:
Display Card: Select the display card used to represent the records in the graph.
Form Tab: Select the form tab used to show the values of a record.
| Once the action sets are created, you can use them in business views to start data management operations, possibly using steppers or workflows. |
Business Views
In an application, business users can browse the MDM Hub content using user-friendly views calledBusiness Views.
A business view is a set of related business entities. It starts from aRoot Business Entitywhich, throughTransitions, supports navigating to otherBusiness Entities. These business entities can also through transitions lead to other business entities, etc.
For example, theCompany Hierarchybusiness view uses theCost CenterandEmployeeentities, and uses the relations that linkemployee to managers,cost centers to parent cost centers, andemployees to cost centersfor the transitions.
Business views rely on the model structure, as Business Entities are based on entities and Transitions are based on reference relationships. Business views are functional views on subsets of the overall model.
Creating Business Views
To create a business view:
In the model, expand the node of entity that should be at the root of your business view.
Right-click theBusiness Viewsnode under this entity and selectAdd Business View…. TheCreate New Business Viewwizard opens.
In theCreate New Business Viewwizard, enter the following values:
Name: Internal name of the business view.
Label: User-friendly label for this business view. Note that as theAuto Fillbox is checked, theLabelis automatically filled in. Modifying this label is optional.
ClickNext. The second step of the wizard configures the business entity at the root of the business view.
Browsing Form: Select the form that is used for showing single records for this entity.
Root Filter: This SemQL condition filters the records of the root entity that should appear when opening the business view. For example, in a business view representing the hierarchy ofCost Centers, we should filter only the cost centers with no parent cost center.
Root Label: Label used for the root business entity. It defaults to the label of the root entity.
Root Plural Label: Plural Label used for the root business entity. It defaults to the plural label of the root business entity.
ClickNext. The third step of the wizard allows you to select reference relationships to the root business entity for which you want to create a transition.
Select all the references for which you want to create a transition.
ClickFinish.
The business view editor opens on the new business view, with the transitions and child business entities created for the selected references.Optionally configure the business view by setting the following properties:
Name and Definition
Description: optionally enter a description for the business view.
Required Role: Optionally select a role required to open this business view. This role is required in addition to the privilege grants on the entities composing the business view.
Icon and Label
Icon: Select an icon from the image library to represent the root business entity.
Creating Transitions
To include more entities in a business view (for example, childCost CentersorEmployeesreporting to the cost centers), it is necessary to create more transitions to other business entities.
To create transitions:
In theBusiness Vieweditor, scroll down to theTransitiontable.
This table shows the business entities and the transitions that relate them.Select the parent business entity for the new transition.
Click the
In theCreate New Transitionwizard, enter the following values:
Transition Path: Click the
Name: Enter the name for the transition.
ClickNext. The second step of the wizard configures the child business entity of the transition:
Filter: The business entity shows records related by the transition path to the parent business entity, filtered in addition using this condition.
For example, a business view has aCompanyroot business entity and aContactchild business entity. When showing a given customer, the business view will show only the contactsrelated the given customer. The Filter condition applies in addition to that, for example to show only the contacts with a phone number.Select whether you want to create a new business entity or use an existing one.
If you choose toCreate a New Business Entity, select anEntityand give aNamefor the new business entity.
If you choose toUse an Existing Business Entity, select an existingTarget Business Entity. Note that this option is only possible if this business view already contains a business entity pointing to the same entity as the transition path.
ClickFinish
Using Existing Business Entity for Infinite Hierarchies It is possible to create hierarchies with an infinite depth by creating a transition from a business entity to itself. For example, to create a hierarchy of cost centers, create a business view with aCost centeras the root business entity (call itCostCenterBusinessEntity) and a root filter selecting only the cost centers with no parents (e.g.:ParentCostCenter is null). Then add a transition using the self-relation linking parent and child cost centers, and select the existingCostCenterBusinessEntity. |
Configuring Embedded Collection Transitions
If the form used for a business entities contains embedded collections, then you can configure an embedded collection transition to configure which records are displayed in this embedded collection and how browsing takes place when clicking these records in the embedded collection.
To configure an embedded collection transition:
In theBusiness Vieweditor, scroll down to theTransitiontable.
Select the parent business entity for the new the transition.
Click the
In theCreate New Transitionwizard, set the following values:
Embedded Collection: Select one of the embedded collections of the form of your business entity.
Name: Enter the name for the transition.
ClickNext. The second step of the wizard configures the child business entity of the transition:
Select whether you want to create a new business entity or use an existing one.
If you choose toCreate a New Business Entity, select anEntityand give aNamefor the new business entity.
If you choose toUse an Existing Business Entity, select an existingTarget Business Entity.
ClickFinish
Configuring Business Views
In addition to the structure based on business entities and transitions, the business view also includes options to configure how it appears in the application.
You typically configure the following:
In theTransition, you configure how the transition appears.
In theBusiness Entity, you configure how child records appear when displayed in a collection or individually.
In a business entity, you configure the action available in the menu by selecting anAction Set.
If a business entity uses a form with references to other entities, theReference Browsingoptions define how to navigate these references.
If a business entity uses a form with embedded collections, theEmbedded Collectionoptions define how to navigate when clicking an item in these collections.
In theBusiness ViewTransitionandBusiness Entity, you also configure the tree view showing the business viewhierarchy.
Configuring Transitions
To configure a transition:
In theBusiness Vieweditor, scroll down to theTransitiontable.
Select the transition in the table.
In thePropertiesview, configure the following properties:
InName and Definition, modify theName,Transition Path,FilterandTarget Business Entityof the transition.
Icons and Labels
Icon: Icon used to represent the child records of the transition.
Use Custom Label: Select this option to use aLabeland aPlural Labeldifferent from those of the child entity.
Display Properties
Form Tab Condition: SemQL condition that must be met for the transition tab to appear in the business view.
For example, you may only want to add theContactstab if there is a contact, which can be expressed withAny Contacts have ( 1=1 )Display Form Tab: Select how the tab should appear. With the label, icon, or both.
Configuring Business Entities
To configure a business entity:
In theBusiness Vieweditor, scroll down to theTransitiontable.
Select the business entity in the table.
In thePropertiesview, configure the following properties:
Name and Definition
Modify theNameand modify the selectedEntity.
Display Properties
Customized Display Card: Display card used to represent one record in the editor header for this business entity.
Browsing Collection: Collection used to display a list of records.
Browsing Form: Form used to display one record.
Outline Visible: Defines whether the outline should be displayed for this business entity.
Display Record Count: Defines when the record count should appear, after the customized display card in the editor header, for this business entity.
Collection Configuration
Customized Sort: Select this option to sort the records in the browsing collection according to the Sort Expression.
Sort Expression: Sort expression applied to the data in the browsing collection.
User-Defined Sort: Check this box to allow users to customize the sort.
Allow Table,Allow List,Allow Grid: Select the views available for the collection. These must be supported by the selected Browsing Collection.
Configuring an Action Set
To configure the actions available for a business entity:
In theBusiness Vieweditor, scroll down to theTransitiontable.
Select the business entity in the table.
In thePropertiesview, select theName and Definitionfinger tab.
Select anAction Setthat will define which actions are available to manage the data on that business entity.
Configuring References Browsing
Reference Browsing options define how to navigate references appearing in the form defined for a business entity.
In theBusiness Vieweditor, scroll down to theTransitiontable.
Select the business entity in the table.
In thePropertiesview, select theReferences Browsingfinger tab.
This table lists the references of the browsing form selected for the business entity.Click the
For each reference listed, select aBrowsing Target:
Not Browsablemakes the reference not navigable.
Formopens a popup with the selectedFormwhen the reference is clicked.
Business Viewopens the selectedTarget Business Viewin the same editor.
Configuring Embedded Collections Browsing
Embedded Collection Browsing options define how to navigate when a user clicks an item from an embedded collection placed in the form defined for a business entity.
In theBusiness Vieweditor, scroll down to theTransitiontable.
Select the business entity in the table.
In thePropertiesview, select theEmbedded Collectionsfinger tab.
This table lists the embedded collections of the browsing form selected for the business entity.Click the
For each collection listed, select aBrowsing Target:
Not Browsablemakes embedded collection items not navigable.
Formopens a popup with the selectedFormwhen the collection item is clicked.
Business Viewopens the selectedTarget Business Viewin the same editor.
Configuring Business View Search
A business view can be searched at every level. You configure the search capabilities for each business entity of your business view.
Semarchy xDM provides the following built-in search types to look for data in business views:
Text: The text search looks up all attributes of the business entity marked asSearchablein the model that match your search pattern. Users can use the
%wildcard to match a number of characters or_to match a single character.By Form: This method shows a default form with all the business entity attributes available, and pickers to select values to filter these attributes.
Advanced Search: Advanced search allows users to specify which attributes to search on and the operators to use for comparison.
SemQL: SemQL search allows users to use SemQL queries to search. With SemQL, users can use attributes in parent entities or child entities, as well as the SemQL library of functions.
In addition to these search types, you can design your ownSearch Forms.
The business entity search configuration defines the search methods (built-in search methods and search forms) available to filter the records of a business entity in a business view.
To define the search configuration:
In theBusiness Vieweditor, scroll down to theTransitionstable.
In this table, select the business entity you want to enable search for.
In thePropertiesview, select theDisplay Propertiesfinger tab.
TheSearch Configurationproperty lists the search methods enabled for this business entity.Click theEditbutton to open theAvailable Search Methodsselection dialog.
This table lists the available search methods (built-in and search forms) available for the root business entity.Select theAvailablecheckbox for each search method you want to make available.
Order the search methods using theMove UpandMove Downbuttons.
Configuring the Hierarchy in a Business View
A business view can be browsed using a hierarchical tree view into which users can expand and select transitions and child records as tree view nodes.
To configure the hierarchical tree view:
In theBusiness Vieweditor, scroll down to theHierarchy Configurationsection.
SelectEnable Hierarchy Viewto enable the hierarchy. This adds a "Toggle Hierarchy" button to the business view toolbar. Users use this button to show/hide the hierarchy.
SelectOpen Hierarchy by Defaultto show the hierarchy to the user the first time he browses the business view.
Select inDisplay Root Node Ashow the root node (representing the collection of root business entity records) should appear, that is with an icon and/or a label. The icon and label used for the root node are theIconandRoot Plural Labeldefined in theIcon and Labelsection of the business view editor.
Select each transition in theTransitionstable of theBusiness Vieweditor, and then in thePropertiesview, select theHierarchy Configurationtab.
SelectEnable in Hierarchyto enable this transition in the hierarchy. If the transition is disabled, then it does not appear in the hierarchy. A record for which no child transition is enabled cannot be expanded in the hierarchy. This property supports SemQL, which means that you can configure the transition’s presence in the hierarchy based on the data values. For example, use
any contacts have (1=1)to disable the transition to the contacts when there are no child contacts for a customer.SelectTransition Node Visibleto show the transition node between the parent record and the child records. If this property is false, then the child records appear directly under the parent record with no intermediate node representing the transition itself. This property supports SemQL, which means that you can configure the transition visibility based on the data values.
SelectExpand by Defaultto have this transition expand automatically when its parent is expanded.
Select inDisplay Transition Node Ashow the transition node (representing the collection of business entity records) should appear, that is with an icon and/or a label. The icon and label used for the transition node are theIconandRoot Plural Labeldefined in theIcon and Labeltab of the transition properties.
Select each business entity in theTransitionstable of theBusiness Vieweditor, and then in thePropertiesview, select theDisplay Propertiestab.
Select aRecord Node Display Cardfor the business view, that is the display card used to display in the hierarchy each record of this business entity. Note that only the primary text and the avatar (if it is defined) are used in the display card.
Tips for Creating Views and Objects
| You can accelerate the design of applications byDuplicating Objectssuch as the business views, forms and collections. |
Creating Views for Specific Purposes
It is possible to create different business views/collections and forms for specific purposes, or use advanced capabilities (particularly theVisibleflags) to hide/show parts of these graphical components contextually.
It is not necessary to have forms dedicated to data authoring, as the form components support both browsing and authoring patterns.
Selecting Attributes for Forms and Collections
The form/collections used in business views and steppers must be carefully designed keeping in mind the user experience, and should include relevant form fields/columns.
Key Attributes
The key for a record depends on the context (master record, golden record, source record, etc.) as well as the type of entity. To make form design simpler and consistent, you can simply include the primary key attribute for the entity in the form or collection. Depending on the context and entity type, the correct attribute will be displayed. For example, for a source record of a fuzzy matched entity, the SourceID will be displayed.
Unless you want to specifically display one of the key attributes (Golden ID, SourceID, etc.), you should not specifically add one of these in the form or collection.
Steppers Forms
For data authoring purposes, certain attributes are required in the form used in the steppers:
The entity primary key attribute is usually recommended. Depending on the entity type and context, the appropriate attribute is automatically displayed. See thenote about ID Generation and Steppersbelow for more information.
It is also recommended to include the mandatory fields in the data authoring forms unless null values are automatically handled by enrichers and triggers. Otherwise, mandatory value validation may reject data.
If the entity references another entity, it is recommended to include the reference (the foreign display name attribute) to enable attaching a record to its parent. For example an employee is attached to a cost center and a manager. The correspondingFDN_CostCenterandFDN_Managerattributes should be added to the form used to author the employee, for example to assign the employee to a specific cost center or manager.
ID Generation and Steppers TheID Generationused for the entity managed at a level in a stepper forces certain rules in the step and form design.
|
When a form is used for the purpose of data authoring,read-onlyfields appear as not editable. In the same context, certain attributes are automatically hidden:
Attributes from related entities. For example, a form attribute defined with
AccountManager.FirstNamefor a Customer entity will not appear in data authoring on the customer form, as it refers to a related entity. When browsing, it will show a given customer’s account manager name.Attributes that use a computed SemQL expression. For example, a form attribute defined with
CustomerNamewill allow for inputting this attribute’s value. If you define the form attribute withUPPER(CustomerName), it will appear when browsing and display the customer name in uppercase, but will disappear when the form is used for data authoring.
Application Actions and Folders
An application’sActions, organized in a hierarchy ofFolders. Action types include:
Browse a Business Viewto see its data or errors, possibly opening it with a search form.
Global Search, to search the entire application.
Inbox, to open the user’s inbox and see his work to do or done.
Open URL, to open any URL
Profile, to open the user’s profile editor.
Import, to start a data import operation in a stepper or workflow.
Create, to start creating new records using a stepper or workflow.
Browse Entities, to automatically generate a folder to browse entities' data using various pre-built views (golden, master, errors, etc).
Creating Folders
To create a folder:
Double-click theFolders and Actionsnode under the application. TheApplicationopens on theFolders and Actionstree table.
This tree table shows the hierarchy of folders and actions. All folders and actions are under a Root folder which cannot be deleted.Select a folder in the hierarchy and then click the
Add Folderbutton in the toolbar. TheCreate New Application Folderwizard opens.In theCreate New Application Folderwizard, enter the following values:
Name: Internal name of the folder.
Label: User-friendly label for this folder.
Icon: Icon used to display this folder in the navigation drawer
Image: Image used to display this folder as a tile in its parent folder.
ClickFinishto close the wizard. The folder is added to the folders hierarchy.
Creating Actions
To create an action:
Double-click theFolders and Actionsnode under the application. TheApplicationopens on theFolders and Actionstree table.
Select a folder in the hierarchy and then click the
In theCreate New Application Actionwizard, enter the following values:
Name: Internal name of the action.
Action Type: Select the type of the action.
ClickNext
If the action type requires it, enter its configuration parameters and the clickNext
Browse Business View:
Business View: Select the business view to open:
Golden Datais the data certified data in the hub.
Golden Data With Errorsis golden data flagged as erroneous.
Master Datais the latest records from source systems (for ID and Fuzzy Matched entities).
Source Datais data loaded from source system or authored by users.
Source Data With Errorsis data loaded from source system or authored by users, and rejected by the validations.
Use Search on Open: Select this option to open the business view with its configured search methods for the user to filter the data before seeing it.
View Type: Select the type of view to browse.
Open URL:
URL: URL to open.
Target: Target of the URL. You can open the URL in a new tab, the same tab or in an editor.
CreateorImport:
Create ActionorImport Action: Select the corresponding action in an action set to trigger.
Browse Entities Folder:
Lineage Enabled: Selection this option to enable lineage navigation (golden to master records, for example) in the generated views.
In theDisplay Propertiesstep, configure how the action appears:
Label: User-friendly label for this action.
Description: Description for this action. This description appears as a secondary text for this action.
Icon: Icon used to display this action in the navigation drawer
Image: Image used to display this action as a tile in its parent folder.
ClickFinishto close the wizard. The action is added to the folders hierarchy.
Organizing Actions and Folders
Use drag and drop, or theMove UpandMove Downbuttons to organize folders and actions in theFolders and Actionstree table. Note that the order set in this tree table is applied in the application at run-time if you have selectedPositionfor the application’sSort Method. If you have selected theLabelsort method, then items in folders are automatically sorted alphabetically using their label.
| In thePropertiesview, you can configure the display properties of the action in the application. For example, the color of the icon and image alignment. |
Navigation Drawer
TheNavigation Drawer, displayed on the left of the application, enables the definition of shortcuts to application actions, organized into groups. In the navigation drawer:
Navigation GroupscontainNavigation Itemspointing to application actions.
Folder Groupsare pointers to application folders. Such a group directly lists all the actions stored in the referenced folder.
To create a navigation group:
Double-click theNavigation Drawernode under the application. TheApplicationopens on theGroupstree table.
This table shows the folder and navigation groups, and the items under the navigation groups.Click the
In theAdd A Groupwizard, enter the following values:
Name: Internal name of the group.
Label: User-friendly label for this group, as it will appear in the navigation drawer.
Show Label: Select this option to show the label for the group.
Show Divider: Select this option to show a divider before the group.
ClickFinishto close the wizard. The group is added to the navigation drawer.
Use theMove UpandMove Downbuttons to order the elements in the navigation drawer.
To create a navigation item:
Double-click theNavigation Drawernode under the application. TheApplicationopens on theGroupstree table.
Select a Navigation Group in the list.
Click the
Select anActionfrom the application actions, and then clickFinish.
The item is added to the navigation drawer.Select the item and use theMove UpandMove Downbuttons to order this item within its group.
| A navigation item uses the label and icon defined in the application action. |
To create a folder group:
Double-click theNavigation Drawernode under the application. TheApplicationopens on theGroupstree table.
Click the
In theAdd a Folder Groupwizard, enter the following values:
Linked Folder: Select a folder in the application this folder group will point to.
Name: Internal name of the group.
Show Label: Select this option to show the label of the folder in the navigation drawer.
Show Divider: Select this option to show a divider before the group.
ClickFinishto close the wizard. The group is added to the navigation drawer.
Use theMove UpandMove Downbuttons to order the elements in the navigation drawer.
| A folder group uses the label and icon defined in the application folder. |
Global Search Configuration
TheGlobal Search Configurationdefines which business views are searched when a search string is entered in the global search page, and how they are searched.
To configure a global search configuration
Double-click theGlobal Search Configurationnode under the application. TheApplicationopens on itsGlobal Search Configurationsection.
Configure the global search page:
SelectDisplay "Search Form" Dropdownto allow the user to select which of the business views he wants to search. If you do not display this dropdown, all business views will be searched.
SelectSort Business Views Alphabeticallyto have the business views sorted in the drop down and in the search results alphabetically. Otherwise, their position in theSearched Business Viewapplies.
Click the
In this wizard, enter the following values:
Business View: Business view from the application to search for.
Search Type: Type of search used to search the business view. You can use theFull-Textsearch type or select a search form defined for the root business entity of the business view.
Search Parameter: If a search form is selected in theSearch Typethe search text is bound to this search parameter used in the search form query.
Max Results: Maximum number of results displayed in the search results for this business view.
Use theMove UpandMove Downbuttons to order the searched business views in the table.
Repeat the three previous steps to create additional search configurations for other business views.
Validating an Application
An application or a component of the application (business views, forms) must be validated to ensure its correct behavior after deployment and raise possible issues.
To validate the application or one component from the diagram:
In theModel Designview, select the node corresponding to your application right-click and then selectValidate.
If you have unsaved editors, select those to save when prompted.
The validation process starts. At the end of the process, the list of issues (errors and warnings) is displayed in theValidation Logview. You can click an error or waning to open the object causing this issue.
Opening an Application
You can connect to the deployed applications from the Welcome page.
To open an application:
Open a new tab in your web browser and connect to the URL that was provided by your administrator. For example
http://<host>:<port>/semarchy/where<host>and<port>represent the name and port of the host running the Semarchy xDM application. If you are not logged in yet, the Login Form is displayed.Enter your user name and password.
ClickLog In. The Semarchy xDM Welcome page opens.
In the welcome page, click one of these applications to open it.
| If an application cannot be connected, for example if its data location is in maintenance or requires an upgrade, it appears with a disabled avatar. |
| You can bookmark the URL when connected to a given application to return directly to this application. |
| When you connect to an application for the first time, its structure is generated from the model and kept in a cache. To refresh this cache with changes performed in the model or in the application, click theRefreshoption in the profile menu, in the upper right corner of the application. |
Applications Common Configuration
All applications running in a Semarchy xDM instance share configuration properties such as the header logo or the export limits. These properties are configured from the Administration Console. See theManaging the Platformchapter in theSemarchy xDM Administration Guidefor more information.
Managing Image Libraries
Image libraries store images and icons used by applications. Libraries are stored in the repository, are global to the platform and can be used by all applications. Designers use, in applications, images and icons from the libraries by reference, usingImage Library URLs.
Default Image Libraries
When the repository is created, built-in image libraries are seeded with default Semarchy xDM images and icons:
mdiis the library for all the Material Design icons and assets. This library cannot be modified.
defaultcontains Semarchy branding images as well as the default images used by applications, such as the default images for the actions. Icons provided in addition to themdilibrary are in this library. This library cannot be modified.
demo<application_name>_ contains the images/icons used in the<application_name>demonstration application. Such a library is seeded when a demo application is installed.
| The default image libraries content changes with new versions of Semarchy xDM. When an image is removed or changed in the default libraries, it is indicated as deprecated in the validation report for models using it. Model designers should update their models to use newer versions of such images. |
Browsing Image Libraries
The Image Libraries editor, available in theAdministration Consoleview, allows administrators and model designers to view and manage the image libraries.
| TheModel Designplatform privilege is required to manage image libraries. |
To browse the image libraries:
Open theAdministration Consoleperspective.
In theAdministrationview, double-click theImage Librariesnode.
TheImage Librarieseditor opens.
Use the filter box in the image libraries editor to reduce the number of images displayed. You can use the '%' sign to replace any number of character.
Managing Image Libraries
Image libraries are created and modified using file import. You can import two types of files:
Image files, for which you must select a target library.
Zip files, into which images are organized into folders. When a Zip file is imported, one library is created for each folder at the root of the Zip file, and images in this folder are imported into the library. Exporting images creates a file that follows the same format.
To import images:
Open the image libraries editor.
Click theImport Image Librarybutton. The import dialog opens.
Drop either a Zip file or multiple image files into the dialog, or clickBrowseto select a Zip file or multiple image files.
If you import image files, select theTarget Libraryfor these images. SelectCreate a New Libraryto create a new library for these images.
ClickOKto import the images.
To delete images:
Open the image libraries editor.
Click an image to select it. Repeat this operation for all images you want to delete. You can also click theSelect Alloption in the toolbar to select all the filtered images.
Click theDelete Selected Imagesbutton, and then confirm the deletion.
| If you delete all the images from a library, the library disappears. |
To export images:
Open the image libraries editor.
Click the image to select it. Repeat this operation for all images you want to delete. You can also click theSelect Alloption in the toolbar to select all the filtered images.
Click theExport Selected Imagesbutton.
A zip file is downloaded, containing all the selected images stored into folders named after their parent library.
| Using this export mechanism, you can move an image library to a remote repository, or reorganize the library before reimporting it. |
| If an image is removed from a library while being used in a model, it will raise a warning in the model validation report. Designers should fix such stale image references and update their models to point to valid images. |
MODELS MANAGEMENT
Semarchy xDM supports out of the box metadata versioning.
When working with a model in the Semarchy Workbench, the developer works on aModel Edition(version of the model).
Model management includes all the operations required to manage the versions of the models.
Introduction to Metadata Versioning
AModel Editionis at a given point of time eitherOpenorClosedfor editing.
Branching allows you to maintain two or more parallelBranches(lines) of model editions.
Model Editions and Branches
AnOpen Model Editioncan be modified, and is considered asbeing designed. When a milestone is reached and the design is considered complete, the model can be closed. When a model edition is closed, a new model edition is created and opened. This new model edition contains the same content as the closed edition.
AClosed Model Editioncan no longer be modified and cannot be reopened on the same branch. You can only edit this closed edition by reopening it on a different branch.
When a model from a previously closed edition needs to be reopened for editing (for example for maintenance purposes), a new branch based on this old edition can be created and a first edition on this branch is opened. This edition contains the same content as the closed edition it originates from.
Version Numbers
A model has a version number that is automatically generated with the Branch ID number and Model Edition number. The branch and model numbers start at zero and are automatically incremented as you create new branches or model editions.
A model edition appears in the following format:<Model Name> [<branch>.<edition>]. For example, the first model edition in the first branch has the version[0.0]. The fourth edition of the CustomerAndFinancialMDM model in the second branch is namedCustomerAndFinancialMDM [1.3].
A Chronological Example
The following example shows the chronological evolution of a model through editions and branching:
January: a newCustomerHubmodel is created with the first branch and the first edition. This isCustomerHub [0.0].
March: The design of this model is complete. TheCustomerHub [0.0]edition is closed and deployed. The new model edition automatically opened isCustomerHub [0.1].
April: Minor fixes are completed onCustomerHub [0.1]. To deploy these to production, the model editionCustomerHub [0.1]is closed, deployed to production and a new editionCustomerHub [0.2]is created and is left open untouched for maintenance.
May: A new project to extend the original hub starts. In order to develop the new project and maintain the hub deployed in production, a new branch with a first edition in this branch is created, based onCustomerHub [0.1](closed). Two models are now opened:CustomerHub [0.2]which will be used for maintenance, andCustomerHub [1.0]into which new developments are added.
June: Maintenance fixes need to take place on the hub deployed in production.CustomerHub [0.2]is modified, closed and sent to production. A new edition is created:Customer [0.3].
August: Then new project completes.CustomerHub [1.0]is now ready for release and is closed before shipping. A new editionCustomerHub [1.1]is created and opened.
The following schema illustrates the timeline for the edition and branches. Note that the changes in the two branches are totally decoupled. Stars indicate changes made in the model editions:
Month : January March April May June August Branch 0 : [0.0] -***-> [0.1] -*-> [0.2] ----------*-> [0.3] -----------> Branch 1 : +-branching-> [1.0] -**-**-****-> [1.1] --->
At that stage, two editions on two different branches remain and are open:CustomerHub [1.1]andCustomerHub [0.3].
Model Development Lifecycle
This section describes the overall model development and deployment lifecycle.
Initial Setup and Deployment in Development
The following steps are required when creating a new model in a development environment.
Create a New Model. This operation automatically creates the first edition of the model (Typically, with edition number [0.0]).
Design the first iteration of your model by creating thelogical model,certification process rulesand theMDM applications.
Run aModel Validationwhen your model is stable and ready for the first tests.
Create a development data location, using your model edition, to deploy and test your model.
The model is deployed and ready for testing. You can load data into the data location and use the applications to view and manage the data.
Making Changes in Development
After the first deployment and tests, you will repeatedly make changes to the model edition (logical model,certification process rulesorMDM applications) and test them in the development environment.
To test these changes in the development data location:
Run aModel Validationto make sure that the model is valid.
Deploy again the modelagain. This replaces the existing model by the updated one.
The updated model is immediately ready for testing. After the update, make sure to:
Run the data loads for the updated jobs to reprocess the incoming data as needed.
Click theRefreshoption in the profile menu (in the upper right corner of the application) to force a regeneration of the MDM application.
Releasing from Development
When the model is complete and tested, it is ready for release.
To release a model:
Close the model edition. This operation automatically freezes the current model edition and opens a new one.
Deploy the closed model edition using one of the following methods:
Deploying to the same repository:Deploy the closed model editioninto a production data location attached to the same repository.
Deploying to a different repository:Export this model editionfor an import then deployment to a remote repository.
Developing Iteratively after a Release
When you close a model edition, a new model edition (for example, with version number [0.1]) is automatically created.
You can proceed with your next project iteration, starting with this new model edition:
Make changes to this model edition in developmentuntil the next project iteration is ready for release.
When ready,release this model editionfrom development.
If you need to make fixes to a previously released model edition, you canbranch this old model edition, modify, then release it.
Working with Model Editions and Branches
Model edition and branching can be managed from theModel Administrationperspective.
To access the data administration perspective:
Select theOverviewperspective in the toolbar.
In theOverview, click theManage Model Editions…link in theAdministrationsection. TheModel Administrationperspective opens.
Creating a New Model
Creating a new model creates the model with the first model branch and model edition.
To create a new model:
In the menu, selectFile > New > New Model…. TheCreate New Modelwizard opens.
In theCreate New Modelwizard, check theAuto Filloption and then enter the following values:
Name: Internal name of the object.
Label: User-friendly label for this object. Note that as theAuto Fillbox is checked, theLabelis automatically filled in. Modifying this label is optional.
In theDescriptionfield, optionally enter a description for the Model.
ClickFinishto close the wizard. The new model is created, with a root branch and a first edition.
Model Editionperspective opens on the new model.
| A new model is automatically configured to run on the same target database technology as the repository. You can select a differentTarget Technologyin the Model editor. |
Closing and Creating a New Model Edition
This operation closes the latest open model edition in a branch and opens a new one. The closed model edition is frozen and can no longer be edited, but can be deployed to production environments.
| It is only possible to close an open edition. This edition is the latest one from a branch. |
| It is only possible to close an edition that is valid. |
To close and create a new edition:
In theModel Editionsview of theData Administration Perspective, expand the model and the model branch containing the edition you want to close.
Right-click the latest model edition of the branch (indicated as opened) and selectClose and Create New Edition.
ClickOKto confirm closing the model edition.
TheEnter a comment for this new model edition dialog, enter a comment for the new model edition. This comment should explain why this new edition was created.
ClickOK. The model is validated, then a new model edition is created and appears in theModel Editionsview.
| Be cautious when closing a model edition. Closing an edition cannot be undone, and a closed edition cannot be reopened. |
Branching a Model Edition
Branching a model edition enables restarting and modifying a closed edition of a model. Branching creates a new branch based on a given edition, and opens a first edition of this branch.
| It is only possible to branch fromclosedmodel editions. |
When creating a model, a first branch named<model_name>_rootis created with the first model edition. |
To create a new branch:
In theModel Editionsview of theData Administration Perspective, expand the model and the model branch containing the edition from which you want to branch.
Right-click the closed edition from which you want to branch and selectCreate Model Branch From this Edition. TheCreate New Model Branchwizard opens.
In theCreate New Model Branchwizard, check theAuto Filloption and then enter the following values:
Name: Internal name of the object.
Label: User-friendly label for this object. Note that as theAuto Fillbox is checked, theLabelis automatically filled in. Modifying this label is optional.
In theDescriptionfield, optionally enter a description for the Model Branch.
ClickFinishto close the wizard.
In theModel Branch Createddialog, clickYesto open the first edition of this new branch.
The newly created edition opens.
Target Technology
A model is designed for a given databaseTarget Technology(Oracle, PostgreSQL). Although most of the model content is technology-agnostic, the artifacts generated in the data location schema, as well as the certification process, will use capabilities specific to that technology.
When you create a model, it is automatically configured for the technology of the repository into which it is created. When you design the model, some of the capabilities, for example, the database functions available in SemQL, will depend on that technology.
You can configure theTarget Technologyvalue in the Model editor.
Make sure to check the model’s target technology when you start working with a model, or when you import a model from another repository. If you change this technology later, make sure to validate the model in order to have a list of possible issues due to the technology change.
Model Localization
When designing a model, labels, descriptions and other user-facing text strings, are entered to provide a user-friendly experience. These strings are natively externalized in Semarchy xDM, and can be translated (localized) in any language.
A user connecting an application created with Semarchy xDM will see these strings (label of the entities, attributes, list of values, etc.) translated in the locale of his web browser if such translation is available. If no translation is available in his locale for a given text, the default string (for example, the label or description specified in the model) is used. These default strings are thebase translation.
| Make sure you translate the entire model in a given language to avoid partially translated user interfaces. |
Translation Bundles
Strings translation is performed usingTranslation Bundles, attached to model editions. A translation bundle is a properties file that contains a list of key/value pairs corresponding to the strings localized in a given locale. The translation bundle file is namedtranslations_<locale>.properties, where<locale>is the locale of the translation.
The following example is a sample of a translation bundle file for the English language (translations_en.properties). In this file, the label for theEmployeeentity is the stringStaff Member, and its description isA person who works for our company.
... Entity.Employee.Label=Staff Member Entity.Employee.Description=A person who works for our company. Attribute.Employee.FirstName.Label=First Name Attribute.Employee.FirstName.Description=First name of the employee Attribute.Employee.Picture.Label=<New Key TODO> ...
Translating a Model
To translate a model:
The translation bundles are exported for the language(s) requiring translation in a single zip file.
Each translation bundle is translated by a translator using his translation tooling.
The translated bundles are re-imported into the model edition (either each file at a time, or as a single zip file).
To export translation bundles:
In theModel Editionsview (Model Administrationperspective), expand the model edition you want to localize.
Right-click and then selectExport Translation Bundles…. TheExport Translation Bundleswizard opens.
Select the languages you want to translate.
SelectExport Base Bundleif you also want to export the base bundle for reference. The base bundle contains the default strings, and cannot be translated.
Select theEncodingfor the exported bundles. Note that the encoding should be UTF-8 unless the language you want to translate or the translation tooling has other encoding requirements.
Select inValues to Exportthe export type:
Allexports all the keys with their current translated value. If a key is not translated yet, the value exported is the one specified by theDefault Valuesoption.
New keys onlyexports only the keys not translated yet.
All except removed keysexports all keys available, excluding those with no corresponding objects in the model. For example, the key for the description of an attribute that was deleted from a previous model edition will not be exported.
Select inDefault Valuesthe value to set for new keys (keys with no translation in the language).
Use values for base bundlesets the value to the base bundle value.
Use the defined tagsets the value to the tag specified in the field at the right of the selection (defaults to
<New Key TODO>).Leave Emptyset the value to an empty string.
ClickOKto download the translation bundles in a zip format and thenCloseto close the wizard.
To import translation bundles:
In theModel Editionsview (Model Administrationperspective), expand the model edition you want to localize.
Right-click and then selectImport Translation Bundles…. TheImport Translation Bundleswizard opens.
Click theOpenbutton and select the translation file to import. This file should be either a properties file named
translations_<locale>.propertiesor a zip file containing several of these properties files.In theLanguage to Importtable, select the language translations you want to import.
Select theEncodingfor the import. Note that this encoding should correspond to the encoding of the properties files you import.
SelectCleanup Removed Keys During Importif you want to remove the translations for the keys that are no longer used in the model. This cleanup removes translations no longer used by the model.
ClickFinishto run the import.
The translations for the model edition in the selected languages are updated with those contained in the translation bundles. If theCleanup Removed Keys During Importwas selected, translations in these languages no longer used in the model are removed.
Translation and Model Edition Lifecycles
The lifecycle of the translations is entirely decoupled from the model edition and deployment lifecycle:
It is possible to modify the translations of open or closed model editions, including deployed model editions in production data locations.
Translation changes on deployed model editions are taken into account dynamically when a user accesses an application defined in this model edition.
| Decoupling the translation lifecycle from the model edition avoids binding the critical model development and release process to the translation process, as the latter frequently is managed by a separate team. This also allows adding new translations or fixing translations without having to re-deploy a new model edition. |
| When creating a new model edition, the translations from the previous model edition are not copied to the next edition. It is necessary to export and import translations between editions. |
DEPLOYMENT
This process is the deployment in a run-time environment (for production or development) of a model designed with its certification process.
Introduction to Deployment
Deployment consists in deploying aModel Editionin aData Location(a database schema accessed via a JDBC datasource).
Once this model edition is deployed, it is possible to load, access and manage data in the data location using the applications defined in the model.
In this process, the following components are involved:
AData Locationis a database schema into which successiveModel Editionswill be deployed. This data location is declared in Semarchy xDM, and uses a JDBC datasource defined in the application server.
In a data location, aDeployed Model Editionis a model version deployed at a given time in a data location. As an MDM Hub model evolves over time, for example to include new entities or functional areas, new model editions are created then deployed. TheDeployment Historytracks the successive model editions deployed in the data location.
In the deployment process, you can maintain as many data locations as you want in Semarchy xDM, but a data location is always attached to one repository. You can deploy successive model editions into a data location, but only the latest model edition deployed is active in the data location.
Data Location Types
There are two types of data locations. The type is selected when the data location is created and cannot be changed afterwards:
The data location types are:
Development Data Location: A data location of this type supports deployingopen or closed model editions. This type of data location is suitable for testing models in development and quality assurance environments.
Production Data Location: A data location of this type supports deployingonly closed model editions. This type of data location is suitable for deploying MDM hubs in production environments.
| Be cautious when choosing the data location type, as it will determine the type of deployment operations that can be done. It is recommended to use only Production Data locations for Production and User Acceptance Test environments. |
Data Location Contents
A Data Location contains the hub data, stored in the schema accessed using the data location’s datasource. This schema contains database tables and other objects generated from the model edition.
The data location also refers three type of jobs (stored in the repository):
Installation Jobs: The jobs for creating or modifying, in a non-destructive way, the data structures in the schema.
Integration Jobs: The jobs for certifying data in these data structures, according to the model job definitions.
Purge Jobs: The jobs for purging the logs and data history according to the retention policies.
Creating a Data Location
| A data location is a connection to a database schema via a JDBC datasource defined in the application server running Semarchy xDM. Make sure the administrator of this application server creates this datasource, and the database administrator creates the schema for you before creating the data location in Semarchy Workbench. |
To create a new data location:
In the menu, selectFile > New > New Data Location. TheCreate New Data Locationwizard opens.
In theCreate New Data Locationwizard, check theAuto Filloption and then enter the following values:
Name: Internal name of the object.
Label: User-friendly label for this object. Note that as theAuto Fillbox is checked, theLabelis automatically filled in. Modifying this label is optional.
JNDI Datasource Name: Select the JDBC datasource pointing to the schema that will host the data location.
In theDescriptionfield, optionally enter a description for the Data Location.
Select theLocation Typefor this data location.
Select theDeployed Model Edition: This model edition is the first one deployed in the data location.
ClickFinishto close the wizard. The data location is created and the first model edition deploys. TheData Locationopens on the new data location.
To delete a data location:
In theData Locationsview, right-click the data location node and selectDelete. TheDelete Data Locationwizard opens. In this wizard, you only delete the data location definition in Semarchy xDM. Deleting the data stored in the data location is optional.
If you do not want to delete all the data in the data location schema, clickFinish. The data location is deleted but the data is preserved.
If you want to delete all the data in the data location schema:
Select theDrop the content of the database schemato delete the content of the schema. Note that with this option, you choose to delete all the data stored in the hub, which cannot be undone.
ClickNext. The wizard lists the objects that will be dropped from the schema.
In the next wizard step, enter
DROPto confirm the choice, and then clickFinish. The data location as well as the schema content are deleted.
| Deleting a data location is an operation that cannot be undone. |
| Deleting the data location as well as the schema content is a convenient mechanism to reset the hub content at the early stages of the model design. |
Deploying a Model Edition
After the initial model edition is deployed, it is possible to deploy other model editions. This single mechanism is used for example to:
Update the deployed model edition with the latest changes performed in an open model edition.
Deploy a new closed model version to a production or test environment.
Revert a deployed model edition to a previous edition.
| Deployingopenmodel editions is only possible in a Development Data Location. |
To deploy a model edition:
In theData Locationsview, right-click the data location node and selectDeploy Model Edition….
If you have unsaved editors, select those to save when prompted.
In the TheDeploy Model Editionwizard*, select in theDeployed Model Editionthe model edition you want to deploy.
Leave theGenerate Job Definitionoption checked to generate new integration jobs.
ClickNext. The changes to perform on the data location, to support this new model edition, are computed. A second page shows the SQL script to run on the schema to deploy this model edition.
ClickFinishto run the script and close the wizard.
The model edition deploys the jobs first and then runs the SQL code to create or modify the database objects. You can follow this second operation in theConsoleview at the bottom of the Workbench.
The new model edition is deployed and the previous model deployment appears under theDeployment Historynode in the data location.
| Deploying a model edition does not modify data already in place in the data location. |
| Although it is recommended to update both the jobs and schemas at the same time, you may want to update the data structure first then the jobs later. For example, if the data you have in the data editions using this model edition is not fit for the new version of the jobs. In that case, you may want to run some transformation on the data with the updated data structures before updating the jobs. Another use case for not deploying the job definition is when you know that the new and old job definitions are similar and you want to preserve the existing job logs. |
After multiple deployments, you may decide to remove old elements in theDeployment History.
To remove historized deployments:
In theData Locationsview, expand theDeployment Historynode under the data location.
Select one or many historized deployments (hold Shift for multiple selection).
Right-click and selectDelete.
The selected historized deployments are deleted.
Advanced Deployment Techniques
Moving Models at Design-Time
At design-time, it is possible to move models from one repository to another design repository using Export/Import:
Export is allowed from a design repository, but also from a deployment repository.
Import is possible in a design repository, either:
to create a new model from the import
or to overwrite an existingopenmodel edition.
Exporting a Model Edition
To export a model edition:
Open theModel Editionsperspective.
Expand the model branch containing the model edition you want to export.
Select the model edition you want to export, right click and selectExport Model Edition.
In theModel Edition Exportdialog, select anEncodingfor the export file.
ClickOKto download the export file on your local file system.
ClickClose.
Importing to a New Model
To import and create a new model:
In the menu, selectFile > New > New Model from import... TheImport to a New Modelwizard opens.
Click theOpenbutton and select the export file.
ClickFinishto perform the import.
Click theRefreshbutton in theModel Editionsview. The new model appears in the list.
| When importing a model from a different repository, make sure that the correctTarget Technologyis set for this model. For example, a model developed for the Oracle target technology will not work if you attempt to deploy it on a PostgreSQL data location. |
Importing on an Existing Model
To import and replace an existing model:
Open theModel Editionsperspective.
Expand the model branch containing the model edition you want to replace.
Select theopenmodel edition you want to replace, right click and selectImport Replace…. TheImport-Replace Model Editionwizard opens.
Click theOpenbutton and select the export file.
ClickFinishto perform the import.
ClickOKto confirm the deletion of the existing model.
The existing model edition is replaced by the content of the export file.
Deploying to a Remote Location
Frequently, the deployment environment is separated from the development environment. For example, the development/QA and production sites are located on different networks or locations. In such cases, it is necessary to use export and import to transfer the model edition before performing the deployment in production.
A remote deployment consists in moving aclosedmodel edition:
From a design repository or a deployment repository used for Testing/UAT purposes;
To a deployment repository.
Remote Deployment Architecture
In this configuration, two repositories are created instead of one:
ADesignrepository for the development and QA site, withDevelopmentdata locations attached to this repository.
ADeploymentrepository for the production site.Productiondata locations are attached to this repository.
The process for deploying a model edition in this configuration is given below:
The model edition is closed in the design repository.
The closed model edition is exported from the design repository to an export file.
The closed model edition is imported from the export file into the deployment repository.
The closed model edition is deployed from the deployment repository into a production data location.
Exporting a Model Edition
To export a model edition:
Open theModel Editionsperspective.
Expand the model branch containing the model edition you want to export.
Select theclosedmodel edition you want to export, right click and selectExport Model Edition.
In theModel Edition Exportdialog, select anEncodingfor the export file.
ClickOKto download the export file on your local file system.
ClickClose.
Importing a Model Edition in a Deployment Repository
To import a model edition in a deployment repository:
Open theModel Editionsperspective.
SelectFile > New > Import Model Edition. TheImport Model Editionswizard opens.
Click theOpenbutton and select the export file.
ClickNext.
Review the content of the Import Preview page and then clickFinish.
| When importing a model edition, the root model and the branches containing this model edition are automatically created as needed. |
| When importing successive versions of model editions in a deployment repository, it is not recommended to skip intermediate versions, as it is not possible to import these intermediate versions later. For example, if importing version 0.1 of a model, then importing version 0.4, the intermediate versions - 0.2, 0.3 - can no longer be imported into this repository. |
Elements not Exported with the Model Edition
When exporting then importing a model to a different repository, note that some elements are not included in the model and need to be reconfigured in the remote environment. These elements are listed below.
Elements used by the model and applications:
Applications Common Configuration
Image Libraries
Installed Plug-ins
Role definitions
Variable Value Providers
Elements not used by the model, but required for operations:
Notification Servers
Notification Policies
Continuous Loads
Purge Schedules
Integration Batch Poller Configuration
Data Location Status
A data location status is:
Ready: A data location in this status can be accessed in read/write mode, accepts incoming batches and processes its current batches.
Maintenance: A data location in this status cannot be accessed. It does not accept incoming batches but completes its current batches. New loads cannot be initialized and existing loads cannot be submitted.
When moving a data location to a Maintenance status, the currently processed batches continue until completion. Loads submitted after the data location is moved to Maintenance will fail. They can be kept open and submitted later, when the data location is restored to a ready status.
| When a data location is in maintenance mode, it appears with a disabled avatar in the welcome page. |
Changing a Data Location Status
To set a data location status to maintenance:
In theData Locationsview, expand the data location node.
Right-click the data location and selectSet Status to Maintenance.
ClickOKin the confirm dialog.
The data location is moved to Maintenance mode.
To set a data location status to ready:
In theData Locationsview, expand the data location node.
Right-click the data location and selectSet Status to Ready.
The data location is moved to a ready state.
Using the Maintenance Mode
The Maintenance status can be used to perform maintenance tasks on the data locations.
For example, if you want to move the data location to a model edition with data structure changes that mandate manual DML commands to be issued on the hub tables, you may perform the following sequence:
Move the data location to Maintenance mode.
Let the currently running batches complete. No batch can be submitted to this edition.
Deploy the new model edition.
Perform your DML commands.
Move the data location from Maintenance to the Ready status. Batches can now be submitted to this data location.
Using this sequence, you prevent batches being submitted while the hub is in Maintenance.
The data location is automatically set to maintenance when deploying a model edition and then automatically reverted to ready state.
SECURING DATA
Securing Data Access
This section describes how to define the access policies for the data model. They are implemented in the form of model and entity privileges.
They are enforced when the user accesses the data through the graphical user interface and integration points.
Introduction to Security in Semarchy xDM
There are two levels of security in Semarchy xDM:
Platform-Level Securitydefines access to features in the platform. This type of security is covered in theSemarchy xDM Administration Guide.
Model Level Securitydefines security privileges to access and modify data in the data location into which the model is deployed.
Both levels of security are role-based:
Privileges are granted toRolesdefined in Semarchy xDM. These roles map to roles defined at the application server level.
Roles are given toUsersdefined in the application server.
Semarchy xDM does not store users and roles associations. These are provided by the application server which delegates users and roles management to a security provider (SSO, LDAP, etc.)
For more information about role creation and management, refer to theSemarchy xDM Administration Guide.
Model Privileges Grants
Model Privileges Grantsdefine the access privileges for users via their roles.
A Model Privilege Grant is associated to a role and contains a set of Entity Privileges granted to this role. You can only define a single Privilege Grant for each role in a model.
Entity Privileges
AnEntity Privilegedefines the privilege of a role for an entity (or a subset of the entity records) and its attributes.
Entity-Level and Attribute-Level Privileges
An Entity Privileges includes privileges defined at:
Entity-level: Such a privilege is the default privilege for all the attributes of the entity.
Attribute-level: Such a privilege overrides the default entity-level privilege, allowing specific privileges per attribute.
Examples:
Entity-Level Privileges: Users with theFinancerole can edit (read/write) cost centers; other users can only see them (read). Users with theContractorrole cannot see (none) this entity.
Attribute-Level Privileges: All users can see (read) general information from the employee entity, but they cannot see (none) sensitive information (Personal Email, SSID, Salary, etc. attributes). Only users with theHRrole can see these attributes.
Privileges Types
The types of privilege are listed in the table below:
| Privilege Type | Entity-Level | Attribute-Level | Description |
|---|---|---|---|
None | Yes | Yes | No privilege is granted to the user for the entity or attribute. |
Read | Yes | Yes | Allows the user to view the entity or attribute. |
Read/Write | Yes | Yes | Allows the user to view the entity or attribute. In addition, the user can edit the values in data authoring. Note that this privilege only allows modifying data. To create new records in a stepper, theCreationprivilege is needed. |
Allow Export | Yes | N/A | Allows the user to mass-export records from this entity in Excel format. Note that exported columns are filtered using the privileges granted on attributes. If a user can Export an entity, but does not have read or read/write privileges on certain attributes, these attributes will not appear in the export. |
Allow Creation | Yes | N/A | Allows the user to create new records for this entity in steppers. |
Allow Checkout | Yes | N/A | Allows the user to edit records for this entity in steppers. |
Allow Removal | Yes | N/A | Allows the user to remove records for this entity in steppers. |
Allow Delete | Yes | N/A | Allows the user to remove records for this entity. This entity must also haveDelete Enabledselected. |
| If a user has read or read/write privileges on one attribute of an entity, then he also has read privileges to see the built-in attributes managed by the platform (for example:publisher,update date, etc.). |
Privilege Precedence
Privileges apply in order of precedence:Read/WritethenReadthenNone. As a consequence, a user always has the best privileges associated to his roles
For example: John has several roles given to him:
FinancehasReadprivileges on theCustomerentity.
HRhasNoneprivileges on theCustomerentity.
SaleshasRead/WriteandCreationprivileges on theCustomerentity.
The resulting privilege for John isRead/WriteandCreationon theCustomerentity.
| Make sure you take into account all the roles of a user when reviewing his privileges. Any given role may grant this user additional privileges. |
Row-Level Filtering
Entity privileges supportRow-Level Filtering, to apply privileges only to a subset of the records of the entity. The subsets are defined using SemQL filters.
To define different privileges for different subsets of records in the same entity, it is possible to create several entity privileges for the same entity. Each privilege will address a subset of the records, defined by aFilter.
Row-level security filters can useModel Variablesto customize the data access privileges for the connected user’s session.
For example, the Privilege Grant for theGenericUserrole contains:
A first Entity Privilege allowingReadaccess to theemployeeentity.
A second privilege grant for the sameemployeeentity, allowingRead/Writefor certain attributes. This second privilege grant contains the following SemQL filter:
employeeName=:V_USERNAME.
Both these grants apply to the sameemployeeentity and the sameGenericUserrole. The second one only applies to the filtered subset of records in this entity passing this filter. This subset corresponds to the connected user’s employee record.
| Row-Level Filtering is not supported for duplicate management. A user who performs duplicate management operations for an entity must be granted privileges on all the records of the entity. Read privilege is required on all records of a duplicate manager, and write privilege is required to manipulate these duplicates. |
Built-in Roles
The following roles are built in within the platform:
semarchyConnect: This role must be granted for a user to log in. It should be granted by default to all users connecting to Semarchy xDM. This role is hard-coded and cannot be modified or used in the model.
semarchyAdmin: This role has full access to all the features of the platform. This role is hard-coded and cannot be modified. It can be used in the model, for example in privilege grants.
| When creating a new model, a model-level privilege grant is automatically created for thesemarchyAdminrole withGrant full access to the modelselected. By modifying this privilege grant, the model designer can reduce the privileges of thesemarchyAdminrole on the data, and prevent platform administrators from accessing data in the hub. |
How are Privileges Applied?
The Semarchy Workbench Data Management user interface layout and actions are automatically modified depending on the user privileges:
Entities/Attributes/Records with no read privilege do not appear.
Entities/Attributes/Records with no read/write privilege cannot be modified in authoring.
Records cannot be added in steppers without the creation privileges.
Records cannot be removed from steppers without the removal privileges.
Exporting Data (Excel) is not possible without the export privilege.
Setting up Model Privileges
| In this section, we assume that roles and users are already defined at the application server level, and that Roles are already defined in Semarchy xDM by the administrator. For more information about role creation and management, refer to theSemarchy xDM Administration Guide. |
To add a model privilege grant:
Connect to an open model edition.
In theModel Editionview, right-click theModel Privilege Grantsnode and selectAdd Model Privilege Grant…. TheCreate New Model Privilege Grantwizard opens.
In theCreate New Model Privilege Grantwizard, check theAuto Filloption and then enter the following values:
Role Name: Select a role defined in Semarchy xDM by the administrator.
Name: Internal name of the object.
Label: User-friendly label for this object. Note that as theAuto Fillbox is checked, theLabelis automatically filled in. Modifying this label is optional.
In theDescriptionfield, optionally enter a description for the Model Privilege Grant.
Check theGrant full access to the modeloption if you want to give full privileges to this model. Checking this box overrides all privileges granted at the entity or attribute level.
Check theGrant access to integration web servicesoption to allow this role to connect to the REST API interface for this model. Note that entity/attribute level privileges are also needed. This option only allowsconnectingto the web service interface.
ClickNext.
In theEntities Privileges Grantspage, select theEntitiesfor which you want grant privileges and click theAdd >>button to add them to theSelected Entities.
ClickNext.
In the next page, select the default privileges for the selected entities:
Default Access Type: SelectNone,ReadorRead/Write
Allow Creation: Select this option to allow this role to create new entity records in a stepper.
Allow Checkout: Select this option to allow this role to edit entity records into a stepper.
Allow Export: Select this option to allow this role to export entity records.
Allow Removal: Select this option to allow this role to remove entity records from a stepper.
Allow Delete: Select this option to allow this role to delete entity records.
ClickFinishto close the wizard. TheModel Privilege Granteditor opens.
PressCTRL+Sto save the editor.
In the Model Privilege Grant editor, you can refine the default privileges in theEntity Privilegestable:
You can modify theAccess Typefor each entity.
You can expand the entities and modify theAccess Typefor specific attributes.
You can change theCreation,Checkout,Removal,DeleteandExportprivileges on entities.
You can add privilege grants for more entities, or new privilege grants for entities already present in the grant.
You can set afilterfor each privilege grant to enable row-level security.
| Be cautious when checking theGrant full access to the modeloption in a privilege grant, as it overrides all privileges granted at the entity or attribute level. |
Reviewing the Privileges
It is recommended to review the privileges of the users before releasing a model.
To test the security for a given user:
Login to the Semarchy xDM Workbench using this user’s credentials.
Connect to the Data Location using an application.
In the workbench toolbar, select the user name in the upper right corner and then selectUser Information.
The platform, entity-level and attribute-level privileges for the connected user, computed from his various roles, are listed in the user information window.
Defining Data Retention
This section describes how to define the retention policies for historical and lineage data.
Introduction to Data Retention
The MDM hub stores the lineage and history of the certified golden data, that is the data that led to the current state of the golden data:
The built-inLineagetraces the whole certification process from the golden data back to the sources. It traces source data changes pushed in the hub - through external loads - or performed into the hub - for example using steppers.
Historizationtraces the changes made to the golden and master data.
Preserving the lineage and history is a master data governance requirement. It is key in a regulatory compliance focus. However, keeping this information may also create a large volume of data in the hub storage.
To keep a reasonable volume of information, administrators will schedule purges for this data. To make sure lineage and history are preserved according to the data governance and compliance requirements, model designers will want to defineData Retention Policyfor the model.
Data Retention Policies
Data Retention Policies are defined with:
AModel Retention Policy, defining the retention duration for history and lineage data in the hub. This policy applies by default to all entities with no specific policy.
Entity Retention Policiesthat override the model retention policies for specific entities.
For example:
The hub is configured to retain no history at all. This is the general policy.
Employee data is retained for 10 years.
Product data is retained forever.
Data Purge
Depending on the retention policy defined for the model, data purge takes place in the deployed hub.
The purge deletes the following elements of the lineage and history:
Source data published to the hub via external loads
Data authored (created, modified or overridden) in the hub
Traces of deleted data
Golden and master data history (if historization is configured)
Errors detected on the source and authored data by the integration job
Duplicate choices made by users in duplicate managers.
Note that the duplicate management decisions still apply after a purge, but information about the time of the decision and the decision maker is deleted.
| The purges only impact thehistory and lineageof the data the data location. They do not delete actual golden and master data. |
Optionally, the following repository artifacts can also be deleted as part of the purge process:
Job logs, batches and loads for which all the processed data has been purged.
Direct authoring, duplicate manager and workflow instances for which all the changed data has been purged.
| Job logs, batches, loads, direct authoring, duplicate manager and workflow instances are purged whenalltheir data have been purged. Therefore they are purged based on thelongest retention policyof all the entities that they manage. |
Deploying a Purge
When a model is deployed, a purge job is automatically created in the deployment data location. This job purges data and artifacts according to the retention policy defined in the deployed model.
Purge jobs are scheduled by the hub administrator as part of the data location configuration. For more information, refer to theSemarchy xDM Administration Guide.
| Regardless of the frequency of the purges scheduled by the administrator, the data history is retained is as defined by the model designer in the data retention policies. |
Defining the Data Retention Policies
The model retention policy applies to all the entities with no specific retention policy.
To define the model data retention policy:
Connect to an open model edition.
In theModel Editionview, double-click theRetention Policiesnode. TheData Retention Policyeditor opens.
In theData Retention Policyeditor, set theDefault Retention Policyfor the model:
Retention Period: Select the default retention type:Forever,No RetentionorPeriod.
If you have selectedPeriod, set aTime UnitandTime Durationfor the retention period.
In theDescriptionfield, optionally enter a description for the retention policy.
PressCTRL+Sto save the editor.
In addition to the model retention policy, you can also define in this editor the entity-specific retention policies.
To define an entity retention policy:
In theData Retention Policyeditor, click theAdd Entity Retention Policy. TheCreate New Entity Retention Policywizard opens.
In theCreate New Entity Retention Policywizard:
Select in theEntityfield the entity for which you want to define a specific policy.
Retention Period: Select the retention type:Forever,No RetentionorPeriod.
If you have selectedPeriod, set aTime UnitandTime Durationfor the retention period.
In theDescriptionfield, optionally enter a description for the entity retention policy.
ClickFinishto close the wizard.
PressCTRL+Sto save the editor.
| You can only have one entity retention policy per entity of the model. |
| The retention policy has no effect unless the model is (re)deployed and the administrator of the hub schedules the purge job for the data location. For more information, refer to theSemarchy xDM Administration Guide. |
Last updated 2018-06-14 11:52:52 UTC