All Release Notes
Related information
Version 6.2.2
New features
This release contains the following enhancements:
-
Null value management:
-
Previously, a field's null value management settings were applied anytime the field was used in a decision tree's comparison node. To provide more flexibility, comparison nodes now include settings for each field to determine the outcome when one or both of the compared values is null. This makes it possible to apply different matching behaviors for the same field in different parts of the decision tree. For example, one node can require an exact match, where both records must contain a value, while another node can use a "not mismatched" strategy, which considers records matching unless their values are explicitly different. Note that these new settings apply only to the decision tree processing phase of matching. When upgrading from a previous version, existing field configurations default to No match for the null value options.
-
A new Do nothing option was added to the Matching fields tab's options. This option ensures that null values are not considered as similarity criteria during the pre-processing clustering phase. When null values are ignored, matching performance and quality can improve as it can prevent unnecessary decision tree computations. For instance, it can prevent false positives from being passed to decision tree processing for evaluation.
-
-
The following new options were added for specifying when golden records are automatically created for business objects:
-
Only duplicates (available for a business object parent entity only): A new golden record is automatically created only when the add-on identifies a positive match between records. With this option selected, child entities must use the Any duplicates option described below.
-
Any duplicates (available for child entities only): If the parent has positive matches, a new golden is created for the parent and its children. If matches are found for children of the same parent, but not the parent, a new golden is created for the child only.
-
-
This release includes the option to show whether records were matched through a transitive match. Transitive matching refers to the inclusion of records based on indirect relationships. For example, if Record A matches Record B, and Record B matches Record C, then Record A and C are grouped together due to their transitive connection through B. Administrators can configure display of this information using the new Show match type property, located on each matching policy. When enabled, users will have a new Match type column in the Manage group screen that shows whether the match type is Direct, Transitive, or N/A for the survivor record.
-
Performance improvements: This release includes multiple backend updates and enhancements to improve the add-on's performance.
-
Improved exception handling:
-
Workflows that include matching operations are now stopped when an exception occurs in the add-on. An error message detailing the issue displays in the workflow UI.
-
When performing a grouping operation, an exception is thrown when the center record cannot be found. Previously, only a
NullPointerExceptionwas thrown. Now, an error message is logged and shown in the UI. The message includes the group Id to support location and correction of the issue.
-
-
Logging enhancements:
-
Warnings for
SurvivorServiceImpl#getFieldSurvivorsByMergeFunctioncould be repeatedly logged, leading to a performance cost and difficulty in parsing the logs. To alleviate this, the warnings are consolidated and logged after longer increments. -
A log entry is now created when pausing and resuming the matching trigger. The log includes information on the affected table.
-
Logs created when merged data includes many null fields were condensed to improve readability.
-
-
API enhancements: The simulate match API previously returned a list of
MatchingEngineResultobjects that contained primary keys for the target and pivot records. This required additional resources to look up the record's correspondingAdaptation. Now theAdaptationis returned instead of the primary key.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Note:
As the add-on UI is under development, some images in the documentation might not exactly match those in the UI.
Changes in Functionality
In addition to existing functionality, the Purge old results service removes any records that are orphaned for any of the following reasons:
-
If record was, but is no longer linked to a golden record (for example if the golden was deleted).
-
If a record was deleted in the main table, but still exists in the RecordMetadata table.
Upgrade notes
Take the following into account if you upgrade TIBCO EBX® Match and Merge Add-on:
-
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
-
If upgrading from any version prior to 6.2.0, a performance improvement was made to better balance the weights of fields that participate in matching. The automatic balancing occurs during the pre-processing phase. This helps mitigate issues that might occur when fields include many values. As a result of this update, matching operation results might be impacted. To mitigate any negative impacts, you can test and adapt your existing weights to achieve the desired matching outcome. Alternatively, you can set the
ebx.clustering.balanceproperty tofalsein theebx.propertiesfile to restore the legacy behavior.
Changes to third-party libraries
This release contains no updates to third-party libraries.
Closed Issues
This release contains the following closed issues:
Known Issues
This release contains the following known issues:
-
The following workaround is for a known issue in EBX® (CP-27860). When child dataspaces or snapshots are created before enabling the EBX Match and Merge Add-on on a data model, some operations can fail after the add-on is enabled. For instance, child dataspace merge might not work correctly, or snapshots might have errors.Workaround: Disable the add-on before performing the dataspace merge, or creating a snapshot; enable the add-on after completing the operation:
-
Open the data model, navigate to Configuration > Add-ons.
-
To disable the add-on, tic the box next to Match and Merge and select Delete from the Actions menu.
-
Publish the data model.
-
After performing the desired operations, enable the add-on by re-creating the Match and Merge record in the Add-ons table.
-
Publish the data model.
-
-
Foreign key alignment of recursive foreign keys is not possible.
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 6.2.1
New features
This release contains the following enhancements:
-
The Merge screen now supports display of custom views for tables and Business objects. This feature had previously been implemented for only the Manage group screen, but is now available for both screens.
-
The first time users access the Manage group and Merge screens, they now display as they are configured in EBX®. If multiple views are configured for a user, they are prioritized as follows: favorite view, recommended view, then default view. Any changes users make to the view, or record orientation (vertical or horizontal) are maintained and displayed the next time users log in.
-
Administrators can now specify a default view mode (horizontal or vertical) when configuring matching tables.
-
The Manage group view now includes filter options. You can enable pre-defined filters that hide any merged records from the view, or hide all fields except those configured to participate in matching operations.
-
The display of the add-on's services now reflects the permissions set on the dataset. If a service is hidden for a user on a dataset, it no longer displays as an option in the add-on's screens.
-
Updates were made to improve performance when the add-on creates or modifies records.
-
It is now possible to configure computed value fields to participate in matching operations.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Note:
As the add-on UI is under development, some images in the documentation might not exactly match those in the UI.
Changes in Functionality
This release contains no changes in functionality.
Upgrade notes
Take into account the following when upgrading from TIBCO EBX® Match and Merge Add-on version 3.0.0:
-
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
-
If upgrading from any version prior to 6.2.0, a performance improvement was made to better balance the weights of fields that participate in matching. The automatic balancing occurs during the pre-processing phase. This helps mitigate issues that might occur when fields include many values. As a result of this update, matching operation results might be impacted. To mitigate any negative impacts, you can test and adapt your existing weights to achieve the desired matching outcome. Alternatively, you can set the
ebx.clustering.balanceproperty tofalsein theebx.propertiesfile to restore the legacy behavior.
Changes to third-party libraries
The Spring framework was updated to version 6.1.15.
Closed Issues
This release contains the following closed issues:
Known Issues
This release contains the following known issues:
-
Foreign key alignment of recursive foreign keys is not possible.
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 6.2.0
New features
This release contains the following enhancements:
-
The Ensure group management parameter was added to the Manage group workflow service. When this option is enabled, the workflow's Accept option only displays if the current group is resolved and has no suspect records.
-
The Back to the main view button was re-labeled to Review the merged group to more closely describe the button's behavior.
-
The Full text algorithm was updated to improve the consistency of matching operation results.
-
When merging records or managing groups, a new vertical display option is available. This option displays the records in columns instead of rows. Administrators can set the default view with the new
ebx.addon.mame.manageGroup.record.displayproperty in theebx.propertiesfile. Use the valuesverticallyorhorizontallyto define the view orientation. Please note that the vertical display option is not available when merging business objects. -
The UI for viewing pre-processing information was improved as follows:
-
Now, it only displays a field configured for matching one time, even if it is used in multiple decision tree nodes.
-
A value column was added to display the values being compared.
-
The fields are grouped by their tables.
-
-
The UI for the decision tree result explanation was updated to improve readability.
-
It is now possible to transfer configuration information for data models located inside a registered module via the EBX® staging feature.
-
The foreign key alignment phase was added to the progress bar when running a matching operation.
-
A warning is now displayed if an algorithm that is incompatible with the defined search strategy is selected in the decision tree. This warning no longer displays when evaluating matching results.This release contains the following performance, technical, and API enhancements:
-
The pre-processing phase now takes into consideration the decision tree content to reduce the cluster size, which can lead to improved performance.
-
During the pre-processing phase the balancing of matching field scores was improved so that each field equally affects the final similarity score.
-
When simulating a match operation using the API, a procedure is no longer opened every time. This prevents a lock of the dataspace and any negative effects on
TableTrigger. -
The separate procedures for matching, grouping, and merging were merged into a single procedure.
-
Performance was improved when running a match operation with a selection of records.
-
The REST API to accept or reject suspect records is now available.
-
It is now possible to create a primary key generator to ensure that specific values from a composite primary key are survived into a golden record.
Note:
As the add-on UI is under development, some images in the documentation might not exactly match those in the UI.
Changes in Functionality
This release contains no changes in functionality.
Upgrade notes
Take into account the following when upgrading from TIBCO EBX® Match and Merge Add-on version 3.0.0:
-
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
-
If upgrading from any version prior to 6.2.0, a performance improvement was made to better balance the weights of fields that participate in matching. The automatic balancing occurs during the pre-processing phase. This helps mitigate issues that might occur when fields include many values. As a result of this update, matching operation results might be impacted. To mitigate any negative impacts, you can test and adapt your existing weights to achieve the desired matching outcome. Alternatively, you can set the
ebx.clustering.balanceproperty tofalsein theebx.propertiesfile to restore the legacy behavior.
Changes to third-party libraries
This release contains no changes to third-party libraries.
Closed Issues
This release contains the following closed issues:
Known Issues
This release contains the following known issues:
-
Foreign key alignment of recursive foreign keys is not possible.
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 6.1.3
New features
This release contains the following updates focused on performance improvements:
-
Foreign Key alignment during the merging phase was optimized.
-
Instead of relying on snapshots, a new method is used to check the differences between transactions.
-
The API was updated to improve performance when getting lists of suspects or records contained in a group.This release contains the following enhancements:
-
New rules are applied to survivor record selection for the instances where no records, or multiple records satisfy required conditions.
-
When running the Purge old results service, out-of-date matching selections are now also purged.
-
Foreign key alignment of composite primary keys during automatic merge was improved.
-
The relationship between selected records is now removed for any suspect record that is accepted as a match when managing groups.
-
The readability of information written to the logs regarding foreign key alignment was improved.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains the following changes to functionality and behavior.
-
If a field configured to participate in matching meets either of the following criteria, it is automatically disabled during the pre-processing phase to improve performance:
-
The field's type is
boolean. -
The entire column is
null.
-
-
Special characters are now preserved when using the add-on's distance matching algorithms.
Changes to third-party libraries
The Spring framework was updated to version 5.3.29.
Closed Issues
This release contains the following closed issues:
Known Issues
This release contains the following known issues:
-
Foreign key alignment of recursive foreign keys is not possible.
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 6.1.2
New features
This release contains the following new features:
-
It is now possible to use the replication feature on Business objects.
-
Data comparison nodes now have the option to use operators or algorithms to determine how to values are compared.
-
The following user experience enhancements were made:
-
The display order of fields in merge policies was updated so that Auto create new golden is now at the top. This is more intuitive as this value can impact the availability of other options.
-
The explanation for the evaluation of matching execution now includes additional parameters to better explain the results.
-
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains no changes to functionality and behavior.
Changes to third-party libraries
The FasterXML jackson-databind library was updated to version 2.15.2.
Closed Issues
This release contains the following closed issues:
Known Issues
This release contains the following known issues:
-
Foreign key alignment of recursive foreign keys is not possible.
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 6.1.1
New features
This release contains the following new features:
-
The add-on was adapted to ensure compatibility with the EBX® staging feature.
-
A page loading indicator now displays when performing a manual merge.
-
When configuring inline matching for a workflow, it is now possible to use the Matching policy parameter to specify a matching policy other than the default policy configured for the table. The default policy is used when this parameter is undefined.
-
All snapshots related to the add-on now include a
MAME_prefix.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains the following changes to functionality and behavior:
-
The Search before create service no longer needs to be enabled by an administrator. Additionally, when running the service, the matching policy is selectable. The chosen matching policy will determine which fields are available to search and the criteria used to locate potential matches. The Search before create service is still available from the matching policy configuration, however it is marked as deprecated.
-
When a matching table configuration is deleted, now all related replications are also removed.
-
The following default options were changed in comparison node configurations:
-
The default value for Comparison function is now All fields match.
-
For the Full text, Fuzzy, and Hybrid fuzzy algorithms, the Minimum score now defaults to a value of 100.
-
-
The Consolidated view was deprecated. It is still available to access, but it is planned to be removed in a future release.
Changes to third-party libraries
The Spring framework was updated to version 5.2.24.
Closed Issues
This release contains the following closed issues:
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 6.1.0
Released: March 2023
New features
This release contains the following new features:
-
The add-on was adapted to ensure compatibility with the way EBX® now handles URL encoding. See the EBX® release notes for additional information.
-
Group management:
-
The Manage group screen user interface and user experience were updated and improved.
-
Any existing custom table views can now be applied in the Manage group screen.
-
All group management functions are now available when accessing this feature through a workflow. Additionally, the Close button is now available on this user service in workflows.
-
-
Manual merge:
-
The user interface and user experience were updated and improved in this release for the Merge view. The view includes a new merge progress tracker that includes sub-steps when merging a business object's child tables.
-
A new Undo button was added to the Merge view. This button allows you to revert the most recent values selected for a golden record. The reverted value is highlighted in the UI.
-
The merge Summary screen was updated to display successfully merged records along with the icons to access the record's detailed view.
-
-
Hybrid fuzzy algorithm:
-
When using the hybrid fuzzy algorithm in a decision tree node, you can now customize the values used by the algorithm for: Synonyms, Levenshtein 1, and Levenshtein 2.
-
The Levenshtein transformation logic was updated to ensure that short strings return more accurate results.
-
-
The No fields match option was removed from decision tree comparison nodes. If you used this option in one of your add-on configurations, navigate to: Administration > TIBCO EBX® Match and Merge Add-on > Table activation and settings. The Status column will alert you to any configurations where you must select a different comparison function.
-
The option to automatically trigger matching operations is now available for business objects. Only the After submission option is available to use with business objects, either On creation or On update.
-
Evaluation of matching:
-
When evaluating matching results and viewing results for a data comparison node, the displayed information now includes the table that each compared field belongs to.
-
The similarity results were updated to include additional information of why records do not complete the pre-processing stage.
-
-
New Java APIs were introduced to:
-
Evaluate matching results.
-
Retrieve the trusted sources list.
-
Perform multiple merges in one transaction.
-
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
The behavior of the Modify merged records option was reverted. You can once again choose what the add-on does when users modify merged records. An additional option was also included that is a combination of Keep records in group and Change to <unset>.
Changes to third-party libraries
The SnakeYAML library was updated to version 2.0.
Closed Issues
This release contains the following closed issues:
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 3.6.1
Released: February 2023
New features
This release was updated to ensure compatibility with TIBCO EBX® Add-ons Bundle version 5.6.1.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains no changes in functionality.
Changes to third-party libraries
This release contains no changes to third-party libraries.
Closed Issues
This release includes no closed issues.
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 3.6.0
Released: November 2022
New features
This release contains the following new features:
-
The UI was updated for manual merge operations on the tables that comprise business objects.
-
The stewardship UI was updated so that lineage and metadata can be viewed for a business object's related tables.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains no changes in functionality.
Changes to third-party libraries
The FasterXML jackson-databind library was updated to version 2.13.4.2.
Closed Issues
This release includes the following closed issues:
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 3.5.0
Released: September 2022
New features
This release contains the following new features:
-
Matching and merge operations can be configured on multiple tables that comprise EBX® business objects.
-
The Manage group view is able to display records from multiple tables in a business object.
-
To improve the user experience, the progress bar was updated.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains no changes in functionality.
Changes to third-party libraries
The Apache Taglibs library was removed from this release.
Closed Issues
This release includes the following closed issues:
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 3.4.1
Released: July2022
New features
This release contains no new features.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains no changes in functionality.
Changes to third-party libraries
This release contains no changes to third-party libraries.
Closed Issues
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 3.4.0
Released: June 2022
New features
This release contains the following new features:
-
The following algorithms were added:
-
Phonetic full text: An algorithm best used for strings. It can recognize the phonetic equivalent of two words spelled differently using the Beider-morse phonetic tokens. The comparator takes into account the synonyms and stop words defined in the data model.
-
Hybrid fuzzy: An algorithm best used for string and text data types. It can capture complex relationships between two field values by automatically customizing transformations for a given criterion. Points of note about this algorithm:
-
Does not return a 100% matching score on synonyms.
-
Strings that are similar, but contain characters like spaces or dashes do not return a 0% matching score.
-
When strings fall within the acceptable distance score they are not considered as the exact same. For example, a distance of 1 returns .95, a distance of 2 returns .9, etc.
-
-
-
A script task was added to run a matching operation from a workflow.
-
Fields can now have a weight of 0. Setting the weight to 0 excludes the fields from the pre-processing phase.
-
An API was added to check whether the add-on has any ongoing background operations.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
The options to specify what happens after a merged record is modified were removed. Now, the following behavior is applied when a record is in the merged state and the modified field:
-
Is not configured to use in matching operations: The add-on executes an automatic merge using the settings defined in the default matching policy's merge policy.
-
Is configured to use in matching operations: The record is changed to the <unset> state.
Changes to third-party libraries
This release contains no changes to third-party libraries.
Closed Issues
This release contains the following closed issues:
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 3.3.5
Released: June 2022
New features
This release contains no new features.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains no changes in functionality.
Changes to third-party libraries
This release contains no changes to third-party libraries.
Closed Issues
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 3.3.4
Released: May 2022
New features
This release contains no new features.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains no changes in functionality.
Changes to third-party libraries
The Spring framework was updated to version 5.2.22.
Closed Issues
This release contains no closed issues.
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 3.3.3
Released: April 2022
New features
This release contains no new features.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains no changes in functionality.
Changes to third-party libraries
This release contains no changes to third-party libraries.
Closed Issues
This release contains the following closed issues:
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 3.3.2
Released: April 2022
New features
This release contains no new features.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains no changes in functionality.
Changes to third-party libraries
The Spring framework was updated to version 5.2.20.
Closed Issues
This release contains no closed issues.
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 3.3.1
Released: March 2022
New features
Performance was improved to retrieve groups containing suspect records.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains no changes in functionality.
Changes to third-party libraries
This release contains the following changes to third-party libraries:
-
The Spring framework was updated to version 5.2.19.
-
The FasterXML/jackson-databind library was updated to version 2.13.2.1.
Closed Issues
This release contains no closed issues.
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 3.3.0
Released: February 2022
New features
This release includes the following new features:
-
You can now replicate matching metadata. This allows you to use third-party tools to access and query the metadata.
-
The following algorithms are now available:
-
Fuzzy full text: This algorithm works best for strings and can find the similar and fuzzy matches of the words in the compared values. It is based on the Levenshtein distance algorithm. The comparator takes into account the synonyms and stop words defined in the data model.
-
Range: This algorithm matches values within a predefined range. The two values are considered a match if the distance between two values is within the specified range.
-
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains the following changes in functionality:
-
When Auto create new golden is enabled, the requirement was removed for the matching table's primary key to be an auto-incremented integer.
-
When executing add-on operations, only the add-on's table triggers are disabled. All user custom triggers remain enabled. Note that table triggers related to metadata linked fields are not recommended as it might affect the accuracy of match and merge operations.
Changes to third-party libraries
This release contains no changes to third party libraries.
Closed Issues
When matching is set to After submission and batch creation of records occurs using the EBX® API, the number of submitted or created records is limited to 2000 records in an API call for tables that have a single primary key. For a composite primary the limit is divided by the number of fields that makes up the primary key. For example, with a table that has a primary key comprised of 2 fields, the limit would be 1000 records. When your requirements exceed these limits, it is recommended that you pause the matching trigger, load the data, and resume matching. Also, note that when matching is triggered the dataspace is locked until matching completes. Any incoming requests during this time are blocked and will return errors.
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Version 3.2.1
Released: December 2021
New features
This release includes no new features.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
The Apache Log4j library was upgraded to version 2.17.1.
Changes to third-party libraries
This release contains no changes to third party libraries.
Closed Issues
This release contains no closed issues.
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
When matching is set to After submission and batch creation of records occurs using the EBX® API, the number of submitted or created records is limited to 2000 records in an API call for tables that have a single primary key. For a composite primary the limit is divided by the number of fields that makes up the primary key. For example, with a table that has a primary key comprised of 2 fields, the limit would be 1000 records. When your requirements exceed these limits, it is recommended that you pause the matching trigger, load the data, and resume matching. Also, note that when matching is triggered the dataspace is locked until matching completes. Any incoming requests during this time are blocked and will return errors.
Version 3.2.0
Released: November 2021
New features
This release includes the following new features:
-
You can now exclude records from matching operations by specifying a field value. If a record contains the specified value, it is not included in matching operations. Using this functionality can help improve performance and improve the quality of matching results.
-
You can now modify records that were included in a merge operation.
-
The new Evaluate matching service allows you to test matching results. This can help you fine tune configuration settings to achieve desired results.
-
Adding an association to your data model allows you to view: a hierarchy of golden and merged records, and a tab when viewing golden record details that displays all merged records that target the golden.
-
Matching now supports use of non-Latin based text.
-
A new merge function on field survivorship allows you to add a constant value that is merged into golden records on survivorship.
-
The Full Text algorithm is now available to use in matching for fields with Text and String data types.
-
You can now use a foreign key field as a field in a decision tree's comparison node.
Attention:
When upgrading from version 3.0.0, you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains the following functionality changes:
-
The default mode for Align foreign keys is now set to Manually.
-
The Case sensitivity option is now only available for fields with String and Text data types.
Changes to third-party libraries
This release contains no changes to third party libraries.
Closed Issues
This release contains no closed issues.
Known Issues
This release contains the following known issues:
-
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
When matching is set to After submission and batch creation of records occurs using the EBX® API, the number of submitted or created records is limited to 2000 records in an API call for tables that have a single primary key. For a composite primary the limit is divided by the number of fields that makes up the primary key. For example, with a table that has a primary key comprised of 2 fields, the limit would be 1000 records. When your requirements exceed these limits, it is recommended that you pause the matching trigger, load the data, and resume matching. Also, note that when matching is triggered the dataspace is locked until matching completes. Any incoming requests during this time are blocked and will return errors.
Version 3.1.1
Released: September 2021
New features
This release includes updates and enhancements to improve the performance of matching operations.
Attention:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
Changes in Functionality
This release contains no functionality changes.
Changes to third-party libraries
This release contains no changes to third party libraries.
Closed Issues
This release contains no closed issues.
Known Issues
This release contains the following known issues:
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
When matching is set to After submission and batch creation of records occurs using the EBX® API, the number of submitted or created records is limited to 2000 records in an API call for tables that have a single primary key. For a composite primary the limit is divided by the number of fields that makes up the primary key. For example, with a table that has a primary key comprised of 2 fields, the limit would be 1000 records. When your requirements exceed these limits, it is recommended that you pause the matching trigger, load the data, and resume matching. Also, note that when matching is triggered the dataspace is locked until matching completes. Any incoming requests during this time are blocked and will return errors.
Version 3.1.0
Released: July 2021
New features
This release includes the following new features:
-
A new REST API is available to perform matching related operations. You can use Swagger to view the API documentation. See REST overview for more information.
-
Matching operations after record submission are now available for records created using the EBX® API. There are some limitations to this functionality. See the Known issues section below for details.
-
Several matching services are now available to use in workflows and perspectives. See Perspectives and workflows for more information.
-
The add-on now supports link tables when matching with relationships. Additionally, to avoid confusion with EBX®'s concpet of associations, this feature was renamed to Use relationships.
-
The add-on's matching algorithm library now includes the Soundex algorithm. This algorithm works well for short strings such as names. See Matching algorithms for more information.
-
The list of algorithms is now filtered based on the search strategy configured for a matching field. Additionally, the validation report will display a warning if the configured algorithm is not compatible with the field's search strategy. See Matching algorithms for more information.
-
Administrators can now set a matching policy as the default policy from the Matching policies tab in the Actions menu.
-
A merge policy's record selection mode now includes the First acquired option to select the oldest record during survivorship.
-
The Customize source value for new golden now supports foreign keys.
-
Matching metadata can now be added to tables by creating a custom view. See Displaying matching metadata for more details.
-
The add-on's Consolidated view the Actions menu where you can execute matching related operations.
-
When viewing table data, you can now access the Metadata view's Lineage tab by selecting View lineage.
-
When a group contains only one record that does not need to be merged, a new Set as golden button allows you to set its state to golden.
Changes in Functionality
This release contains the following functionality changes: When using the Run match service to run a match on all records in a table, you must now select all records in the table instead of selecting nothing.
Changes to third-party libraries
This release contains no changes to third party libraries.
Closed Issues
This release contains no closed issues.
Known Issues
This release contains the following known issues:
-
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
-
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
-
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
-
When matching is set to After submission and batch creation of records occurs using the EBX® API, the number of submitted or created records is limited to 2000 records in an API call for tables that have a single primary key. For a composite primary the limit is divided by the number of fields that makes up the primary key. For example, with a table that has a primary key comprised of 2 fields, the limit would be 1000 records. When your requirements exceed these limits, it is recommended that you pause the matching trigger, load the data, and resume matching. Also, note that when matching is triggered the dataspace is locked until matching completes. Any incoming requests during this time are blocked and will return errors.
Version 3.0.1
Released: March 2021
Bug Fixes
Version 3.0.0
Released: March 2021
Release overview
The TIBCO EBX® Match and Merge Add-on locates and merges duplicate data values. The goal is to obtain a singular record that most accurately defines a business entity. The add-on identifies these as golden records.
Features available to administrators in include the ability to:
-
Make data available to the add-on by registering and activating matching on tables.
-
Configure a matching policy that determines how the add-on compares data. A matching policy also specifies whether users must start the matching process manually or if it starts automatically when data is created or updated.
-
Create a merge policy that establishes how duplicate data values are merged into a golden record. Merge policies can also execute automatically after a matching process runs.
-
Set up trusted sources that define a hierarchy of data sources. The add-on uses the source's trust level to during matching to choose the most accurate record and during a merge to select values to merge.Administrators can use Administrator Guide to learn more about configuration options available in this release.Features available to business users include the ability to:
-
Select data and initiate matching and merge operations. Search for records that might match when creating a new record.
-
Manage records that grouped together after running a matching process. The add-on groups records that are suspected of containing duplicate data or are positively identified as being duplicate. When managing groups you can also specify that a record is a false-positive match so that it no longer matches with the records in its group.
-
View record metadata applied by the add-on to categorize records. Two different views allow users to view individual record, or an entire table's metadata. Some information displayed includes the record's history, lineage, state, and records identified as false positive matches.
Business users can learn more about available actions and view in the User Guide.