By Artem Korsakov ·
Preparing data for automation starts with a simple question: do different systems understand the same record in the same way? In the CRM a supplier may be called “Sever”, in the accounting system it may be stored as a legal entity, and in the procurement spreadsheet it may have an old abbreviation. The data transfer technically succeeds, but the invoice is linked to the wrong counterparty. For a business, such a result is worse than a visible exchange error: the interface looks fine until wrong information reaches a decision.
In short: choose the data for one process, define the master record and the matching rules. Check duplicates, required fields, units and versions. AI can suggest matches and explain the attributes, but merging records and replacements are confirmed by the owner of the reference data.
Why a working integration can carry errors
The status “successfully transferred” confirms a technical action. It does not check whether the meaning is correct. One hundred units in the source may mean pieces, packages or metres. The same material name can hide different sizes. A contact person may change employer, while their old link remains in the CRM.
That is why delivery of a record and its business content are checked separately. Handling repeated submissions solves the problem of technical duplicate operations. It does not determine whether two existing cards are the same supplier. This article is specifically about reference data and the matching rules needed before an exchange is launched.
You do not have to clean all of the company's data at once. If you are creating a material request service, it is enough to start with the products in use, departments, sites and suppliers. A ten-year-old archive can be left out of the first scope if that does not break the chosen process and the agreed search of history.
Assign an owner to each type of data
For each reference list, you need to define the master system and the person responsible. For financial details, this may be the accounting database and the accounts department; for product characteristics, the catalogue and procurement; for contact data, the CRM and sales. You need a specific agreement: the universal rule “everything important is stored in the ERP” does not work here.
The 1C (a Russian accounting and ERP platform) Standard Subsystems Library describes tools for working with classifiers (1C is a widely used Russian business accounting platform), including address and bank data. This is an example of how structured data may already be supported by existing tools. The existence of the library does not guarantee that the needed function is enabled and configured in your configuration.
Also assign a change procedure. Who creates a new card, who checks required fields, who allows a merge, and how do other systems find out about it? If any department can independently rename an entity, a one-off clean-up will not keep quality up for long.
Check keys and links between systems
A name is convenient for a person but poorly suited as the only key. A card needs a stable identifier from the source system. A mapping table links it to the identifier in another system, the date of the check and the person responsible. After renaming, this link must remain.
Duplicate counterparties are searched for by a combination of attributes. For Russian legal entities, verified registration details can be used together with the organisational structure, but a matching name alone proves nothing. A branch, a separate legal entity and a group of companies require different representations. For foreign counterparties, the set of identifiers depends on the country and the available documents.
Contacts, contracts and bank accounts store separate links. Merging two suppliers can affect order history and user access. That is why a reference list cannot be cleaned by simply deleting “extra rows”. First you need to see the dependent records and agree on how to preserve what they belong to.
Units of measure require explicit rules
Data quality for AI is especially visible in units and characteristics. A model may understand that “pk.” means a package but does not know how many pieces are in a specific package without a source. The factor applies to a specific product and packaging version. The word “box” has no factor at all.
An illustrative example: the order states 12 packages, and the agreed reference data for this item records 20 pieces per package. The order corresponds to 240 pieces. The warehouse received 200 pieces. The undelivered balance on the order is 40 pieces, that is, two such packages. If the supplier stated that it shipped all 240, the receiving discrepancy requires a separate check; the comparison with the order alone does not yet prove a shortage. Comparing the numbers 12 and 200 without units makes no sense.
Illustrative example: 12 packages of 20 pieces, 240 ordered, 200 received. The balance on the order is 40 pieces, that is, two packages. A new packaging factor applies only to new deliveries.
If the supplier changed the packaging to 25 pieces, the old factor cannot be silently rewritten throughout the history. For a new delivery, the current basis and the effective date are recorded. The previous order continues to mean its 240 pieces unless its terms are changed separately. This is a matter of versioning reference data and documents. Text recognition does not solve it.
How to run a small audit in existing tools
Export an agreed set of fields from one process and keep an original copy. Check empty required values, repeated keys, unexpected formats, unused categories and references to missing records. Distinguish “value unknown”, “not applicable” and an actual zero: for calculations these are different states.
Excel may already include Power Query tools. The official documentation describes column profile and quality and warns that by default the analysis may cover only the first 1,000 rows. To check the full set, the mode must be changed. This is a useful concrete check: a tidy first thousand does not prove the whole export is clean.
The result of the audit is a register of issues with owners. An abstract quality percentage in a presentation will not help here. For example: 14 products without a unit, 6 disputed supplier matches, 3 missing links to sites. Such figures are taken from your own export and labelled with the date and scope of the check.
Where AI helps and where strict checks are needed
AI is useful for grouping similar names, extracting characteristics from descriptions and suggesting possible matches. The input is a limited reference list and confirmed examples. The output should be pairs of records, matching attributes, differing fields and a link to the source of each significant attribute.
The employee sees, for example: the same item code and size, but the material and version differ. That is a reason to clarify. It is too early to merge such records. In a text explanation the model may be wrong, so the original values are shown alongside. For financial details, units and critical characteristics, checking is mandatory regardless of the assistant's confident tone.
Exact rules find empty keys, invalid formats and uniqueness violations. AI helps where descriptions differ but the meaning may be the same. With this division, the model does not spend money checking an empty field, and similarity is not taken as proof of identity.
Training example: AI suggests pairs of records from two reference lists. For each pair, you can see which attributes match and which differ. The link is confirmed by the owner of the reference data.
Why merging duplicates requires a separate decision
The Dataverse documentation describes detecting and merging duplicates together with related records. It also states the limitations of specific modes. For your own solution, the general conclusion matters: merging affects links, so it must be designed separately from finding matches. The capabilities of someone else's platform cannot be automatically attributed to your CRM.
The proposed safe working route: candidate, review by the owner, confirmation of the master card, transfer of agreed links, marking the replaced record and checking the result. The original identifiers are kept in the mapping table. If a record is already used in closed documents, the correction procedure is determined by the person responsible for the relevant system.
Reversibility must also be checked in advance. Being able to restore a reference list from a copy does not guarantee that an external message already sent or a document already created can easily be undone. That is why the first processing run is done on a copy or in report mode, without bulk writing of changes into working systems.
What counts as ready for launch
For one process, formulate specific conditions. All active records have an identifier; required units are known; critical links are confirmed; disputed matches go to a manual queue; new records go through the same review procedure. The acceptable error threshold is set according to the consequences of an error; there is no universal number.
Check several complete scenarios: a new supplier, a rename, a packaging change, a repeated upload, correction of a wrong match. It is important to see what the system does when data is missing. Confidently filling in a guess is worse than an explicit “needs clarification” status.
Cleaning reference data ends with a working support procedure. Removing all the red rows is not enough. Appoint an owner of the queue, a response time and a date for the next check. If the number of disputed records grows faster than the team can resolve them, it is too early to scale the exchange.
What product can be built first
A small solution can accept exports, show a field profile, suggest matches, collect owners' decisions and produce a verified table for import. Connection to working systems is added after the rules and permissions have been checked. Such a product helps prepare a specific process without trying to build a corporate platform for managing all data.
For an estimate, prepare two anonymised exports that currently have to be matched by hand, and a few known difficult cases. trike_dev can help define the rules, design the review interface and connect AI for suggestions with clear reasoning.
Related reading: Data exchange between systems: how not to lose requests on failures; An automation brief: what the client should describe.
Частые вопросы ### Does the whole archive need to be cleaned before automation?
No, first define the data involved in the chosen process and the necessary history. The rest can be processed later. The boundaries must be explicit so that the user does not take a partial reference list for the company's complete archive.
Can AI merge similar records by itself?
It can suggest candidates, but text similarity does not prove identity. For the first version, a merge is confirmed by the data owner after checking the details and dependent records. Payment information and technical characteristics are handled with particular care.