Compare Datasets consultants

Q: How does the Compare Datasets node match records between two datasets?

The node matches records using one or more key fields that you specify, such as an email address, product ID, or employee number. It compares records with matching keys and reports which fields differ. Records without a matching key in the other dataset are flagged as additions or removals.

Q: What output does the Compare Datasets node produce?

The node produces up to three output branches: records found only in the first dataset, records found only in the second dataset, and records found in both datasets but with differing field values. Each output includes the full record data so you can process the differences in downstream nodes.

Q: Can I compare datasets from different sources, like a CRM and a spreadsheet?

Yes. You can pull data from any two sources (databases, APIs, spreadsheets, CSV files) using their respective nodes, then feed both datasets into the Compare Datasets node. The comparison works on the structured data regardless of where it originated, as long as both datasets share a common key field.

Q: How do I handle large datasets with thousands of records?

The Compare Datasets node can process thousands of records, but performance depends on your automation platform's memory and execution limits. For very large datasets (tens of thousands of records), consider batching the comparison by splitting data into chunks, or running the comparison during off-peak hours to avoid timeout issues.

Q: Can I schedule automatic reconciliation runs?

Yes. Combine the Compare Datasets node with a schedule trigger to run reconciliation at set intervals: daily, weekly, or at any frequency you choose. The workflow pulls fresh data from both sources, compares them, and reports or acts on the differences automatically each time it runs.

Q: What happens when field formats differ between datasets?

If the same data is formatted differently in each source (for example, dates as DD/MM/YYYY versus YYYY-MM-DD), the comparison will flag them as different. To avoid false positives, add data transformation nodes before the comparison step to normalise formats across both datasets.

We can help you automate your business with Compare Datasets and hundreds of other systems to improve efficiency and productivity. Get in touch if you’d like to discuss implementing Compare Datasets.

Get in touch

Book a call

About Compare Datasets

Compare Datasets is a workflow node that takes two sets of data and identifies the differences between them. It compares records field by field and outputs three groups: items that exist only in the first dataset, items that exist only in the second dataset, and items that exist in both but have different values. Data teams, operations managers, and finance departments use it to catch discrepancies between systems without manually cross-referencing spreadsheets.

Common use cases include reconciling CRM records against billing system data, identifying new or removed products between catalogue versions, detecting changes in employee records across HR systems, and verifying that data migrations transferred all records correctly. Any time you need to answer the question “what changed between these two sets of data?”, this node handles it programmatically.

Osher uses Compare Datasets as a core component in automated data processing workflows that keep multiple systems in sync. We build reconciliation pipelines that pull data from two or more sources, compare them automatically, and take action on the differences: creating missing records, flagging discrepancies for review, or updating stale data. This replaces the manual spreadsheet comparisons that consume hours of your team’s time every week. See how we applied similar data reconciliation techniques in our BOM weather data pipeline project.

Compare Datasets FAQs

Frequently Asked Questions

Common questions about how Compare Datasets consultants can help with integration and implementation

How does the Compare Datasets node match records between two datasets?

What output does the Compare Datasets node produce?

Can I compare datasets from different sources, like a CRM and a spreadsheet?

How do I handle large datasets with thousands of records?

Can I schedule automatic reconciliation runs?

What happens when field formats differ between datasets?

How it works

We work hand-in-hand with you to implement Compare Datasets

As Compare Datasets consultants we work with you hand in hand build more efficient and effective operations. Here’s how we will work with you to automate your business and integrate Compare Datasets with integrate and automate 800+ tools.

Step 1

Pull Data from the First Source

Add a node that retrieves records from your first data source. This could be a database query, API call, spreadsheet read, or CSV import. Ensure the output includes a unique identifier field that exists in both datasets.

Step 2

Pull Data from the Second Source

Add a second node that retrieves records from your comparison source. This is the dataset you want to compare against the first. Make sure it includes the same unique identifier field for matching.

Step 3

Configure the Compare Datasets Node

Connect both data sources to the Compare Datasets node. Specify the key field (or fields) used to match records between the two datasets, such as email address, product SKU, or record ID.

Step 4

Select the Comparison Fields

Choose which fields beyond the key field should be compared for differences. You can compare all fields or select specific ones. This controls what the node considers a ‘changed’ record versus an unchanged one.

Step 5

Process the Differences

Add downstream nodes to handle each output branch. For new records, you might create entries in the target system. For removed records, flag them for review. For changed records, update the relevant fields or notify your team.

Step 6

Test with Known Differences

Run the workflow with test data where you know the differences in advance. Verify that the node correctly identifies additions, removals, and changes. Confirm that downstream actions handle each case as expected.

Transform your business with Compare Datasets

Unlock hidden efficiencies, reduce errors, and position your business for scalable growth. Contact us to arrange a no-obligation Compare Datasets consultation.

Get in touch

Book a call