Remove Duplicates from Record Collection Variables in a Flow

Last Updated on June 25, 2022 by Rakesh Gupta

Big Idea or Enduring Question:

• How do you remove the duplicates from the record collection variables or collection variables in flow without writing code?

Objectives:

After reading this blog, you’ll be able to: 

• Understand different flow operators in assignment element
• Understand how to remove common records b/w record collection variables (1 & 2) and store remaining records of record collection variable (1) 
• Understand how to remove uncommon records b/w record collection variables (1 & 2) and store duplicate records in record collection variable (1) 
• Understand how to remove duplicate records from record collection variables (1 & 2) and only store unique records 
• Understand how to remove duplicate records from both (1 &2) the record collection variables and only store remaining records across both record collection variable
• and much more

One of the very common questions people ask me is how to remove the duplicate records if I have two or more record collection variables. This article will go through various use cases to understand how to remove the duplicates from the record collection variables.

Prerequisite

Before discussing the different use cases, let’s pause here and spend some time setting up the data.

Go ahead and set up the test data as per the below screenshot. Setting the test data makes it easier to understand the duplicate record removal from the record collection variables instead of spending time validating outcomes.

The next step is to set up a basic flow to have two record collection variables of contacts with different query parameters.

Step 1: Define Flow Properties   

1. Click Setup.
2. In the Quick Find box, type Flows.
3. Select Flows then click on the New Flow.
4. Select the Screen Flow option and click on Next and configure the flow as follows: 
   1. How do you want to start building: Freeform
5. It will open the flow designer for you.

Step 2: Adding a Get Record Element to Find Contacts where Department = IT

The next step is to use the Get Records element to find the contacts whose department is IT. 

1. On Flow Designer, click on the +icon and select the Get Records element.
2. Enter a name in the Label field; the API Name will auto-populate.
3. Select the Contact object from the dropdown list.
4. Select All Conditions Are Met (AND). 
5. Set Filter Conditions
   1. Row 1:
      1. Field: Department
      2. Operator: Equals
      3. Value: IT
6. How Many Records to Store:
   1. select All record
7. How to Store Record Data:
   1. Choose the option to Automatically store all fields. 
8. Click Done.

Step 3: Adding a Get Record Element to Find Contacts where Lead Source = Web

The next step is to use the Get Records element to find the contacts whose Lead Source is Web. 

1. On Flow Designer, click on the +icon and select the Get Records element.
2. Enter a name in the Label field; the API Name will auto-populate.
3. Select the Contact object from the dropdown list.
4. Select All Conditions Are Met (AND). 
5. Set Filter Conditions
   1. Row 1:
      1. Field: Lead Source
      2. Operator: Equals
      3. Value: Web
6. How Many Records to Store:
   1. select All record
7. How to Store Record Data:
   1. Choose the option to Automatically store all fields. 
8. Click Done.

In the end, your Flow will look like the following screenshot:

Now two record collection variables hold the following data. 

Record Collection Variable (IT Dept. Contacts) – 1

RecordId	Last Name
003B000000KI9sxIAD	2
003B000000KI9t2IAD	3
003B000000KI9t7IAD	4
003B000000KI9tCIAT	5

Record Collection Variable (IT Dept. Contacts) – 2

RecordId	Last Name
003B000000KI9snIAD	1
003B000000KI9sxIAD	2
003B000000KI9t2IAD	3
003B000000KI9tHIAT	6
003B000000KI9tMIAT	7

Business Use case I

Jason Herr is a Solution Architect at Gurukul on Cloud (GoC). He wants to understand how to remove uncommon records b/w record collection variables (1 & 2) and store remaining records of record collection variable (1).

At the end, he is expecting the following records in Record Collection Variable (IT Dept. Contacts). 

Record Collection Variable (IT Dept. Contacts)

RecordId	Last Name
003B000000KI9t7IAD	4
003B000000KI9tCIAT	5

A solution for the above business requirement

This is a very common scenario for Salesforce Flow. Let’s try to understand Remove All operator in assignment element. 

Operator	Description	Supported Data Types	Example
Remove All	All instances of Value are removed from the collection in Variable.	Collection of the same data type or object type Variable of the same data type or record variable of the same object type For text collections only: Multi-Select Picklist Picklist $Flow.CurrentRecord	Before the Assignment: {!collText} is Red, Orange, Red, Yellow {!varText} is Red Assignment: {!collText} Remove All {!varText} After the Assignment: {!collText} is Orange, Yellow

Using Assignment Element to Remove All

The next step is to remove common records b/w record collection variables (1 & 2) and store remaining records of record collection variable (1). We will use the Assignment element

1. On Flow Designer, click on the +icon and select the Assignment element.
2. Enter a name in the Label field; the API Name will auto-populate.
3. Set Variable Value
   1. Row 1:
      1. Field: {!IT_Dept_Contacts}
         
      2. Operator: Remove All
         
      3. Value: {!Web_Sourced_Contacts}
4. Click Done.

Now your Flow will look like the following screenshot:

Almost there! Once everything looks good, click the Save button.

Let’s debug the flow to validate the outcome and match it with the expected outcome:

Business Use case II

Jason Herr was very happy with his learnings so far. He wants to learn how to remove uncommon records b/w record collection variables (1 & 2) and store duplicate records in record collection variable (1) 

At the end, he is expecting the following records in Record Collection Variable (IT Dept. Contacts). 

Record Collection Variable (IT Dept. Contacts)

RecordId	Last Name
003B000000KI9sxIAD	2
003B000000KI9t2IAD	3

A solution for the above business requirement

This is a very common scenario for Salesforce Flow. Let’s try to understand Remove Uncommon operator in assignment element. 

Operator	Description	Supported Data Types	Example
Remove Uncommon	The items in the Value collection are found within the Variable collection. The found items are kept, and all other items are removed from the collection in Variable.	Collection of the same data type or object type	Before the Assignment: {!collText1} is Red, Orange, Yellow, Green {!collText2} is Orange, Green, Blue Assignment: {!collText1} Remove Uncommon {!collText2} After the Assignment: {!collText1} is Orange, Green

Update Assignment Element to Remove Uncommon

The next step is to remove uncommon records b/w record collection variables (1 & 2) and store duplicate records in record collection variable (1). We will use the Assignment element

1. Update the Assignment element.
2. Set Variable Value
   1. Row 1:
      1. Field: {!IT_Dept_Contacts}
         
      2. Operator: Remove Uncommon
         
      3. Value: {!Web_Sourced_Contacts}
3. Click Done.

Almost there! Once everything looks good, click the Save button.

Let’s debug the flow to validate the outcome and match it with the expected outcome:

Business Use case III

Jason Herr wants to learn how to remove duplicate records from record collection variables (1 & 2) and only store unique records. 

At the end, he is expecting the following records in Record Collection Variable (IT Dept. Contacts). 

Record Collection Variable (IT Dept. Contacts)

RecordId	Last Name
003B000000KI9snIAD	1
003B000000KI9sxIAD	2
003B000000KI9t2IAD	3
003B000000KI9t7IAD	4
003B000000KI9tCIAT	5
003B000000KI9tHIAT	6
003B000000KI9tMIAT	7

A solution for the above business requirement

This is a very common scenario for Salesforce Flow. Perform the steps below: 

Update Assignment Element to Keep Unique Records

The next step is to remove duplicate records from record collection variables (1 & 2) and only store unique records. We will use the Assignment element

1. Update the Assignment element.
2. Set Variable Value
   1. Row 1:
      1. Field: {!IT_Dept_Contacts}
         
      2. Operator: Remove All
         
      3. Value: {!Web_Sourced_Contacts}
   2. Click Add Assignment
   3. Row 2:
      1. Field: {!IT_Dept_Contacts}
         
      2. Operator: Add
         
      3. Value: {!Web_Sourced_Contacts}
3. Click Done.

Almost there! Once everything looks good, click the Save button.

Let’s debug the flow to validate the outcome and match it with the expected outcome:

Business Use case IV

Jason Herr wants to learn how to remove duplicate records from both (1 &2) the record collection variables and only store remaining records across both record collection variable.

At the end, he is expecting the following records in Record Collection Variable (IT Dept. Contacts). 

Record Collection Variable (IT Dept. Contacts)

RecordId	Last Name
003B000000KI9snIAD	1
003B000000KI9t7IAD	4
003B000000KI9tCIAT	5
003B000000KI9tHIAT	6
003B000000KI9tMIAT	7

A solution for the above business requirement

This is a very common scenario for Salesforce Flow. Perform the steps below:

Salesforce Flow – Create Collection Variable to Store Contacts

1. Under Toolbox, select Manager, then click New Resource to store multiple contact records.
2. Input the following information:
   1. Resource Type: Variable
   2. API Name: varR_ContactsPlaceholder
   3. Data Type: Record
   4. Object: Contact
   5. Click the Yes checkbox – Allow multiple values (collection)
   6. Check Available for Input
   7. Check Available for Output
3. Click Done.

Update Assignment Element to Keep Unique Records

The next step is to remove common records from both (1 &2) the record collection variables and only store remaining records across both record collection variable. We will use the Assignment element

1. Update the Assignment element.
2. Set Variable Value
   1. Row 1:
      1. Field: {!varR_ContactsPlaceholder}
         
      2. Operator: Add
         
      3. Value: {!IT_Dept_Contacts}
   2. Click Add Assignment
   3. Row 2:
      1. Field: {!IT_Dept_Contacts}
         
      2. Operator: Add
         
      3. Value: {!Web_Sourced_Contacts}
   4. Click Add Assignment
   5. Row 3:
      1. Field: {!varR_ContactsPlaceholder}
         
      2. Operator: Remove Uncommon
         
      3. Value: {!Web_Sourced_Contacts}
   6. Click Add Assignment
   7. Row 4:
      1. Field: {!IT_Dept_Contacts}
         
      2. Operator: Remove All
         
      3. Value: {!varR_ContactsPlaceholder}
3. Click Done.

Almost there! Once everything looks good, click the Save button.

Let’s debug the flow to validate the outcome and match it with the expected outcome:

Formative Assessment:

I want to hear from you!

What is one thing you learned from this post? How do you envision applying this new knowledge in the real world? Feel free to share in the comments below.