by So you have a list of strings or integers and find duplicate items. It’s a common problem. You need to remove the duplicates to be able to process them. I don’t know how, or your strategy is taking too long to process. There are multiple ways to address this, but I want to share the most efficient way with you.
Test Array
So let’s say that you have an array of elements like strings or integers. I’ll provide an example of objects (list of SharePoint items for example) in another article since it’s a bit more complex, but I’ll link it here once it’s published.
Let’s use our createArray function to get a test array in a compose action.
createArray('Bob','Manuel','Bill','Manuel')
Here’s what we get when we run the Flow.
I know it’s not the most creative example, but it does the trick. As you can see, my name is duplicated in the list, so let’s look at how to remove it less efficient way.
The less efficient way
Our first instinct would be to do the following:
- Define a variable
- Use an Apply to Each to check all items in the array.
- Check if the item exists in the variable.
- If not add it to the variable.
Something like this:
It would work but take a long time since Power Automate needs to cycle through all objects. You could parallelize it, but the issue still stands. Also, each iteration of the Apply to Each and Append to Array Variable counting toward your action limit for each item it processes. So, we would have a slower and more costly solution to work with.
Now, let’s look at a better solution.
The efficient way
A better way is to use the “union function”. With only one action we can do everything that we need. So you only need an action that supports formulas, like the “compose action” for example, you can have the same result in a fraction of the time.
So let’s add a new compose action and build the formula.
Here’s the formula:
We need to use the “union function” on the same array twice. It looks like a typo but this is what we need. The function will return the following:
The function will get all items from both arrays and return the combined list of items without duplicates. If you provide the same elements on both sides, you will take advantage of the “without duplicates” part of the function and get a list off items without duplicates.
You can use this formula in any action that supports formulas, so if you want to use it in a “set variable” action you can, for example.
Simple and only using one action!
Is it that faster?
So we have a fast solution, and that only requires one action. I know that “fast” is relative, so here’s both running side-by-side with the same information.
With only 4 items it takes 4 seconds to process, so imagine what it would take with bigger lists.
Final Thoughts
I could have shown you only the solution, and that’s all you need, but it’s essential for me to show why something is better and why a strategy works. This way you can learn more
Please don’t fall into the trap of using Apply to Each actions when they are not needed. This solution is exponentially faster and with the added value of being simpler. Can’t beat that!
You can follow me on Mastodon (new account), Twitter (I’m getting out, but there are still a few people who are worth following) or LinkedIn. Or email works fine as well 🙂
Photo by Joanna Kosinska on Unsplash
