r/learnjavascript • u/webdevolution_ • Jul 05 '21
Remove duplicates from arrays using Set
18
u/NANOwasFound Jul 05 '21
What is the use of ... before new? Sorry if it's dumb I am very new to js
44
u/throwaway1253328 Jul 05 '21 edited Jul 05 '21
it's the spread operator
const a = [1, 2, 3, 4, 5]; const b = [6, 7, 8, 9, 10]; // [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] const combined = [...a, ...b]; // can also be used with destructuring const [first, ...rest] = combined; // 1 console.log(first); // [2, 3, 4, 5, 6, 7, 8, 9, 10] console.log(rest); // 2, 3, 4, 5, 6, 7, 8, 9, 10 console.log(...rest); // or with objects const aObj = { x: 1, y: 2 }; const anotherObj = { z: 10, ...aObj }; // { x: 1, y: 2, z: 10 } console.log(anotherObj);7
18
u/dvlsg Jul 05 '21
It's just turning the
Setback into anArray.const filteredArray = Array.from(new Set(array));Would do basically the same thing.
5
1
4
u/Delfaras Jul 05 '21
it's the spread operator:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_syntax
3
7
u/Web-devil Jul 05 '21
That’s nice to know
Problem is, we all know wtf a set it in math, but can’t remember the method lol
1
3
u/HashFap Jul 05 '21
Only works on primitive values like strings, numbers, and booleans.
14
u/revenezor Jul 06 '21
This is not true. It works on all types, including objects. Duplicates of object references are removed, and there is no such thing as duplicates of object literals as each new object is unique, even if it shares the same properties.
1
Jul 06 '21
How does it consider an object when making the set? Is it if all fields are the same?
4
u/revenezor Jul 06 '21 edited Jul 06 '21
Do you mean how does it determine whether two objects are the same? The same way as the
===operator. If two elements are a reference to the same object then the elements are considered “equal” and the second one is removed, or rather never added to the set.const foo = {a: 1} const bar = foo const duplicates = [foo, bar] const uniques = […new Set(duplicates)] console.log(uniques) // [{a: 1}]Whenever you create an object it is considered not equal
!==to any previous object, even if its properties match a previous object’s.const foo = {a: 1} const bar = {a: 1} const notDuplicates = [foo, bar] const uniques = […new Set(notDuplicates)] console.log(uniques) // [{a: 1}, {a: 1}]0
u/FriesWithThat Jul 06 '21
If someone has a use case for it, they can create a new array with all identical values removed with something like this:
const data= [ 1, 2, 1, { id: 10, other: 'blah' }, { id: 15, other: 'meh' }, { id: 10, other: 'blah' }, ]; const newObj = {}; const filterValues = data.filter(el => { if (!newObj[JSON.stringify(el)]) { newObj[JSON.stringify(el)] = true; return el; } return false; }); console.log('filterValues:', filterValues);3
u/nadameu Aug 01 '21
This might not work if "id" and "other" are declared in a different order when creating the object.
1
1
1
Jul 05 '21
In the situation you are using a strict af Typescript config, you can also do this
const array: any[] = [ 1, 5, 1, 3, 2, 1, true, false, true, 'Matt', 3, 2 ];
const filteredArray: any[] = Array.from(new Set(array));
console.log(filteredArray); // [1, 5, 3, 2, true, false, "Matt"]7
Jul 05 '21
const array: any[] = [ 1, 5, 1, 3, 2, 1, true, false, true, 'Matt', 3, 2 ];
const filteredArray: any[] = Array.from(new Set(array));This just removes all type safety... With OP's example in TS strict mode, the resulting array is typed as
(string | number | boolean)[]which is much better thanany[].2
u/Gibbo3771 Jul 05 '21
Being fair, OPs dataset is actually pretty rare to come across anyway. Normally (ideally, really) an array should contain elements that are all of the same type.
1
Jul 05 '21
Either way it works with TypeScript strict mode without using an
anyescape hatch, though.1
Jul 05 '21
I see where you’re coming from, but you shouldn’t ever mix types within your arrays anyways for simplicity sake. The point was not the “escape hatch”, the point was
Array.from. Unless you have the specific TSConfig flag, you’ll need to useArray.fromover the prettier[… new Set()]syntax.2
Jul 05 '21 edited Jul 06 '21
I see what you mean now, but what you're really talking about is developing in an environment where you can not use the iteration protocol, not a "strict TypeScript config". For TypeScript this means your tsconfig targets ES5 or lower and you have the
downlevelIterationflag set to false. In that case you're likely targeting IE11 with minimal/no polyfills, andArray.fromwon't work anyway (not supported in IE11, and sinceSetis only partially implemented in IE11 you would need a specific polyfill that can actually makeArray.from(new Set())work).I see where you’re coming from, but you shouldn’t ever mix types within your arrays anyways for simplicity sake.
For unknown-length arrays containing primitives like strings and numbers, I generally agree, but there are many cases when a mixed type array is perfectly reasonable (tuples, arrays of discriminated union objects, etc). My point was that if you're using a strict TypeScript config, you should really avoid using
anyas much as possible.TS can deal with a mixed type array in a type-safe way, but
anyjust turns off type safety entirely by design.1
Jul 06 '21
Oh, damn. This is good to know. I had to turn downLevelIteration on to use the prettier method in a project I’m working on, but didn’t did too deep into why I had to in the first place. We only target modern browsers, and for whatever reason the project defaulted to false for that flag.
I had assumed it was a general default for typescript configurations.
Thank you!
1
u/20EYES Jul 05 '21
Typically if I need a unique set I just use Set in the first place. A Set can't be populated with duplicate elements so I just insert whatever comes up and if it's duplicate no worries.
What is the use case for starting with an array full of possible duplicates and then converting to a set?
I'm probably overlooking something obvious lol.
3
1
u/JazzApple_ Oct 28 '21
Totally right that if you can just use a set from the start, but there are so many potential use cases… often relating to user input.
0
Jul 05 '21
[deleted]
1
u/20EYES Jul 05 '21
As in the user provides an array?
1
u/WebDevImpasta Jul 05 '21
No, the user would just provide data or values. And then you would design the program to store the data into an array. But for whatever reason, you don't want duplicates, you can get a new array without any duplicates.
Or if you consume an API that produces duplicates.
0
-14
u/Dazzling-Wafer Jul 05 '21
In reality, you don't have to find a 'fancy' way to remove duplicates at all. You just declare a removeDuplicates() function in the helpers folder and then reuse it everywhere.
9
u/jnbkadsoy78asdf Jul 05 '21
// helpers/remove-duplicates.js export default function(array) { return [...new Set(array)]; }Something like that? Looks fancy.
2
u/steezefries Jul 06 '21
Oh cool bro could you go ahead and imagine what that function definition might look like bro?
30
u/[deleted] Jul 05 '21
Thanks for not making this a 9 minute YouTube video with electro bass music and text on screen