r/learnjavascript • • May 14 '21

Remove Duplicate Value from array

Post image
234 Upvotes

89 comments sorted by

View all comments

20

u/slowthedataleak May 14 '21

Does this perform better than filter()?

78

u/StoneCypher May 14 '21

no, dramatically worse.

this is actually a terrible approach, and a big part of the reason why you shouldn't learn javascript from memes by dev.to spammers on reddit

16

u/[deleted] May 14 '21

It's less performant, but not dramatic. Whether or not this is faster is an implementation detail in the runtime essentially. It is a lot more concise though, and hard to get wrong. 99/100 times i'd just import uniq / uniqBy from lodash anyway.

-26

u/StoneCypher May 14 '21

It's less performant, but not dramatic.

It's the construction and iteration of an entirely unnecessary unordered container

It's not clear how you'd make this "dramatically worse" that this isn't, other than to just make it wrong, or do something nonsensical

Yes, it's dramatic. It's large constant big o lg n twice plus small constant big o n lg n for literally no reason.

.

Whether or not this is faster is an implementation detail in the runtime essentially

this isn't even close to correct

.

It is a lot more concise

Not than the good answer

.

99/100 times i'd just import uniq / uniqBy from lodash anyway.

Wow, please don't give advice anymore

I've had to update my libraries 11 times this week because people like you imported lodash or underscore for no good reason, and they're full of bugs and security problems

It's so hard to just have decent testing or doc generation because of things like this :(

This is the mindset that left a serious chunk of the internet broken over padLeft

13

u/[deleted] May 14 '21

this isn't even close to correct

It's very correct, i welcome you prove otherwise.

Not than the good answer

Please provide the good answer then

Wow, please don't give advice anymoreI've had to update my libraries 11 times this week because people like you imported lodash or underscore for no good reason, and they're full of bugs and security problems

I don't think that's correct. If you honestly believe you write fewer bugs and more secure code than lodash, please provide some kind of proof, because you're likely wrong.

Yes, it's dramatic.

No it's not dramatic.

I did a jsbench in chrome test at various array sizes, 10, 100, 1000, 10_000, 100_000, ..., on three different approaches - Set, filter + indexOf, filter + object.

For small arrays filter + indexOf is the fastest. For very large arrays filter + object is fastest. For the middle Set / filter + object are not significantly different.

And of course the filter versions could be written with a loop and be faster.Overall though a pointless optimization unless perf testing your app reveals that you need it. Lodash takes size of array into account and chooses approach based on that, so you get the optimization for free.

-11

u/StoneCypher May 14 '21

Whether or not this is faster is an implementation detail in the runtime essentially

this isn't even close to correct

It's very correct, i welcome you prove otherwise.

What's there to disprove? You just said randomly "this is an implementation detail." You didn't make a technical claim.

This is like saying "node.js is angry which is an implementation detail" and when someone says that doesn't make sense saying "prove me wrong"

No, dear heart. You prove yourself right. Other people don't prove you wrong.

"An implementation detail" means that something is a decision for the underlying machine, that is out of the developer's hands.

You just said "the shape of the code that the developer chooses to write is not up to the developer."

That's absolute nonsense.

Every part of that code is hard-defined by the specification. None of it is an implementation detail.

That phrase has a meaning. You're hard wrong.

 

Please provide the good answer then

There's tons of them. I prefer one that you probably won't prefer, because this should be a tiny library function tested and put away in some module somewhere, and even though it's a one-liner nobody will ever see, you're crying out for "simplicity"

The function can't be well written; the name is ambiguous. You need to provide a stub that is clear about what is being done with holes, undefined, and symbols.

Holes are a problem because the various implementations mostly lose them. However, under most interpretations of the ECMA-262 concept of an lvalue, they need to be kept.

Symbols are a problem because the concept of uniqueness is a conceptual error around them in general. Two copies of the same symbol are not equal. equivalence is broken for them on purpose, and uniqueness is conceptually defined in terms of equivalence.

undefined is a problem

.

No it's not dramatic. [...] I did a jsbench

I see that you:

  • didn't give numbers

  • didn't show your test code

  • didn't share the share-able bench in case there were mistakes

  • didn't cover correctness topics

  • didn't specify whether your implementations gather the same results

  • said there were differences in speed but also argued against my saying there were differences in speed

  • said "the one in the middle isn't too far from one side" which is obviously going to be the case because the one side is the other good approach

  • compared several tiny sizes and referred to one of them as large

  • used a benchmarker that can't see micro-timing at all

  • ignored the o claim

Compelling stuff

Wanna show us this bench?

I benched it too and it turns out the solution I don't like took infinity hours and the solution I do like gave me a pony

 

For very large arrays

100k is not a very large array. Lawdy.

Node authors regularly deal with two or three orders of magnitude larger than this.

The US map on d3 has more than 50x this many SVG tags. SVG tags are not simple.

 

I don't think that's correct. If you honestly believe you write fewer bugs and more secure code than lodash, please provide some kind of proof, because you're likely wrong.

I didn't say anything like this.

What I did say is "don't involve a specific library that's had three criticals in the last two weeks to get away from a simple one-liner."

You know, the three one-liners you claim to test in a jsbench later?

It's very easy to avoid lodash in favor of the standard library.

I didn't say my code has fewer bugs. I said the standard library does.

I don't need to prove that. It's tautological. Anything written using the standard library, such as lodash, already includes every bug in the standard library.

Do I believe that chrome has fewer security bugs than lodash, which had three criticals in the last week?

Emphatically yes.

You want me to accept a benchmark you wrote in 15 minutes, but also you're skeptical that Chrome has fewer security bugs than lodash, one of the least secure libraries on NPM 😂

 

Overall though a pointless optimization

It's annoying when novices respond to "don't do it that way, it's bad for a bunch of reasons including correctness and performance" with "YoU'rE jUsT mAkInG uNnEcEsSaRy OpTiMiZaTiOnS"

Correctness is not an optimization

lodash does what I believe to be the wrongest things with uniqueness around holes, and around Symbols (I do agree with what it does around undefined)

I doubt you even thought to check

 

Lodash takes

Lodash has had three unrelated security vulnerabilities marked critical or worse in the last two weeks

I cannot take seriously the opinions of someone who keeps pushing for one-liners to be replaced with it, after they were told that once already

If you're arguing against maintenance, arguing for things with a history of grim security problems is arguing against yourself

12

u/[deleted] May 14 '21

Please be more concise, I have no interest in countering a proof by verbosity where you've introduced tons of new arguments that had nothing to do with the original post or discussion and contradict your own self many times.

-11

u/StoneCypher May 14 '21

A third time, I'm requesting that you provide the perf you claim to have written

You're the one who brought lodash up, not the original post or discussion

Now that I'm pointing out why adding lodash is a security problem, which is a hard fact but which you refuse to accept, you're babbling on about how everything has to be scoped to the original post or original discussion

Show us your perf that you're trying to stand on as evidence. I'm starting to think it isn't real, because I wrote one that got really different results

1

u/[deleted] May 14 '21 edited May 14 '21

Lodash's vulnerabilities only matter in relation to the code that you write that is allegedly better, since you are advocating to write it yourself rather than let someone else do so.

You yourself said that this is a tiny library function, a one liner, and that you could write just as concisely as [...new Set(arr)], and that it'll be dramatically more performant than the OP on an array with millions of items. But you refuse to do so for some reason. Can't be time since you have the time to repeatedly write large walls of text.

Furthermore I've looked through Lodash CVEs and I see none relating to function in question. The majority of them seem to be prototype pollution from functions that merge objects, which is outside of the scope of deduping an array. So it was a moot point to begin with designed to distract from the actual point.

Finally, since you allegedly have the ability to write code that is faster, safer, more maintainable, and more concise than lodash - why are you so selfish as to keep this code to yourself? Publish it and let the community benefit.

-3

u/StoneCypher May 14 '21

A fourth time, I'm requesting that you provide the perf you claim to have written

→ More replies (0)