r/ProxyEngineering • • Jun 08 '26 Galaxy Brain Hooray Thanks

Hot Take šŸ”„ My Samsung TV is literally being rented out as a data collection tool and everyone is fine with it???

146 Upvotes

So I was reading some blogs last night and I basically couldn't sleep after. There's this Israeli company called Bright Data, yes you know it, everyone knows it and everyone is fine with this company for some reason. Well, they embeds code into free streaming apps, apps that are already on your Samsung TV or LG TV right now, and when you click "agree" on some fine print screen asking if they can "occasionally use your device resources" your TV basically becomes a node in a massive proxy network that AI companies use to scrape the entire internet. Usually these prompts appear so that they can use the data to target and personalize ads, but this is different. Your TV. The one sitting in your living room. Scraping websites 24/7 using YOUR home IP address and YOUR bandwidth. Brother, what the hell?? The actual "consent" is buried in an opt-in screen that says they'll "occasionally" use your device. Occasionally. The actual SDK settings allow up to 200GB a month. On some configurations it keeps running in the background even when the app is closed. Even when you're watching something else. As long as the TV is plugged in, it's working for them. The thing that really got me was that on iPhones the traffic bypasses your VPN. Like if you have a VPN specifically because you don't want companies snooping on your traffic, this SDK just routes around it anyway. What the fuck mate, honestly. Security researchers said the authentication on the peer channel is basically weaker than most malware. Some thoughts for you. Oh and who is buying all this scraped data? AI companies. Your TV is training ChatGPT or whatever and you're getting absolutely nothing for it except maybe "fewer ads" in some random Roku app called Petflix that you downloaded once and forgot about or you even don't do that because the tvs come preinstalled with these apps. Google and Amazon and Roku have all banned this SDK from their platforms now which honestly tells you everything you need to know. Samsung has said nothing as far as I checked. LG said it's "not officially supported" which is corporate speak for "we know and we don't care." I already blocked the domains on my router (proxyjs.brdtnet.com and a few others, easy to find) but I shouldn't have to do that. I paid almost $2000 for this TV. I didn't agree to turn it into a botnet for OpenAI's lunch. Dumb TVs were fine. If you have better solutions, let me know, coz at the moment, I'm fuming.


r/ProxyEngineering • • May 22 '26 All Seeing Upvote Hooray Heartwarming Bravo

Discussion šŸ’¬ I reverse engineered how TikTok's detection works

130 Upvotes

Spoiler alert: none of you are going to like what I found.

Been trying to wrap my head around for months and after a while, packet captures, JS deobfuscation, (if you don't know what deobfuscation is, do a research), burning through around $800 in proxies "for science," I finally have a clear picture of what's actually going on under Chinese engineered masterpiece. The proxy debate is a distraction. TikTok doesn't primarily flag you on IP. It flags you on a fingerprint cluster score, a weighted combination of signals that builds a confidence interval on whether you're a real human on a real device. The things that move that score are not what most people would think and I can confirm this because I've been lurking around reddit to see what people has come up with. WebGL renderer string consistency across sessions. Canvas noise variance patterns where they track how your noise changes, not just that it exists. Gyroscope and accelerometer absence on sessions claiming to be mobile. Touch event timing distributions because real thumbs are messy and yours aren't. Battery API polling behavior. The gap between your declared timezone, your DNS resolver location, and your actual behavioral patterns. Here's the secret about residential proxies. Your IP is "clean" (yes, for those who are still bowing down to your fake fraud scam websites), your ASN is fine, but your device screams bot from miles away. You're spending $ to hide the one thing they're barely even checking anymore. What flipped the results for me was moving to real cloud phone environments with genuine sensor data. Not emulators, not spoofed headers, actual ARM instances with real hardware entropy. Shadowban rate dropped from around 60% to under 8% almost immediately after switching. I'm not dropping provider names. That part you figure out yourself. Downvote if I'm wrong. Upvote if you've seen the same thing. This might be too niche for some of ya knuckleheads


r/ProxyEngineering • • Oct 07 '26

Hot Take šŸ”„ TikTok bans scraping in their ToS and some guy just posted 5.6B videos on HuggingFace

128 Upvotes

I was looking for some agent models on Hugging Face, then after tinkering around I went to check whats new in the dataset tab. To my surprise some dev dumped metadata for about 5.6 billion TikTok videos on Hugging Face. It includes captions, hashtags, sound IDs, shop product and seller IDs, every engagement count. 460GB of data. Now, to my knowledge, TikTok does not allow scraping of their videos "legally", according to the devs own explanation they went straight at the private mobile API with generated device identities that pass as Android phones, reverse engineered request signatures and a spoofed TLS handshake. What that means is that no account and no login was required. Can you believe this? They claim 3 billion profiles, almost 6 billion videos and 2.8 billion comments in THREE WEEKS. Meanwhile what I've seen on this sub in particular, probably few others too, but this one in particular is that people here spends half if not more time arguing about whose IP pool is cleaner. And TikTok is one of those more difficult targets to scrape. What I'm trying to say is that nobody built this by being clever with IPs, checking whether they leak DNS and other data, whether they are "clean" or not abused or whatever, they built it by making the approach look like a legitimate phone. I mean, that is just crazy. Anyways, I saw that the free version is non commercial only, the dataset I mean, if you want creator profiles, daily updates basically run the whole thing yourself, or to use it for anything that makes you money, you should visit their site the code is up there for $1,699.

Now why this caught my interest in the first place. TikTok's terms ban this kind of scraping, and the official research API is mostly for academics in a few regions. Reddit is a good contrast, since it's taking scraping to court with the Perplexity case and the judge let most of it go forward. With this dataset I haven't seen anything like that. It's just there on HuggingFace with a free download link, and I haven't found anything on the web that TikTok is doing anything about it. Maybe that will soon change.


r/ProxyEngineering • • Jun 23 '26

Hot Take šŸ”„ so I accidentally learned how the entire proxy industry works, please tell me I'm not the only one

124 Upvotes

Started because I was overthinking this far too long and ended up reading about how free streaming apps on Samsung and LG TVs are literally enrolling your home IP into a residential proxy network while you watch Netflix or Prime on the other input. Like you click "agree" on some vague popup about "device resources" and your TV starts routing web traffic for whoever's paying for residential IPs that month. Up to 200GB or more, who knows?? All that while you're asleep. On your $2000 TV. Yall are fucking nuts if you think this is normal. And apparently that's just a thing that's happening and we should be all cool with it?? Boy whattahelly

Anyways, I went looking for who actually buys this traffic and why residential IPs are worth so much. And that's where it gets kinda funny. Turns out a huge chunk of the demand is scrapers trying to get past bot detection. Which, sure, makes sense, residential IP looks like a real human etc. Except I kept reading and finding that whole premise is like three years out of date. Cloudflare, Akamai, DataDome, the holy trinity. They're not primarily checking your IP anymore. They're fingerprinting the TLS handshake, the HTTP/2 frame ordering, your WebGL renderer string, how your canvas noise changes across sessions, whether your "mobile" session has gyroscope data. Your IP can be spotless and your Python requests library is still announcing itself on the first packet. Someone here on this sub literally tested this, datacenter IPs with proper TLS impersonation outperformed residential IPs with bare requests headers like 91% vs 60% on Cloudflare-protected site?? whaat?

So your TV is being used as a proxy node. The proxies are being sold to scrapers. The scrapers are getting detected anyway because the fingerprint gives them away. The TV thing was pointless to begin with basically. Am I looking to deep into this? Coz I feel like this shouldn't be normal, I mean the whole ordeal, not me looking into it. And then separately, completely disconnected from all this ordeal, some guy on Reddit figured out Spotify serves authenticated AAC files via CDN links that stay valid for 2 hours. Bro is going to get a DMCA notice fast if not already got it. What I'm trying to say is that Spotify already went through this exact thing. The tools are already built and already taken down. What else are we gonna have to deal with? And I'm talking about a small percentage of people that actually care, while the majority just clicks through all these popups without questioning a single thing.

Anyway I blocked proxyjs.brdtnet.com on my router, learned what JA3 fingerprinting is, and now I'm questioning every "agree" popup I've ever clicked. Normal tuesday, folks


r/ProxyEngineering • • Mar 18 '26 Regret Mindblown

Proxies are literally useless for anonymity and I'm tired of pretending they're not

83 Upvotes

everyone on reddit acts like buying some sketchy proxy service makes them edward snowden or something lmao. Your isp can still see you connecting to the proxy. the proxy company is 100% logging your shit regardless of what their website says. and when the cops show up with a warrant guess whos getting thrown under the bus. "bUt rEsiDeNtiAl pRoXiEs", yeah bro you're routing your traffic through someones compromised device, real cool. totally not helping a botnet or anything. Don't eveb get ne started on socks5, they are not even encrypted unless you set it up with ssh which like 90% of you don't even know how to do. and vpns are the same scam with better marketing. they all keep logs no matter what they claim. The only legit way to stay anonymous is tor and even that gets pwned by three letter agencies on the regular. everything else is security theater for people who watched mr robot once. If you're doing anything actually illegal (in minecraft obviously) and you're relying on nordvpn or some $8 proxy you're gonna have a bad time

anyway downvote me idc im right


r/ProxyEngineering • • Mar 25 '26

Hot Take šŸ”„ These big tech companies really out here acting like they own the entire internet and I'm about to break

79 Upvotes

Aight so no cap, this is actually wild when you think about it. All these massive corpos like OpenAI, Google, Meta, etc deadass scraped the ENTIRE internet to train their AI models and make billions of dollars, right? They yoink everyone's blog posts, art, code, writing, whatever, didn't ask for no damn permission, didn't pay a single soul, and now they're sitting on these absolutely unhinged valuations. But the SECOND some indie dev or enthusiast researcher from India or Pakistan wants to scrape publicly available data for their project, suddenly it's all "oh nooo our precious ToS violated" and "you're literally stealing our data" and they'll sue you into the shadow realm or IP ban your whole existence. Like brother... you LITERALLY built your whole business model on scraping other people's stuff without permission but now YOU'RE the victim?? The math ain't mathing fellas. Reddit really said "let's sell all our users' content to AI companies for mad stacks" then immediately locked down their API so regular people can't touch it anymore. Twitter same energy, Elon really acting like public tweets or exes, haha lol exes, whatever you wanna call them, are some kind of proprietary asset now lmaooo. LinkedIn out here going after people for scraping publicly visible profiles that USERS CHOSE TO MAKE PUBLIC like sir??? The hypocrisy is sending me. If scraping is theft, then these companies are lowkey the biggest thieves in human history. But nah they got whole legal teams and lobbyists so suddenly it's "fair use" and "innovation" when THEY do it. When you do it? Straight to jail apparently. Either the public web is public or it's not. You don't get to have it both ways just cuz you're worth billions and got lawyers on speed dial.

Rant over but y'all KNOW I'm spitting facts


r/ProxyEngineering • • May 20 '26

Excel can scrape websites directly and dump the data into your sheet??

63 Upvotes

I had to share it with you all. Every week I'd manually open a supplier's pricing page, copy the table, paste it into Excel, fix the formatting that always broke, delete the garbage rows, and repeat for 24 different sites. Easily around 2 hours every Friday. I just assumed that's how it worked. Nobody told me there's another way. Go figure. Turns out VBA can open a web page, parse the HTML, and pull exactly what you need, no browser, no copy-paste, no cleanup:

Sub ScrapePrices()
Dim http As Object, html As Object
Dim rows As Object, i As Integer
Set http = CreateObject("MSXML2.XMLHTTP")
http.Open "GET", "https://example-supplier.com/prices", False
http.Send
Set html = CreateObject("HTMLFile")
html.body.innerHTML = http.responseText
Set rows = html.getElementsByClassName("price-row")
For i = 0 To rows.Length - 1
Cells(i + 1, 1) = rows(i).innerText
Next i
End Sub

It hits the URL, grabs the HTML, finds every element with the classĀ price-row, and writes each one into your sheet. The whole thing runs in under 15 seconds. What used to ruin my Friday afternoons now happens while I get coffee and chat with colleagues without feeling anxious of the remaining work that has to be done. Also, you do need to peek at the page source (F12 in your browser) to find the right class or tag to target, but that takes about 2 minutes once you know what you're looking for. Works on any site that doesn't require a login or JavaScript to load the content. If you've been manually copy-pasting from the same websites over and over, you're probably one macro away from never doing it again.

TLDR: The code uses MSXML2.XMLHTTP and HTMLFile to fetch and parse a webpage.

If y'all have further improvements, let me know, I'm fairly new to the automation world, but boy do I love it already


r/ProxyEngineering • • Apr 17 '26

Hot Take šŸ”„ Headless browsers are destroying the open web and I'm tired of pretending they're not

57 Upvotes

Listen yall, I know I'm will get roasted for this and frankly I do not care, I love engaging in heated discussions and spark them, so here's my take:

Headless browsers like Puppeteer and Playwright are single-handedly ruining the internet for everyone, and developers using them for scraping are just entitled thieves with extra steps. One may say; "But I need the data for my startup" Cool story. Plenty of these in r/Entrepreneur, r/startups That data cost someone real money to generate, host, and serve. You bypassing their API limits and ToS because you don't want to pay for proper access is literally no different than walking into a store and stealing inventory because "the prices are too high." Again, you may say: "It's publicly available information" well. so is the Mona Lisa, but you can't just cut it out of the frame because it's "publicly visible., I mean R. Atkinson once did it in Mr Bean but that's another story. And before the "but Google does it" fellers shows up, yes, Google scrapes. Google also generates massive traffic back to your site and has negotiated agreements with publishers. Your startup that's extracting all the value while giving nothing back is NOT the same thing. What I hate the most is that every scraper I've ever met acts like they're some kind of digital Robin Hood. Like bruh, are you for real. You're just creating an arms race of increasingly aggressive anti-bot measures that make the internet worse for actual humans with screenreaders, slow connections, and privacy tools. Tools like datadome, akamai, imperva, recaptcha exists because of yall. Headless browsers have legitimate uses (testing, automation of YOUR OWN stuff), but majority of the tutorials and use cases are just "here's how to steal content and pretend it's ethical."

Rant done, go on, change my mind in a sentence or two


r/ProxyEngineering • • Jul 03 '26

Hot Take šŸ”„ The NetNut FBI seizure raises a question nobody asks: where do residential IPs actually come from?

56 Upvotes

I went down the rabbit hole reading about the NetNut situation last night and honestly, the FBI seizure wasn't even the craziest part for me lol

The craziest part was realizing that somewhere out there, a guy is probably watching Netflix on his Samsung TV while his TV is simultaneously acting as infrastructure for somebody running a scraping operation on the other side of the world (I have a really old TV that and I don't watch it that much honestly, so I think I avoided this bullet)

LIKE WHAT? That's not even a joke anymore

According to the news coverage happening at the moment, researchers found proxy SDKs embedded in a huge number of smart TV applications. Not sketchy APKs from some random forum. actual apps running on LG and Samsung TVs

I think that in this case, the conversation quickly shifted from NetNut to a much bigger question that I don't think someo people don't think about:

Where do residential IPs actually come from???

Don't get me wrong, I don't think about this question either that much and it's a really complex answer to a complex question

The sourcing side always felt like somebody else's problem honestly

But after reading about the alleged connection between NetNut and the Popa botnet, I started realizing how little visibility most of us actually have into the supply chain behind residential proxies

What surprised me most wasn't the claim that millions of devices were involved. It's 2026. Every month there's another story involving millions of compromised devices, so unfortunately that part barely registers anymore

The surprise part for me happened when learning how many layers can exist between the person buying a residential proxy and the device providing that residential IP (like wth)

The part I keep coming back to is whether this is even a problem that can be solved completely

If residential proxy inventory passes through multiple layers of aggregators, partners, SDK providers and resellers, how many companies can honestly say they know the origin story of every IP in their network

At what point does a provider stop being a network operator and become a network consumer just like the rest of us?

What struck me while reading all this is that the proxy industry spends an enormous amount of time talking about performance metrics. We compare success rates, country coverage, session length, pool sizes, pricing, uptime and all the usual stuff. Yet I can barely remeber seeing a serious discussion about where these networks actually come from. Maybe that's because the answer is complicated, but after reading through the NetNut reporting it suddenly feels like one of the most important questions we could be asking.

SO very few people seem interested in tracing the supply chain behind the product itself, which is funny because that's ultimately the foundation everything else is built on

One thing that comes to mind in this situation is that some providers seem to be putting more effort into transparency than others and I've seen that happen pretty recently. I've seen providers like nodemaven, Iproyal or proxygonzo openly talk about IP quality filtering, network quality standards, and support processes. Others have started publishing more information about sourcing, partnerships, compliance policies, or how they acquire residential inventory in the first place. This should probably be the standard moving forward for everyone that's affected in this case

I'm not saying anyone deserves a free pass, and honestly this whole story makes me want to be more skeptical rather than less. But I do think there's a meaningful difference between providers that are willing to discuss where inventory comes from and providers that treat the entire supply chain as a black box.

Maybe that's where the industry needs to go next. We already compare success rates, uptime, sticky sessions and pool sizes. Maybe a few years from now we'll also be comparing transparency reports, sourcing disclosures, consent models and network provenance and being sceptical of each proxy provider?

Sources where I found this story: https://hivesecurity.gitlab.io/blog/netnut-popa-botnet-fbi-seizure-residential-proxy/

Also huge props to this user as he predicted the future lmao - https://www.reddit.com/r/ProxyEngineering/comments/1u00w9f/my_samsung_tv_is_literally_being_rented_out_as_a/


r/ProxyEngineering • • Jun 09 '26 Galaxy Brain All Seeing Upvote Sad WOW

Best antidetect browser + proxies combos (guide on personal experience)

52 Upvotes

Hey all, I wrote this post thinking it could be really valuable who do not know about the best antidetect browser + proxy pairing, so this might be able to help you out, I have been quite extensively working on some really big multi-accounting workflows, so I hope this helps you out or at least gives you some valuable advice

I juggles between a lot of social media platforms and have a lot of ad accounts so if anyone is in the similar space as me, I am happy to help

What i mean by multi accounting is that I’ve gone through workflows such as:

  • running approx hundred of browser profiles

  • managing social media accounts that i managed to keep stable for a long time

  • scaling automation workflows using these accounts

  • testing GEO-sensitive setups using good quality IPs from various countries

  • warming accounts that need to stay alive

  • browser automation

  • scraping social platforms

  • maintaining session trust over long periods

And after A LOT of testing, I made one big conclusion which I think some people don’t think about

People massively underestimate how much the browser + proxy pairing affects stability

You might think about these things first:

ā€œWhat’s the best antidetect browser?ā€
ā€œWhat’s the best proxy provider?ā€

And I think that’s not really the right quesiton, I think the right question

Which combination breaks the least under your actual workflow?

Because once you scale browser-based automation, tiny problems start becoming big problems

  • session invalidation

  • repeated email verification

  • suspicious login prompts

  • failed warmups

  • dead cookies

  • browser memory leaks

  • profile corruption

  • unstable automation flows

I started testing different combinations specifically to figure out:

  • which setups survive longer

  • which setups feel stable after weeks

  • which ones create hidden maintenance problems

  • which ones scale cleanly

How I tested everything and each proider

My testing was real and detailed

I wasn’t running ā€œspeed testsā€ or checking homepage load times as i tested these setups in actual multi-accounting environments

Main workflows I tested

  • TikTok account warmups

  • Facebook account management

  • Reddit automation

  • X/Twitter workflows

  • browser scraping

  • session persistence testing

  • automation through Playwright/Puppeteer

  • anti-detect browser automation

  • long-running login sessions

  • GEO switching

  • cookie persistence

What were my success metrics for each provider

I mainly tracked:

  • session longevity

  • account survival rate

  • random re-verification frequency

  • IP consistency

  • browser stability over time

  • memory usage

  • automation interruption frequency

  • login invalidation rate

  • CAPTCHA frequency

  • profile corruption issues

Testing duration

Most combinations were tested for:

  • 2–6 weeks

  • multiple browser profiles

  • repeated automation cycles

  • varying concurrency levels

Because if you think about it almost every setup looks ā€œgoodā€ during the first 24 hours

The real problems usually appear later in the workflow and automation aspect

Here's my list

1. NodeMaven + Multilogin

This was probably the cleanest long-session setup I tested because it lasted the longest without any issues

And what was refreshing for me is that this combo solved one of the biggest problems I had which is session instability during long-lived browser workflows (SO CRUCIAL FOR THESE SETUPS) and it was so refreshing to have this

I mainly tested this combo for:

  • TikTok warmups

  • Reddit account management

  • persistent social sessions

  • browser automation with long login retention

  • affiliate account management

The problem I had before with previous providers

With many rotating residential providers, I kept seeing:

  • random session invalidation

  • cookie distrust

  • weird location inconsistencies

  • browser sessions suddenly behaving differently

  • accounts getting logged out unexpectedly

So why is their combo good?

NodeMaven’s sticky residential + ISP sessions paired really naturally with Multilogin’s browser persistence and they seem to have some sort of collaboraiton so their pairing works REALLY well together. Also what I liked is that nodemaven now providers some sort of quality guarantee, so if some of their IPs are not working well, they will give you back the money and I have experienced only one IP not working properly

Their setup actually solved a lot of problems for me which were:

  • mid-session IP inconsistency

  • session resets

  • account distrust

  • login interruptions

What stood out most is that the sessions simply behaved predictably and i could just keep the automation going

Technical observations that might interest you as well

  • Lower CAPTCHA trigger rate

  • Better cookie retention

  • More stable long-running sessions

  • Fewer browser trust resets

  • Better automation continuity over multiple days

Weaknesses

The setup is not optimized for:

  • hyper-aggressive rotation

  • ultra-cheap scaling

  • high-churn scraping

This pairing clearly favors:

  • quality

  • stability

  • persistence

over raw volume and scraping large volume of data

2. Dolphin{anty} + Decodo

This setup felt very different operationally

IT was Much faster and muuch more rotation-oriented

I tested this combo mostly for:

  • rapid account cycling

  • Facebook setups

  • TikTok profile scaling

  • GEO testing

  • account segmentation

What worked well

Dolphin’s workflow style naturally favors:

  • quick profile management

  • high operational speed

  • fast deployment

And Decodo’s rotating residential pools paired well with that

This setup worked especially well when:

  • profiles didn’t need ultra-long persistence

  • accounts rotated frequently

  • GEO flexibility mattered

  • account quantities increased quickly

Problems it solved

Compared to stickier setups:

  • easier traffic separation

  • easier profile scaling

  • faster environment resets

  • simpler operational management

Issues I still encountered

Under longer sessions, I occasionally noticed:

  • more re-verification

  • shorter ā€œtrust lifespanā€

  • more session inconsistency over time

This setup felt really optimized for speed and scale, but it might just be my use case

3. AdsPower + Webshare

This was one of the most budget-efficient setups I tested. Probably the cheapeast option you can find but this can possibly break your workflow if it runs for too long

I used this combo for

  • lightweight account farming

  • testing automation flows

  • burner workflows

  • lower-risk scraping

  • mass profile creation

What surprised me

AdsPower tolerated lower-quality proxies better than I expected

Some antidetect browsers become extremely unstable once proxy quality drops slightly

AdsPower stayed relatively usable even with:

  • cheaper residential pools

  • mixed IP quality

  • higher profile quantities

Problems I encountered

Webshare is actually quite known for having the cheapest proxies possible on the market,g but their proxies can get quite bad during longer-period use, which can be a problem if you need to have a stable working flow of many accounts

So critical issues with this setup can be:

  • longer session persistence

  • trust-sensitive platforms

  • older account management

I started seeing:

  • more login challenges

  • more verification prompts

  • weaker session trust over time

BUT, if you are on budget, and just starting out, or a freelancer who doesn't simple have the money for, this is a really good combo for:

  • testing

  • scaling cheaply

  • operational experimentation

+ and the setup honestly performed better than expected

4. Octobrowser + MarsProxies

This setup felt very natural and I was able to use it for quite a long time actually

I used them for managing multiple accounts and was able to run sessions with 15-20 accounts with ease in one go

This combo also works well with:

  • browser automation at higher concurrency

  • regional account testing

What stood out

MarsPorixes routing flexibility was probably the biggest strength because they are a bigger provider and were able to provide me with good stability

Especially I think thy have good:

  • GEO precision

  • ASN targeting

  • traffic routing consistency

  • large-scale reliability

Problems it solved

This setup significantly reduced:

  • GEO mismatches

  • localization inconsistencies

  • unstable routing behavior

Especially useful when:

  • testing regional content

  • scraping localized SERPs

  • running region-sensitive workflows

Downsides

Honestly this setup doesn't have many downsides, I didn't experience a lot of problems but like a few random proxy kills and banned IPs so for my workflow this worked like a charm

And for smaller operators, it’s definitely an overkill

5. Undetectable + Oxylabs

This setup performed extremely well under heavier automation loads as I believe oxylabs proxies are also built for that in some way. This is basically a setup for enterprise users mostly? (because I don't see any regular users or people needing tihs in really big scale)

I used this combo for:

  • browser scraping

  • higher concurrency automation

  • Playwright integrations

  • larger scraping sessions

What I liked

The setup stayed surprisingly stable under load (which is expected out of an enteprrise proxy provider any basically any antidetect browser combo)

Some combinations start degrading badly once:

  • concurrency increases

  • memory usage grows

  • browser workers scale

This one remained relatively clean operationally

Problems it solved

  • worker instability

  • session crashes

  • automation interruption

  • browser lag under scale

Technical observations

  • Good browser stability

  • Lower crash frequency

  • Strong scaling consistency

  • Better concurrency handling

6. Incogniton + IPRoyal

This felt like the best beginner-friendly pairing because iproyal has cheaper prices, and incognitioin is quite intuitive as a browser

I used this combination for:

  • smaller affiliate workflows

  • basic browser automation

  • moderate account quantities

  • testing ideas quickly

Why I liked it

Simple to setup, it’s actually really cheap and it’s low-friction

Not every workflow needs enterprise infrastructure so the combination between incognition + iproyal gives you this

Sometimes you just need:

  • acceptable stability

  • decent residential IPs

  • manageable costs

  • enough flexibility to test ideas

And honestly, this setup delivered that pretty well

What does each setup realistically cost per month?

SetupBrowser costProxy costApprox. monthly budgetbest use case
Multilogin + NodeMavenMultilogin Pro (10 profiles) ~$11/moNodeMaven 20GB plan ~$75/mo~$86/moLong-term account management
Dolphin Anty + Decodo10$ per month (for 10 profiles)81.25$ for 20GB~$60–120/moFast account scaling
AdsPower + Webshare9.6$ per month (for 20 profiles)27$ per month (for 1000 proxies)~$20–50/moBudget testing
Octobrowser + MarsProxies9$ per month (for 20 profiles)75$ for 20GB$90+/moGood for multi accounting as well as the NodeMaven workflow
Undetectable + Oxylabs11$ per month (for 20 profiles)100$ per month$110+/moEnterprise workflows and scraping
Incogniton + IPRoyal13.99$ for 10 profiles52.5$ for 10GB~$30–60/moBeginners

r/ProxyEngineering • • May 11 '26

Stop throwing residential proxies at everything, your fingerprint is the actual problem

47 Upvotes

Aight, listen up, Imma keep it real with you. I know this is going to rub some people the wrong way, but I've been doing this long enough to feel confident saying it, most of you don't have a proxy problem, you have a fingerprint problem, and you're spending $200+/month on residential bandwidth to brute-force your way around it. I get it. Residential proxies feel like the safe default. The IP looks clean (for those who are checking by these fraud scores), it passes basic geo checks, and every provider markets them like they're the golden ticket. if your TLS fingerprint screams "Python requests library" or your browser automation is leaking navigator properties that no real Chrome session would ever have, it genuinely does not matter how pristine your IP is. Cloudflare, Akamai, DataDome, they all fingerprint the client now, not just the address. A burnt datacenter IP with a properly spoofed JA3 hash and realistic header order will outperform a fresh residential IP attached to a naked requests.get() call nine times out of ten. I ran a test a few weeks ago, across about 15 mid-sized e-commerce sites protected by Cloudflare. Datacenter proxies with curl-impersonate had a ~91% success rate. Residential proxies with default Python requests headers? Around 60%. The residential IPs were objectively "better" IPs, they just didn't matter because the request itself was the red flag. I think the proxy industry benefits from people not understanding this. The less you know about TLS fingerprinting and HTTP/2 header frames, the more bandwidth you burn through rotating IPs trying to find one that "works." That churn is literally their revenue model. Before you upgrade your proxy plan, spend an afternoon with curl-impersonate or look into how got-scraping handles fingerprint randomization. Learn what JA3 and JA4 fingerprints actually are and how to check yours against real browser signatures. You might find that the $30/month datacenter plan you dismissed does the job just fine once your client stops identifying itself as a bot on the first handshake. Now, I'm not saying residential proxies are useless, for account management, social media automation, and anything session-heavy where the IP itself gets scored over time, they're still the right call. But for scraping? Fix your fingerprint first. Then decide if you actually need the expensive IPs


r/ProxyEngineering • • May 05 '26

If you're not automating in 2026 you're literally working for free

46 Upvotes

Hey folks. Everyday, more and more posts appear talking about manually rotating proxies, manually checking if IPs are burned, manually switching geos. Brother WHAT are you doing. I used to spend like 4 hours a day babysitting my scrapers. Checking logs, restarting sessions, swapping out flagged IPs one by one like some kind of digital janitor. Then I actually sat down and automated the whole thing and now I check on it maybe twice a week. The setup isn't even complicated either. Residential proxy with auto rotation, a simple health check script that pings every 10 minutes, and a fallback pool if success rate drops below 80%. That's it. That's the whole system. Sure you gotta count on the provider's proxies for everything to work, but then again, you can use a few providers and diverse the pool to other ones in case the primary proxies fail. And before someone says "but my use case is different" no it's not. If your running scrapers or managing accounts and your still doing stuff by hand your leaving money on the table. Actual money. I calculated it once and I was basically paying myself $4/hr to do work a cron job could handle. The residential proxy part is important too btw. I tried automating with datacenter IPs first and it was a nightmare. Half my automations would break within hours because everything got flagged. Switched to residential and suddenly my scripts actually ran longer than a single afternoon without catching bans. Not saying that dc proxies are useless either, it just worked for me. I've seen some people using resi + dc combo all the time and it's solid, sure if you know what you're doing and not targeting specific hard targets like Amazon, zillow, or whatever. The world is changing people. Automate it, set up alerts, and go touch grass. Your scrapers don't need you hovering over them 24/7. And if it feels like you are stuck in one place, there are many already proven researches here and github


r/ProxyEngineering • • May 21 '26 All Seeing Upvote Diamonds are Forever

I barely knew Python, now I build scraping infrastructures with ease

38 Upvotes

Over the last year I started experimenting heavily with Cursor, Claude and GPT for coding

Not because I wanted to become a ā€œreal developer,ā€ but because I kept running into scraping problems where existing tools got me 80% moving forward when I was stuck with some scraping projects

And honestly, the biggest realization for me was this:

AI coding is insanely useful for people who understand scraping workflows better than programming itself

Most of what I do is SERP/data scraping, browser automation and scraping platforms thta aggressively rate limit or fingerprint traffic

Also I wouldn’t call myself a proper Python developer. My knowledge is actually really basic

But with AI-assisted coding I’ve managed to build:

  • proxy rotation systems

  • local APIs

  • scraping helpers

  • queue-based workers

  • Telegram monitoring bots

  • browser automation utilities

  • retry/failure handling

  • image processing tools

  • account management workflows

What surprised me most is that the hard part usually isn’t the code itself.

It’s:

  • structuring the architecture properly

  • handling retries

  • avoiding race conditions

  • making sessions stable

  • organizing proxies cleanly

  • preventing spaghetti code

  • figuring out where bottlenecks actually happen

That’s where AI became really helpful for me case

For example, instead of prompting:

ā€œmake me a scraperā€

I started prompting like:

ā€œBuild a modular Python scraping system with rotating residential proxies, retry logic, queue workers, FastAPI endpoints, logging, rate limiting and browser session management.ā€

The difference in output quality is massive

One workflow I’ve been using a lot lately:

  • FastAPI running locally

  • scraping workers communicating through localhost

  • rotating residential/mobile proxies

  • anti-detect browsers for harder targets

  • Telegram bots for monitoring/control

  • queue systems handling retries automatically

Honestly, AI feels especially powerful for writing ā€œglue infrastructure.ā€

Things like:

  • proxy rotation layers

  • retry logic

  • task queues

  • logging systems

  • browser wrappers

  • local APIs

  • automation scripts

  • monitoring tools

A lot of this used to feel inaccessible unless you were a serious engineer

Now solo scrapers can prototype infrastructure ridiculously fast

I’ve also noticed stable sticky sessions matter WAY more than raw proxy speed once you scale browser-based scraping or account workflows. Been testing some setups with a few proxy providers like nodemaven and brightdata recently for that reason because cleaner residential sessions + an antidetect browsers reduce random failures quite a bit actually

Biggest lesson so far: don’t try to build perfect systems immediately as this will not work, at least it didn't work for me

Now I mostly:

  1. Build minimum working infrastructure

  2. Stress test it

  3. Fix bottlenecks as they appear

Way faster than overengineering from the start

Curious if other scrapers here are using AI coding tools heavily now for infrastructure/tooling?

Feels like the barrier to building custom scraping systems dropped massively in the last year.


r/ProxyEngineering • • Aug 08 '26

Discussion šŸ’¬ FBI and Google disrupted a residential proxy network that had silently enrolled millions of consumer devices as traffic relays without owners' consent

36 Upvotes

In July 2026, the FBI, Google's Threat Intelligence Group, and additional partners executed a coordinated takedown of a major residential proxy network that had compromised an estimated 2 million consumer devices worldwide. The FBI seized hundreds of associated domains, and Google pushed updates to automatically block applications containing the network's proxy SDKs.

This was the second such disruption in 2026, following a similar action in January against another residential proxy provider.

---

## What residential proxy networks do

Residential proxy services route internet traffic through real household IP addresses instead of data centers, making the traffic appear to originate from a legitimate home user. This is valuable to data scrapers, ad fraud operators, credential stuffing crews, and state-sponsored espionage groups because it bypasses anti-bot detections, rate limits, and geo-restrictions.

The privacy issue is how these networks acquire their pool of residential IPs. The disrupted network built its pool by distributing SDKs (software development kits) embedded in apps and firmware for devices commonly found in homes — smart TVs, streaming boxes, Android phones. These SDKs turned those devices into exit nodes, routing third-party traffic through the device owner's IP address, without meaningful consent.

---

## Two primary infection vectors

**1. Pre-installed malware on cheap, off-brand Android TV boxes**

Low-cost streaming devices manufactured overseas have been documented arriving with proxy firmware baked in from the factory. One related botnet operation, disrupted in 2025, was found to have compromised over 10 million devices this way.

**2. Trojanized apps with embedded proxy SDKs**

Free apps — particularly free VPNs and apps marketed as "passive income" tools for sharing bandwidth — bundle SDKs that silently enroll devices as exit nodes. Research found that over 42% of apps on some smart TV app stores contained proxy SDKs.

---

## The scope

In a single week in June 2026, Google observed **316 distinct threat clusters** using the network's exit nodes, including cybercriminal and state-sponsored espionage groups. Documented activity included:

- Password-spray attacks
- Mass content scraping
- Advertising fraud
- Account takeovers
- Credential stuffing
- Data harvesting for AI model training

A separate investigation by a media foundation traced a scraping campaign in May 2026 that scattered across more than **1.4 million IP addresses**, targeting public-interest journalism sites. The traffic was routed through compromised home devices that had no idea they were participating.

Google also assessed with "high confidence" that many popular residential proxy brands were whitelabeling this network — meaning companies that appeared to be independent proxy services were actually drawing from the same pool of hijacked home devices.

---

## Why this is a privacy issue

When your device becomes an exit node, strangers' internet traffic passes through your home network and is associated with your IP address. This means:

- **Your IP address can be linked to criminal activity** you had nothing to do with
- **Unauthorized traffic passes through your local network**, potentially exposing other devices on the same connection
- **Your network identity is being commodified and sold** to third parties without your knowledge or consent
- **The device manufacturer or app developer made a decision about your data and network resources** that you were never informed of

The FBI issued a public service announcement in March 2026 warning that once a device is compromised, its IP address can be used by threat actors to mask illegal activity, making the consumer appear responsible.

---

## The structural problem

Google acknowledged that when one proxy network is disrupted, operators simply purchase capacity from competing networks, effectively becoming resellers. They observed this behavior after the January 2026 disruption. The residential proxy market is interconnected through whitelabeling agreements, and the AI industry's massive demand for web scraping has created financial incentives that will keep this ecosystem alive regardless of individual law enforcement actions.

---

## What you can do

- **Audit your streaming devices.** Off-brand Android TV boxes and devices marketed as unlocking free streaming are the highest risk category. The malware in these devices is baked into the firmware at manufacture — no software-side fix can fully remediate a device compromised at the factory.
- **Remove free VPN apps** and any apps you don't recognize or use, especially on smart TVs.
- **Run an open-source DNS sinkhole** on your network to log and monitor outbound DNS requests from all connected devices. This lets you see which devices are communicating with suspicious domains and block known malicious infrastructure.
- **Segment IoT and streaming devices** onto a separate network from your primary devices. Most routers support at least a guest WiFi network.
- **Change default router credentials** and ensure WPA2 or WPA3 encryption.
- **Keep all firmware updated**, and only purchase streaming devices from manufacturers that provide regular security updates.

---

## Sources

- Google Threat Intelligence Group blog post on the disruption
- FBI Public Service Announcement I-031226-PSA (March 2026) on residential proxy networks
- KrebsOnSecurity reporting on the botnet linkage
- SecurityWeek, Reuters, and Infosecurity Magazine coverage of the takedown
- Qurium Media Foundation investigation into the scraping campaign
- Synthient and Spur proxy tracking reports


r/ProxyEngineering • • Jul 03 '26

Discussion šŸ’¬ First BrightData and now NetNut? What's happening?

36 Upvotes

Ok so this is gonna be a bit of a ramble but I need y'all to hear this. I work adjacent to the scraping/data space (not naming employer, y'all know how it is) and I always kind of just you know, accepted that "residential proxies" were this slightly grey but mostly fine thing. Like yeah obviously somebody's home IP is being used, somebody agreed to something somewhere, moving on. Then yesterday I see netnut is just gone. Not down, not maintenance page, straight up FBI DOJ IRS-CI seizure banner, Google and Lumen and Shadowserver all stamped on it too like it's a whole coalition operation. Like bruh, I have never in my life seen this on any website. So imagine the shocker lol. And I'm sitting there like wait since when did the IRS care about proxy IPs??? Turns out IRS-CI does financial crime investigations and it could be related to that, but still, seeing that logo on a proxy provider's main website??? Diabolical mate.

So I go searching what's going on and apparently it's tied to this thing called the Popa botnet, basically it's been running for like four years hijacking Android TV boxes (remember that Bright Data SDK thingy?? And Krebs traced a chunk of it back to actual NetNut infrastructure through some ex-employee's domain. Not some randoms, an actual former VP of R&D there. Google's own blog post says they think the network was over 2 million devices at one point and that a lot of those "different" proxy brands you see around are just NetNut wearing a different logo through resellers. I swear I seen a post talking about how there is one or two proxy providers sharing those IPs/reselling whatever. Makes sense now, people?? Am I the only one concerned here with a few other nerds? And to be fair I want to be fair here, Alarum (the parent company) came out and started calling it that a botnet is inaccurate, also said that there's real KYC and consent flows and misuse detection on their end. So it's not like this is some minor thing everyone agrees on, it's actively disputed and I think that matters, I'm not trying to just slam a company because Krebs wrote a scary headline, no. Make what you will out of it, but I things are rising to the surface, more and more often. Again it does make me think about how much of the "residential proxy pool" any of us are touching day to day is actually consented in a way a normal person would recognize as consent, versus consented in the "buried in paragraph 14 of an SDK terms screen" way. Like where's the line between legit residential network and just a nicer branded botnet, and how would any of us even know the difference from the outside.

Not trying to start a witch hunt on the whole industry, genuinely just spent my evening reading legal filings and threat intel blogs instead of sleeping like a normal person, but after Bright Data SDK shennanigans and now Netnut, boy, there's gonna be more stuff coming out. So I figured I'd share since I know a bunch of you here actually work with this infra daily and probably have opinions on these matters.

Sources if anyone wants to go check themselves.

Krebs on Security, the original Popa botnet reporting: https://krebsonsecurity.com/2026/06/popa-botnet-linked-to-publicly-traded-israeli-firm/

Google's threat intel writeup on the disruption: https://cloud.google.com/blog/topics/threat-intelligence/google-continued-disruption-residential-proxy-networks

Alarum's official response: https://alarum.io/alarum-technologies-responds-to-inquiry-into-residential-proxy-networks/

And divinetworks.com itself, which is also showing the seizure page now: https://divinetworks.com/

P.S Some time later I found that they mixed up the domains because .com is down, but not .io which is the main page of netnut.

EDIT:

It would seem that their main website was also seized - https://netnut.io/


r/ProxyEngineering • • Apr 30 '26

Anatomy of residential proxies: how I choose a pool for multi-accounting (without burning money)

35 Upvotes

I’ve been testing a bunch of proxy providers over the past few months for automation and multi-account setups, and honestly… most of what you see on landing pages doesn’t hold up in real usage.. This has happend to me to many times

You know the usual stuff that providers say in their ads:

ā€œ50M+ IPsā€, ā€œ99.9% uptimeā€, ā€œperfect for any use caseā€ and all that jazz (this of course isn't always the case)

Then you actually run traffic through them and things get a bit different from my experience

So instead of trusting marketing, I started focusing on a few things that actually matter:

First: pool quality > pool size
I’d take 1M clean IPs over 50M garbage ones any day.
If IPs are already flagged (Cloudflare, Spamhaus, etc.), it doesn’t matter how big the pool is. One bad IP can ruin an account.

Second: are they actually residential?
Some providers quietly mix datacenter IPs into ā€œresidentialā€ pools. DO NOT TRUST THESE PROVIDERS WHO OFFER LIKE A proxy for 0.49$ per GB - IT'S A SCAM! No proxy is actually this cheap
That’s basically a guaranteed red flag for most platforms

Third: how they behave under real load
Ping doesn’t mean much form my experience. I’ve seen proxies with great latency completely fall apart when a browser opens 20+ connections and tries to load a JS-heavy page

How I tested (nothing fancy)

I didn’t use any fancy dashboards, just simple scripts:

  • Light test (Axios) → check IP, GEO, ASN (to confirm it’s really residential)

  • Heavy test (Puppeteer) → load a fingerprinting site and see how long it actually takes to render

That second one is where most proxies start struggling.

What surprised me

  • Some providers looked decent in simple tests… then completely slowed down in real browser scenarios

  • Lower ping didn’t mean better performance at all

  • And yeah every single one still triggered WebDriver detection

That last one is important.
Proxies help with IP trust, but they don’t fix a bad browser fingerprint.

Quick ranking from what I tested

This is just my experience, not sponsored or anything:

1. NodeMaven
Probably the most consistent overall. Clean IPs and handled heavy pages way better than others. Felt like less babysitting was needed

2. Asocks
Solid budget option. Not amazing, but reliable enough if you’re scaling and watching costs

3. Floppydata
Works fine for lighter tasks. Starts struggling when things get heavier

4. Thordata
Could be useful for specific GEOs, but I wouldn’t rely on it for stable workloads

5. Bright Data
This one was frustrating. Expensive, requires KYC, and I ran into setup issues with their SSL handling. Didn’t feel worth the effort for my use case

Hope this helps


r/ProxyEngineering • • 8d ago

Guides Tiktok Signer Functions Fully Reversed (X-Argus,Ā X-Gorgon,Ā X-Ladon,Ā X-Khronos,Ā TTEncrypt,Ā TicketGuard) with library-ready Mobile API client

33 Upvotes

Hi, new to this subreddit. Had some spare time and decided to reverse engineer Tiktok's VM. This is, as of this message, up to date with the latest Tiktok (Nov 8 2026). Since I've made this public this should get patched pretty quickly. All signer math and key derivations / calculations are included in the README, formatted. All necessary headers have been reversed incl. X-Argus,Ā X-Gorgon,Ā X-Ladon,Ā X-Khronos,Ā TTEncrypt,Ā TicketGuard and some others. I probably won't be updating this again, so let it rip while you can.

If this gets enough attention, I will also publicly post the full lifted VM, including their ISA and how the entire VM works, as well as a source readable re-implementation. Happy scraping!

https://github.com/9xz/reversetok


r/ProxyEngineering • • May 18 '26

FireCrawl just hit 121k GitHub stars and I have a LOT of questions, the hype, the pricing trap, and what's actually going on

33 Upvotes

Okay, so I've been in the web scraping game for quite some time now. I was browsing the GitHub top-100 stars list yesterday and saw it sitting at #73 globally with over 120k stars. That's ahead of Node.js. That's in the same breath as projects that have been around for a decade. For context, at the end of 2024 they celebrated 20k stars. They raised their Series A in August 2025 at 43k stars. Now it's 120k+. That's roughly 3x growth in under a year, for what is essentially a web scraping API aimed at AI developers. What in the world happened? How did a scraping API beat Node.js in stars? The repo describes itself as "search, scrape, and clean the web for AI agents." Useful, I'd say. But 120k-star useful?? There are open-source alternatives like Crawl4AI with 65k stars doing very similar things for free. Is it just incredible timing with the AI/RAG pipeline wave, or is there genuine technical moat here that the community is rewarding? My main main concern is the star count organic? I'm not accusing anyone of anything, but a jump from ~20k to 120k in roughly 16 months is one of the most aggressive trajectories I've seen outside of projects with massive corporate backing (and I'm thinking of Microsoft's markitdown). FireCrawl got $14.5M Series A from Nexus and YC. Is any of that marketing spend showing up in developer mindshare as stars? I'm genuinely curious how you break into the GitHub top-100 that fast. Additionally, can someone explain the pricing to me without making my head hurt? On the surface it looks simple: 1 credit = 1 page scraped. But the moment you turn on anything useful, AI extraction, JSON output, Enhanced Mode, you're burning 5–9 credits per page. The Hobby plan at $16/month gives you 3,000 credits, which sounds great until you realize that's only ~333 pages with JSON + Enhanced Mode enabled. A 500-page website on the Hobby plan exceeds your entire monthly allowance in a single scrape. Now before someone says "just self-host it", that's an option, yes, it's AGPL-3.0 open source. But the self-hosted version is deliberately crippled: no Fire-Engine (their proprietary anti-bot system), no proxy rotation, no Actions endpoint, no browser sandbox. The stuff that actually makes it worth paying for is cloud-only. AGPL also means commercial self-hosting has licensing implications your legal team needs to look at and that's if you're within a company, if you're a an individual developer, well, that can get quite expensive. To be fair, the product genuinely seems excellent. Zapier, Shopify, Replit, and Apple are customers. The clean markdown output uses 67% fewer tokens than raw HTML. The MCP server integration means you can pipe live web data straight into Cursor or Claude. That's real value, and the community clearly feels it. But I keep coming back to the same question: is this one of the best-marketed developer tools of the AI era, or is it genuinely the best technical solution? Someone kindly explain what is going on with firecrawl


r/ProxyEngineering • • Jun 18 '26 Diamonds are Forever

Discussion šŸ’¬ Web Search API for AI Agents

30 Upvotes

Did you guys noticed how nobody talks about the search layer? From what I gathered, every AI agent tutorial obsesses over prompt engineering and tool variety, but the thing that determines whether your agent is useful is how it gets data from the web, rarely someone mentions or even provides decent information about it. Basically, my thoughts are that an AI agent is only as good as the information it can pull in. Simple as that. Best reasoning model in the world wont help if the search layer feeds it garbage or truncated snippets. Thats where a web scraper API comes in. Instead of scraping Google directly and fighting CAPTCHAs, layout changes, and anti-bot walls, you get a simple endpoint that returns structured JSON. Titles, URLs, snippets, sometimes full page content. Some providers also return knowledge panels, "people also ask" sections, even shopping results depending on the query. If you used any scraper, you know what I'm on about. Now for AI agents specifically, the change is that data comes back in a format the LLM can literally work with. No token budget wasted on HTML parsing. Agent sends a query, gets structured results, reasons over them, decides what to do next. No headless browser, no DIY scraping combos that stop working every two weeks or whatever. There are a bunch of providers right now. SerpAPI, Tavily, Exa, Firecrawl, Brave Search API, Serper, others. They take diffrent approaches. Some focus on raw SERP data exactly as it appears on the results page. Others are built specifically for LLM use cases, so they prioritize returning clean extracted content rather then just links or even better the whole info in markdown. That distinction matters alot depending on what the agent needs to do. Where I found it interesting is combining a search API with a content extraction. Agent searches, picks the most relevant results, then pulls full content from those pages in a structured format. That two-step workflow is way more reliable then trying to do everything in one shot. And its basically what tools like Perplexity do under the hood, except you get full control. Caching is worth thinking about too. Most of these APIs charge per request, and agents loop. A ReAct agent might send three or four searches before arriving at an answer. Without caching that adds up fast, both in cost and latency. A single search call taking 3-4 seconds is fine on its own. Chain a few together and the user is waiting 15 seconds or more. Which may sound like nothing, but the seconds adds up overtime. I strongly believe that this space is going to get way more competitive soon. As more people build agents that need real-time web access, demand for fast, accurate, affordable web scraping services is only going up. What you peeps think?


r/ProxyEngineering • • Sep 08 '26

Hot Take šŸ”„ LG recording audio through TVs

29 Upvotes

Take a seat because this is gonna be a bit of a vent. I watched the whole Gamers Nexus video today, this is still fresh and I remember previously there were some people really involved in these LG proxies/ botnet/SDK scandals going on, so guess I'm carrying the torch now lol.

For context, I work adjacent to the scraping/data space so networks quietly doing things they shouldn't is kind of my whole deal, and even I was surprised by how far this one goes. Steve and the Level1Techs crew worked with independent security researchers and packet captured retail LG OLED sets, the G5 among them, through Wireshark. What they found is the TV actively scanning the local network, not just for its own accessories, but for unrelated hardware that are on the same wifi. Phones, watches, tablets, PCs whatever's nearby. On top of the internal IPs it's also grabbing the names and signal strength of neighboring networks and folding in location data. All of that funnels into LG Ad Solutions, the ad arm of the company. LG's public sales figure is something like 216 million TVs sold globally, but their ad division is telling advertisers they can reach 363 million addressable secondary devices in the US alone. IMAGINE?? ACR isn't new, we've known for years that these sets fingerprint on screen audio and video to build a viewing profile regardless of input source. What's new here is the mic behavior. Bench testing showed the set capturing clean audio while the screen appeared powered down in standby. They then pulled the ethernet connection mid test and the TV kept recording locally anyway, and once the network came back it just uploaded whatever it had queued up. EVEN WITHOUT THE NETWORK, I just froze at that time when these things were mentioned during the video.

There's also apparently RCE in webOS currently working through responsible disclosure, which the team is understandably not detailing yet. Their stated recommendation was to keep the TV off the internet entirely and run an external streaming device instead, which is a wild thing to have to say about a product you paid TV money for. Especially Smart TV where majority of the people purchases for, well, smart features.

I am not trying to make this into some Samsung-vs-LG thing either, because there already was a scandal regarding Samsung and this feels like the same underlying issue the whole industry has apparently normalized, where the box in your living room works for someone else by default and opting out means adding hardware or cutting it off from the internet yourself.

P.S LG hasn't commented as of writing.

P.P.S I wrote it here mainly because this is all related to proxies and how the data is gathered/sourced.

Sources:

Gamers Nexus video: https://www.youtube.com/watch?v=6IFVTcM28KA

An article I found just hours ago (take it with a grain of salt, because the website seems sketchy): https://www.notebookcheck.net/LG-smart-TVs-caught-logging-audio-with-screen-off-and-snooping-on-local-devices.1391214.0.html


r/ProxyEngineering • • May 21 '26

Testing 8 different proxy providers

29 Upvotes

Hey folks. We ran a quick benchmark last week across 8 residential proxy providers for a price-tracking use case that we're working on. Sharing the numbers since most "comparisons" out there are clearly affiliate bait. Methodology (disclosing this upfront): 500 requests per provider, same rotating residential tier, tested against Cloudflare-protected targets, Amazon, Walmart included. Measured 200 OK rate and median response time. No free trials, paid out of pocket for each (business budget).

ProvidersSuccess RateMedian LatencyAdditional Comments
Oxylabs97.8%0.74sGood consistent geo coverage, few error codes
Decodo97.1%0.79sStrong value at mid-tier pricing
Bright Data96.6%0.83sPricing is crazy for smaller scale, but stable IPs
NetNut95.2%0.69sFast but more blocks on Walmart
SOAX94.7%0.95sGood for geo-targeting, slower in general
NodeMaven93.8%1.02sSurprised us, better than expected, we haven't heard of NodeMaven before, decided to try it out
Rayobyte91.3%1.14sDecent for low-tier targets
Webshare89.6%1.31sCheap, but comes with downsides, such as latency and success rate

Oxylabs takes the top spot on consistency and geo targeting. Decodo punches above its price. Bright Data's IP quality is still up there but the cost structure doesn't make sense unless you're pushing serious volume, mainly oriented towards Enterprises, which is understandable. If you're running under 50GB/month, skip Bright Data entirely, the per-GB cost at that volume doesn't add up. Unless of course, you'd like to test it out like we did.

Anyone run these against tougher targets like Ticketmaster or Zillow? Would like to understand how the performance shifts on all of these providers once you target something really "heavy".

Any other providers for suggestions to test them out? I know the main ones are the big players in the market, but sometimes "Mid-tier" providers works well too for decent pricing if you're not scaling. I guess it's also possible to just utilize dedicated scraping solutions, might be cheaper, and you don't have to worry about maintaining the IPs, as all of the infra is managed by the provider


r/ProxyEngineering • • Mar 19 '26

Why the Internet keeps asking if you're a robot?

28 Upvotes

You know what's annoying? When you just want to check something really quickly and your bombarded with captchas. I started to suspect that almost half of internet traffic is bots. Not people that actually contribute to the communities, forums etc. But bots. Some scrape websites for Google. Others buy up concert tickets, scrape prices, or spam the hell out of comment sections. Websites are basically getting hammered 24/7. Prime example was when we needed to purchase tickets to SOAD show in July. 2 mins in, all of the tickets were sold out. Like bruh, how quick can you be? I mean the sheer amount of clicks you have to do for banking app, then you get transferred to the main website where the purchase happening, it takes time. Why this happened? We suspected that there were ticket bots operating and bought all of the tickets. I understand that captchas help prevent botting, but god damn it is so annoying when you encounter one. Then again, you could have thought to use a VPN or proxy? Well, congrats, you now look suspicious. Websites can't tell if you're someone protecting their privacy or a bot hiding behind 1,000 different IP addresses. So they hit you with extra CAPTCHAs just in case. Yet again, another bombardment of captchas. I just wish there was another way for all of this.


r/ProxyEngineering • • Jul 31 '26

Hot Take šŸ”„ How to check if your TV box is secretly a botnet (before or after you buy one)

27 Upvotes

Krebs is serving another one. Saw this yesterday and I think that's a genuinely wild escalation of the whole TV box story we've been tracking since the Bright Data SDK and NetNut seizure. Bitsight researcher got his hands on an expired telemetry domain that thousands of H96 streaming sticks were phoning home to, and found something nobody expected. The boxes were reporting themselves as phones. Samsung, Vivo, Huawei, Xiaomi model numbers, coming from what is very clearly a $20 Android TV box you bought off Amazon, Aliexpress, or whatever.

Turns out there's a Chinese company behind it, Zhejiang Fengwo IoT, running an ad fraud operation where these boxes click ads on their own network of AI generated content sites, but only when the visiting device matches that spoofed mobile fingerprint. The part that got me though is the switching logic. When the box detects an HDMI signal (so the TV is actually on and being used) it runs as a residential proxy instead, renting your IP out without your knowledge. When the TV is off, it flips over to the ad fraud job. Basically the device is never idle, it's just deciding which crime to commit based on whether you're watching something. Kind of impressive from a resource management standpoint, if it weren't your electricity and IP reputation hogging the bill.

Given the last few posts here about Bright Data and Netnut, feels like a good time to put together a checklist instead of just doom scrolling about it. A few things that help if you're running any of this infra at home or auditing a network for a client.

Check if the box is Play Protect certified before you buy anything, Google has a lookup tool for this and it takes two minutes. If it's not certified, assume it ships with something extra installed. Synthient keeps a running csv on GitHub of known offenders, streaming boxes plus other consumer IoT stuff, it's worth bookmarking and checking before any purchase. If you're already running one of these boxes, put it on its own VLAN or at minimum a guest network so it can't see anything else on your LAN, these things have basically no authentication and get enslaved into botnets constantly, Synthient documented one called Kimwolf back in January that had millions of devices. Watch your outbound traffic for weird patterns tied to whether the TV is actually being used, that HDMI based switching is apparently a real signature now. And don't forget the digital photo frames either, FBI flagged those specifically as a category that ships with the same residential proxy junk.

None of this fixes the actual problem which is that the entire cheap streaming box market runs on this business model, but at least it's something concrete you can check instead of just hoping your $18 Amazon box isn't secretly working two jobs.

By the way, for those who said just turn off the internet from the TV, well most of the TVs have to be plugged to the internet for the first time in order to set things up, yes I am aware that you can turn it off later on, but that might be too late as it would seem that they would only need a single connection to the internet and you're in.


r/ProxyEngineering • • Jun 12 '26 All Seeing Upvote Hooray

Hot Take šŸ”„ I audited the ASNs across 4 residential proxy providers and I think this whole industry might be one giant resell operation

27 Upvotes

It's me, ya psyop boy. I am 100% sure that this is going to upset some people and that's good coz this has been bothering me for quite some time and I'm done with the silence. Background on me: moderate scale scraping, 40-60k requests a day across a few different projects, been in this space about 3 and a half years. Started noticing something weird in my logs maybe last spring. Certain ASNs kept showing up no matter which provider I switched to. At first I just assumed that's how residential pools work, ISP coverage is what it is, whatever. Then I got obsessive about it and started auditing properly. I tested four providers. I am not naming them because I genuinely cannot afford a lawyer right now but if you've been doing this longer than six months you know exactly who I'm talking about. One is Israeli šŸ‡®šŸ‡±šŸ‡®šŸ‡± company (you already know), it markets itself extremely hard on LinkedIn and twitter and has a very american-sounding brand name, and one is the scrappy european one that keeps showing up in threads like this. The ASN overlap between two of them was 34%. Thirty four percent. Between the other two it was 29%. There is one specific /24 block that appeared in all four providers "proprietary residential networks." Same subnet. Four different dashboards with four different pricing pages all routing me through the same infrastructure. I also pulled whois on a bunch of these IPs and the rotation behavior is what really got me. Not just that the ASNs overlap, it's that the rotation patterns are identical. The TTL behavior. How they handle certain request headers. You don't get identical behavior from independent networks. You just don't. That's not how this works. My theory, and I want to be clear this is a theory and not me accusing anyone of anything specific, is that there's maybe three or four actual upstream aggregators who own the real infrastructure and basically everyone else is a white label on top of that. All those "95 million IP" claims in the marketing materials are all drawing from the same upstream pool. You are paying a premium to a middleman who bought access wholesale and is selling it to you retail with a nice dashboard slapped on top. What this actually means is that when you switch providers because your IPs are getting flagged too much, there's a real chance you are switching to the same IPs with different account credentials. You're not getting a fresh pool. You're just reintroducing yourself to the same exit nodes under a different invoice, huuuuh??? Exactly. I could be completely wrong about this and if someone from one of the big providers wants to come in here and show me actual evidence that their infrastructure is genuinely independent I will delete this post and take the L publicly...NAAWT. But nobody is going to do that because they all have NDAs that prevent them from explaining how anything actually works which is itself kind of telling.

Tell me you have a better idea of this whole ordeal, I dare ya


r/ProxyEngineering • • Mar 18 '26

Is web scraping anonymously actually ethical, or are we just hiding from accountability?

27 Upvotes

I've been thinking about this a lot lately and wanted to get everyone's take on something that seems to divide the tech community: the ethics of anonymous web scraping.

On one hand, we have people arguing that anonymity is essential for web scraping. They say:

- It protects researchers and journalists investigating powerful entities

- It prevents retaliation from companies that don't want their public data analyzed

- It's a defensive measure against overly aggressive anti-bot systems that block legitimate use cases

- Public data is public - why should you need to identify yourself to access what's already available?

On the other hand, there's the argument that anonymous scraping is fundamentally problematic:

- If you're scraping "ethically," why hide your identity?

- Anonymity enables bad actors to steal content, overload servers, and ignore robots.txt

- It makes it impossible for website owners to differentiate between legitimate researchers and data thieves

- You're essentially trespassing while wearing a mask

Here's what really gets me: we tell people to respect robots.txt, rate-limit their requests, and follow "best practices" - but then in the same breath, we're rotating IP addresses, spoofing user agents, and using residential proxies or any other proxies to avoid detection. Sounds like we contradict each other all the time, no?

My controversial take: If you need anonymity to scrape a site, maybe you shouldn't be scraping it in the first place. Either the data should be accessed through an API, or your use case isn't legitimate.