r/javascript • • Aug 08 '22

Web Scraping with JavaScript and NodeJS

https://www.scrapingbee.com/blog/web-scraping-javascript/
590 Upvotes

11 comments sorted by

View all comments

19

u/c_eliacheff Aug 08 '22

As much as I love JS/Typescript, I would still go with (python) Scrapy for scraping (or at least a scraping framework). The numbers of functionalities out of the box is too good (automatic paging, automatic detection and following of links, retry strategies, switchable exporters with no config (json, csv, db, whatever), automatic map to entities, easy config for proxies (rotations, random ones, ...), plus the awesome ecosystem for data processing (pandas, numpy, ...). I don't want to reinvent the wheel for scraping anymore.

4

u/[deleted] Aug 08 '22

and multi-threading

0

u/[deleted] Aug 08 '22

[deleted]

1

u/AegonThe241st Aug 08 '22

No it is not