r/pythonhelp • • Mar 13 '21

Python script: Learning to Webscrape local businesses email and phone number from search engine

Hi,

I run a small local business that was unfortunately heavily hit by the covid-19 crisis through the last year. Trying to improve our situation by finding smarter ways of reaching out to potential business customers and recently discovered the concept of webscraping. I have started to learn a little python at home and am trying to write a webscraping script with the following properties:

  1. used in connection with search engines like google, google maps, bing, etc.

  2. finds all local businesses in a geographically defined area with their associated websites

  3. clicks on every business-website, looks for, and extracts the following business information: Business Name, website url, email, telephone number, address (here I noticed that there may be a challenge due to the fact that not everything is marked as such and may just be plain text, thus I believe that one might have to write something that looks for "some_characters"@"some_other_characters"."x_characters" to identify e.g. an email address hidden in the text of the website?)

  4. moves this information into a csv. file

As I am still fairly inexperienced and trying to learn from others by reading already existing scripts, I was wondering if anyone here knows about a publicly available python script that does something similar as described above that I could read through to understand more and perhaps customize. So excited to learn more about the world of programming and webscraping, thank you in advance for your help and time! :)

- Amy
P.s.: If you're just starting to learn webscraping in python like me I am very happy to recommend this tutorial as it helped me a lot with understanding the basics: https://oxylabs.io/blog/python-web-scraping / https://www.youtube.com/watch?v=mDveiNIpqyw

3 Upvotes

0 comments sorted by