r/learnpython • • Dec 16 '20

Started a week ago and sent my first script

I started learning Python 10 days ago and I just sent my first script to a client. (well friend) She asked if I could scrape a website for her. Pretty damn proud is all........

EDIT:

Never knew this would blow up like it did. Cheers for the awards thats wild. Well a lot of you have been requesting the code, here it is: Scraper

322 Upvotes

46 comments sorted by

57

u/nischalstha07 Dec 16 '20

That’s awesome. Which course did you use?

68

u/SimulationV2018 Dec 16 '20

Just googled and YouTube. Also had a background in reactJS and JS. All self taught.

37

u/looni2 Dec 16 '20

Why not to scrape in JS if you already know it?

51

u/SimulationV2018 Dec 16 '20

I was asked to do it in Python. As they are making the programme with Python

15

u/Joe_Fart Dec 16 '20

Congrats. I am now learning puppetter and javascript in general. Coming from python, so I am your opposite. Honestly I do not feel big adv for javascript in scraping, which you can not find in python. I feel more comfortable using python for manipulating and cleaning data.

15

u/StressedSalt Dec 16 '20

oh u had a background of js ahhag

6

u/canopey Dec 16 '20

i.e. "i have a background in programming"

2

u/nischalstha07 Dec 16 '20

Is automating related or webscraping?

5

u/SimulationV2018 Dec 16 '20

Yea it’s automated with webdriver

1

u/alex123711 Dec 16 '20

How much exp did you have in JavaScript etc?

2

u/SimulationV2018 Dec 16 '20

About 5 months more react front end web dev stuff

21

u/HasBeendead Dec 16 '20

Try BeatifulSoup or Selenium modules.

11

u/SimulationV2018 Dec 16 '20

I am using Selenium on another site to navigate the page

2

u/SimulationV2018 Dec 16 '20

Did not need to use it with this site

5

u/CraigAT Dec 16 '20

Did you create an exe file to send her? Or did she have Python already and you sent the file(s)?

3

u/SimulationV2018 Dec 16 '20

Sent the file

2

u/SimulationV2018 Dec 16 '20

And a link to the repository

7

u/zekeapple Dec 16 '20

That's amazing. Do you have any advice to someone struggling to learn python? Would be helpful for me

27

u/darthminimall Dec 16 '20

Not OP, but my generalized advice to beginners is read less write more. Every time you come across a block of code in a book/tutorial/doc, type it in to your text editor/IDE of choice (don't copy-paste, you'll retain more if you have to type it), then run it. Make sure you understand everything you just typed.

Also, find a problem set (Project Euler, LeetCode, Advent of Code in December, etc.) that you like and solve them regularly. And stick to one book or tutorial rather than jumping around.

7

u/SimulationV2018 Dec 16 '20 edited Dec 16 '20

This is what I do. I learn and try do it for myself. Writing a tutorial about it https://link.medium.com/8tM62VpNgcb

3

u/schokoMercury Dec 16 '20

I was just looking for this... thanks for the advent coding! Extra question... do you recommend any coding camps or coding internships?

0

u/[deleted] Dec 17 '20

[deleted]

3

u/[deleted] Dec 17 '20

8

u/imsecretlyafish Dec 16 '20

What are you struggling with? I'm by no means an expert but I may have a pointer or two.

0

u/[deleted] Dec 17 '20

[deleted]

2

u/imsecretlyafish Dec 17 '20

Corey Schafer's OOP tutorial series has been very helpful for me and many others in that regard.

https://m.youtube.com/watch?v=ZDa-Z5JzLYM&vl=en

Generally, I have a couple of resources I follow when learning something new. If I find a concept hard to comprehend I can just check the other resource. If that doesn't help either, then google-fu is the way. Hope that helps.

5

u/usereddit Dec 16 '20

It's been posted many times, but.. Cory Shaffer's channel.

However, don't just watch it. After every main point he made, or code example explained, I wrote it down within my notes, and tested it out in an IDE.

It's a long process, yes. But, you'll be learning the rest of your life, a few weeks of Youtube learning isn't that bad.

1

u/[deleted] Dec 17 '20

I recently started watching his videos to go along with each chapter in my text and I find it helps me to figure out what I got and what I am still having issues with. He explains everything very well and so far he has proven to be an excellent resource. It helps I can look up a topic and see how he explains it if my text doesn't elaborate enough on a specific method or question I have.

4

u/[deleted] Dec 16 '20

I'm in sort of the same boat. I purchased the python crash course text and have been working through that, but was wondering if a boot camp would also help out a lot. Are they worth the money? Do you have any you would reccomend?

0

u/[deleted] Dec 16 '20

[deleted]

3

u/[deleted] Dec 16 '20

For me it's more just wanting to make sure I learn the things I need to eventually find work with python and maybe some other languages down the road. I worry that since I'm self-teaching I won't learn certain things that are important that you would learn in college or a boot camp maybe.

6

u/imsecretlyafish Dec 16 '20 edited Dec 16 '20

I wouldn't worry about that. At an early stage you don't know what you don't know. As you progress and learn more about python and programming in general, you'll start to see the weak points in your understanding.

At some point for example, you might realize you don't understand the concept of an algorithm well enough. And you'll know that because you'll either need to understand it for a project, or an exercise needs it, or you've summized from reading that it's your only solution to a problem, yet the concept just wooshes over your head.

Then you'll read up, do your best until you're happy you understand it to a satisfactory degree, and rinse and repeat. In my experience, the more you learn, the more apparent the weak points in your understanding become so you always have a notion of what you need to fix, or understand better.

I can't speak about bootcamps. Personally, they don't fit my needs so I don't consider them a good deal. Maybe someone who's been through a bootcamp will be more helpful.

3

u/[deleted] Dec 16 '20

Thanks for the detailed reply! I saw Michael Reeves talk in a stream about the Dunning-Kruger effect where you start off learning something and have way higher confidence than you should, then you get reality checked hard and realize you know very little, then you get kinda down cause of this (and a lot of people give up at this point), then if they don't give up they gradually work back to a higher level of confidence as they learn more. I linked a graphic to what I'm talking about. What you were saying reminded me of the graph.

Dunning-Kruger Graph

4

u/mathmanmathman Dec 16 '20

I worry that since I'm self-teaching I won't learn certain things that are important

No matter where you are learning there will be vastly more things that you do not learn. Even the best boot camps/schools miss tons of topics. You will likely know less than someone just out of a good boot camp (good being the keyword) or a Bachelor's at first, but even they will not be fully prepared for most jobs.

3

u/[deleted] Dec 16 '20

My main concern is learning everything I need to in order to be a decent coder, and to be able to adequately work with others who have been through a boot camp or college. Obviously this would be a while down the road but it's one of my goals I'm shooting towards.

2

u/mathmanmathman Dec 17 '20

Yeah, and that's definitely a reasonable concern.

I guess my personal advice is to learn how to work with people coding more than just "how to code". Obviously, if you are a really shitty coder, people aren't going to hire you, but most jobs need someone who can work on a team and and work through issues that are too big (but not necessarily groundbreaking) for one person.

I guess my point is just don't be too concerned. I've met a lot of people with CS degrees that just can't communicate their work with others or give constructive criticism. I personally am a pretty crappy coder, but I like talking through problems which (I think?) makes me a decent team member.

Just write lots of code and talk to people about it. Most of it will suck, but rest assured, most of the code out there sucks.

0

u/[deleted] Dec 16 '20

[deleted]

3

u/[deleted] Dec 16 '20

Thanks! This helps a lot! The main thing I'm trying to do is get through the basics of python. I'm working through Python crash course by Eric Matthes to try and get a feel for the basics. It also comes with three projects to work on to show you how to apply the different things you learn. I'm hoping once I get through that it will give me enough knowledge to start my own projects. I'm just trying to learn as much as I can for the moment until I finish most of the basics and then I'm wanting to look into more specific career fields and more specialized info.

1

u/appacademy_dev Dec 16 '20

Are you considering bootcamps at all?

2

u/SimulationV2018 Dec 16 '20

No can learn for free

2

u/Brankster00 Dec 17 '20

How would you recommend learning how to find the correct HTML elements when using Selenium? I have been trying to find a good source but really struggle to identify what element to plug into selenium

2

u/SimulationV2018 Dec 17 '20

Inspect the element and use the point the arrow at the element you want to extract. Then copy the class and paste that. Failing that you can try and right - click on the element and copy the selector_path.

2

u/B_Mod_De_Turby Dec 16 '20

Congrats. I’m learning Python too!

2

u/synthphreak Dec 17 '20

Damn. Have you slept in 10 days?

2

u/Impossible-Fact7659 Dec 17 '20

I started learning Python 10 days ago and I just sent my first script to a client. (well friend) She asked if I could scrape a website for her. Pretty damn proud is all........

That's a fairly big leap with only 10 days and no prior experience using Python. I have one friend who could pull that off, but he's been programming for years. Congrats.

Just googled and YouTube. Also had a background in reactJS and JS. All self taught.

Java-Script is an OOP language. Experience with any OOP language is extremely helpful in flattening the learning curve.

1

u/sandiego34 Dec 17 '20

can you share your code, I wanna see how complex is it