r/learnpython • • May 10 '21

Can I learn Python solely for data science and statistics?

I've been learning Python during my free time for a while now and I only just realized how helpful its statistics and data science applications will be for me as an undergrad. I would just like to know if its a viable choice to focus on studying how to use the matplotlib, numpy, pandas, folium and similar libraries and Jupyter Notebook without having to take a Python programming class? I already know the fundamentals, but will I be missing something important by not learning from the very "start", so to say?

348 Upvotes

58 comments sorted by

135

u/notParticularlyAnony May 10 '21

Yes you will be fine. Python is great for data sci and especially machine learning -- you can't really go wrong.

12

u/PandaMomentum May 10 '21

I work on both python/pandas and R but I don't code otherwise and don't consider myself a coder. So yes, sure, the python/pandas/scikitlearn path (plus some SQL for offloading data) is a fine one in my experience. There are some cool python tricks you can use (string manipulation for text analysis e.g.) that can dramatically improve your workflow. There are also some specific things for which there are R libraries but no python equivalent (yet!) like stratified survey sampling, non- response analysis and the resulting weighted analysis with correct variance estimation. Still a step up from having to use SPSS for that, which was the case twenty years ago. And not having to use SAS at all is a huge plus from a budget and code readability standpoint.

9

u/iamNaN_AMA May 10 '21

People in this thread are recommending R but I would add my voice to the chorus of people steering you away from it. Python is more versatile and you never know what your future might hold - it could very well involve more intensive Python. Easy to pivot from Python to R if you need to, which is very unlikely.

6

u/SometimesZero May 10 '21

I learned R for stats but now use Python for most everything except a few statistical techniques (e.g., mixed models) that R still does better, and R’s ggplot2 is hard to beat (though plotnine is a nice Python port). R is just such an ugly language to me that I prefer to work in Python.

4

u/Sporocyst_grower May 10 '21

Yes. But why not R?=

3

u/notParticularlyAnony May 10 '21

b/c R is a weird language with a tiny corner of the programming market, the job market, and op can pick it up if they are forced to at some point in the future. python is easier to learn, with more future.

4

u/Sporocyst_grower May 11 '21

Bruh im working -despise they currently aint paying me- as a data scientist and R is the way to fking go. But i hate python cause i cant get knitr and dynamic documents to work with, im currently working on solving that with jupyuter.

1

u/notParticularlyAnony May 11 '21

You make a compelling case

1

u/orcasha May 10 '21

Unfortunately R isn't widely used in industry.

10

u/Final_Alps May 10 '21

Yes. I am in lead role as a data scientist. I may be now stretched to include data engineering on cloud (GCP) but all I do is data in Python.

If anyone asks me anything about Django, or anything webapp or anything besides handling data in a pipeline and doing math on it, my eyes glaze over and my 10 month old can probably do a better job.

And yeah. My career is built on knowing the math and knowing how to get there in Python (and previously R, STATA, SAS, and SPSS). So again, yes.

4

u/yuckfoubitch May 10 '21

How much SQL do you need to use in your role?

6

u/Final_Alps May 10 '21 edited May 11 '21

On the current project, a fair amount, as we are basically writing python scripts that build and execute big query SQL queries and scripts (coordinated on airflow). But other times it’s pure python and little db work.

SQL should be in your vocabulary, but you do not need to be a master in it.

3

u/gurugeek42 May 11 '21

I think the important thing is figuring out how you get better at python. As a research software engineer, I often see poorly written code written by someone who is very good at data science, just not very experienced with more intermediate/advanced python. As a result, the output is more difficult to understand/change/maintain. As long as you're being exposed to more advanced python than you currently know, you'll learn totally fine, and that exposure doesn't just have to be advanced courses in python, e.g.

  • Reading other's code

  • Reading blogs

  • Reading python documentation and discovering new functions/features

  • Googling "<feature> good practices"

  • Publicising your code, encouraging reviews and feedback from more experienced programmers

2

u/spez_edits_thedonald May 10 '21

I would just like to know if its a viable choice to focus on studying how to use the matplotlib, numpy, pandas, folium and similar libraries and Jupyter Notebook without having to take a Python programming class?

yeah, can't go wrong

2

u/mattstats May 10 '21

It’s great, coming from experience. Something I picked up from a buddy that was(is) far better at programming than me. Bonus points if you try and implement stuff from your courses (and not just stats and and data science! Methods from numerical analysis and so on too)

2

u/[deleted] May 10 '21

I use both Python and R for data analysis. I use Python for mainly other things and honestly since I learnt it earlier I prefer R in statistics. If you are in academia R is very practical, easy to learn and has many tools for scholar friendly data analysis. But if you learn Python you will have a skill to use for many things in addition to statistics such as automating tasks etc.

2

u/Pulsar2021 May 11 '21

Python is a swiss army knife, you can use it for a focused task. May it be data science or just a web application development. You should be fine learning just enough skills to put your algorithm in practice.

1

u/OhhhhhSHNAP May 10 '21

Yes! Learn the basics and then keep going with these libraries and tools. You can incorporate more advanced concepts when you need them and they probably won't take all that long.

If you're going the data science route, you might want to look at Jupyter notebooks if you haven't already. It tends to be preferred for this type of work.

1

u/skellious May 10 '21

yes. this is one of the main uses of python in industry. you will be fine.

1

u/[deleted] May 10 '21

Absolutely you can learn it solely for data science and statistics

1

u/pekkalacd May 10 '21

you can always go back to the very "start" if you would like to. I would recommend a book by Tony Gaddis, "Starting Out with Python"; this is what my community college used. It is very straight forward and takes you through procedural and object oriented programming in python. Other than that, just learning more about oop, the builtin data types, working with multiple python files, & PEP8 would be great just to get a handle on how things are framed when using 3rd party libraries. I am not in data science or trying to go that route, but I have heard that the big two are python and R.

1

u/skurrtis May 10 '21

Yes absolutely python can handle this job, and you'll probably find yourself picking up on the programming side a bit too and be capitalize on pythons flexibility and versatility.

1

u/xyzabc123410000 May 11 '21

Yes you can. Recently become a very popular language in the field due to the packages available to it.

Look into modules such as pandas, numpy, matplotlib, seaborn.

If you are looking into machine learning then scikit learn would be a good package. If you are looking into advanced machine learning, look into tensorflow/keras/pytorch.

There are others that I may have not put down but these are the most notable I can come up with on the top of my head. They should be more than enough to start out with

1

u/[deleted] May 11 '21

Yup

1

u/luneth27 May 11 '21

You'll be fine. I'm a newly graduated math lad, with maybe three or four true programming classes out of the dozens of math classes I've taken. I picked up Python so I could use the libraries Spotipy and NetworkX for my thesis to analyze related artists networks, and I had a completely working script within three days of picking up the language. All in all, it took longer to scrape and format the data than it ever did learning.

Just to give an example of my noobiness, I just learned the difference between () and [] for data structures today, simply because I didn't need to know it until I came across the info by accident.

1

u/meowmemeow May 11 '21

Data science and statistics and pretty plots is all I use python for and i like it:)

1

u/theabominablewonder May 11 '21

I was looking at doing a data science course and I was basically told that if you need to extract data from a variety of sources then python is needed. If you have complete data sets already then you don’t need it and can focus on statistics.

0

u/[deleted] May 10 '21

[deleted]

78

u/notParticularlyAnony May 10 '21

If you only want to do statistics

I agree if you only want to do stats, but if you want to do data science which includes machine learning, and lots of other things, then I'd stick with Python. You pretty much can't go wrong with Python because it has so many possibilities.

R is a more niche language. Python has much wider cache.

11

u/[deleted] May 10 '21

Do you know any resources you can refer me to for Python for stat? I'm currently enrolled in a course on EdX set for later in the year but it'd be nice to have some other references

9

u/badge (ish) May 10 '21

OpenIntro Statistics has labs in Python: https://www.openintro.org/book/os/

2

u/notParticularlyAnony May 10 '21

I'm not sure about books I tend to just use library docs (e.g., statsmodels, numpy, scikitlearn) and then read about individual tests as needed.

5

u/verdifer May 10 '21

R can still be used for Machine Learning, but I found Python had better options for it.

-14

u/synthphreak May 10 '21 edited May 11 '21

data science which includes machine learning

No it doesn't, not any more than chemistry "includes" physics. There is an overlap in skills and knowledge required, yes, but they are neither the same thing nor subsets of each other. This is a common misunderstanding.

Pedantic point though. I agree with everything else you said.

Edit: Downvote all you want. It won’t change the facts.

7

u/notParticularlyAnony May 10 '21

Your point isn't pedantic it is just wrong.

Data science includes machine learning. Just look at any DS curriculum at any mainstream institution (e.g., Berkeley). Dta science skills includes a heavy dose of machine learning skills (and stats and other). Not saying it is a superset-to-subset relationship (that would be bizarre). Just saying data scientists need to know machine learning.

-1

u/synthphreak May 10 '21 edited May 11 '21

data science ... includes machine learning

Not saying it is a superset-to-subset relationship

🤨

Misunderstanding about data science and machine learning is rife, especially data science. There’s so much buzz and hype around both.

Data science the field has become so diluted that it doesn't really have a coherent, objective identity anymore. Spend a day on r/datascience to see what I mean, >50% of the posts are simply debating what the job title "data scientist" even entails.

What's certain, apparently, is that data science is this arcane, mystical field that can bring wads of cash to any company or person who touches it. Details beyond that are subject to the way the wind blows. So universities are racing to the bottom to offer programs called "data science" and stuffing whatever the hot skill set du jour is into it. Machine learning just happens to be that skill set currently.

My point is that looking at a DS curriculum - and I'm sure that curriculum was only created within just the past 2-3 years btw - to draw any meaningful, non-faddish conclusions is a fool's game.

To extrapolate the essential difference and put it in ELI5 terms, data science is statistics, machine learning is engineering. Yes both involve lots of math, stats, data, and computers, so there are very significant content overlaps. But you can't just pluck any data scientist off the street and ask them to scale a deep convnet into production, nor any machine learning engineer and ask them to explain heterogeneous compound symmetry. In academia perhaps you could, but not in industry where most data science and machine learning currently happens.

So no, data science does not subsume machine learning. Their skill sets simply overlap. I hate to pull this card, but my day job is machine learning so I do know what I'm talking about.

3

u/amhotw May 11 '21

data science ... includes machine learning

Not saying it is a superset-to-subset relationship

Amazing catch.

Also, pretty much any degree requires calculus at many institutions. Does that mean psychology includes calculus? No.

-1

u/notParticularlyAnony May 11 '21

Physics requires calculus it doesn’t mean math is a subset of physics. Y’all need to learn how venn diagrams work ffs.

0

u/synthphreak May 11 '21

Sure, and doctors need to know about pharmaceuticals. Does that make them pharmacists? No. Pilots needs to know about physics and meteorology. Does that make them physicists or weathermen? No. It’s just means there is a knowledge overlap between two otherwise distinct professions.

0

u/notParticularlyAnony May 11 '21

there is overlap between two poorly defined categories -- this was my point -- that you overreacted to.

how's that gatekeeping fetish working out for you

2

u/synthphreak May 11 '21

I only gatekeep people who propagate misinformation and speak with confidence they clearly shouldn’t have.

I hope your butt doesn’t hurt too much.

2

u/[deleted] May 10 '21

Thanks, I'll check it out. Can you tell me what's the learning curve like for R?

13

u/[deleted] May 10 '21

Just for context, while R is undoubtedly the leader in stats, the gap is smaller than ever. Unless you're doing really crazy shit, if you want to learn just one, learn python. You can do so much more with it than R.

2

u/spez_edits_thedonald May 10 '21

more bang for your buck learning python. If you were a stats prof who wanted to sometimes code up your stats theory stuff, then R. (my 2 cents).

2

u/guhusernames May 11 '21

Just to add to this- learning r and python together is fully doable, I learned R first and it made python easy to pick up. I think they work well together. In my experience, though if you plan to work mainly with finance or health datasets the examples you might need in R are way easier to find/more applicable than in python. Like when I go to google an issue I'm having in R I find what I wanted and more fast, but for python it can take some digging, and I work primarily with bio/health data

0

u/[deleted] May 10 '21

[deleted]

1

u/[deleted] May 10 '21

Thanks so much for your help and input! I'll get started on this when I have some time to spare.

-9

u/notParticularlyAnony May 10 '21

R is an ugly backwater language FYI :) Python is beautiful and elegant.

But R is useful for stats and people in bioinformatics all use it bless their hearts.

1

u/[deleted] May 10 '21

I might learn a bit of both and see which on I prefer to use, but I am in the health science field so I guess R has its place somewhere?

And yeah I agree, Python is a sexy language

2

u/notParticularlyAnony May 10 '21 edited May 10 '21

I would just talk to the people in your department and find what they recommend my guess is there is a consensus. R and Python are both great, Python is a bit more general and popular and more programmer-focused than R, but R is very popular within some circles and does amazing stats stuff and I am super biased toward Python so don't really listen to me :) (the assignment operator in R was enough to make me want to rage quit).

I would just pick a horse and ride it don't dabble. And if I had to suggest a horse for an undergrad to ride it would be Python because...career prospects, general programming strengths, etc. But if that will hamstring you in your dept that would be a mistake so talk to ppl in your department to make sure I'm not giving you crap advice.

3

u/[deleted] May 10 '21

The funny thing about that is that asking my department for advice on that would be a bit much for them. I'm just a physical therapy student with a keen interest in programming. Frankly, all the faculty members will be more than impressed by a nicely formatted Excel file with neatly labelled graphs (probably even more so if I just click the One-way ANOVA option on the Data tab).

Using Python would be definite overkill in their eyes, but I've always wanted to do more with my interests, see how much it could help me with my academics instead of seeing it as just a hobby.

I'll take your advice and stick with Python. :)

3

u/notParticularlyAnony May 10 '21

in that case just do Python your life will be easier because it is so much easier to learn

3

u/skinner17 May 10 '21

If you are interested in programming go with Python. Even though R is a programming language it's really primarily used for statistical computing.

But both are good. I personally really like the RStudio UI and find it more enjoyeable to do graphs/plots etc. with it. I'd recommend to learn Python first and then a few months later when you're comfortable with the basics try out R and see what it's like to do the same stuff in another language.

3

u/quarmson May 10 '21

the assignment operator is the exact reason I quit.

0

u/[deleted] May 10 '21

[deleted]

2

u/[deleted] May 10 '21

Physical therapy actually. I'm from SEA so I believe there are some differences in academics compared to the west, but long story short, we're required to make an undergrad thesis and most of our research topics usually involve things like treatment efficacy, so hypothesis testing is often used (ANOVA, Z-scores, Chi-squared)

0

u/[deleted] May 10 '21

[deleted]

10

u/spez_edits_thedonald May 10 '21

It's statistical philosophy is more well suited for academia.

That is the most gentle way I have ever seen someone say what you're saying :)

0

u/[deleted] May 10 '21

[deleted]

6

u/[deleted] May 10 '21

I think this is the most correct answer. You may go towards CV and find that embedded systems require C++ or towards the mobile application side of things where you are deploying TF.js or TF lite models. It all really depends and you'll find out as you go OP.