r/learnpython • u/BeforetheBullfight • May 10 '20
Beginner Python data analysis project - critique greatly welcome!
Hi everyone,
I just finished my first Python data analysis project, and being a total beginner, I could really use some other perspectives as to the quality of my final product. I plan on building a portfolio of similar projects and using it for applying to social science research assistant jobs where some programming and analysis ability is highly preferred. Since I am entirely self-taught, I need some concrete examples of what I can do to share with potential employers.
With that said, while I'm sure there are areas that could be improved, would my project be worth sharing with some edits? Or is it actually just a total flaming trash heap that should be scrapped entirely? Positive critique and helpful suggestions appreciated!
Edit: Wow, I did not expect to wake up to so many comments and upvotes! I really needed this boost, to be honest. I'm so grateful for everyone's kindness and suggestions, and I'm definitely going to go back and try and tweak some things! I hope I can pay it forward one day, but for now, to fellow noobs who are trying to set up their first project - just go do it! I basically just forced myself to create something after learning the basics, and it really helps the learning process to break things down and figure out how they work outside of a guided tutorial. Keep coding!
Edit 2: I've uploaded a new version of the project since this was posted, taking into account some of the suggestions everyone has made. It's still not perfect, but I think it's time to move onto learning with new projects.
3
u/synthphreak May 10 '20 edited May 10 '20
This is really, really awesome, and very professionally executed. Like many others on here, I am inspired and would like to do something like it eventually, but I guess I just haven’t found the immediate need yet. I’m already proficient in Python, and already have a great job in research. Nonetheless, I don’t have a publicly-shareable portfolio, so something like this could be quite useful for future job hops
Two questions for you:
Where did you get this project idea from? I see you got the data from ProPublica, but what about the research question? Kaggle or some place like that? Or did you just dream it up? It’s very cool and intrinsically interesting. I had always unthinkingly assumed people got their project from e.g., Kaggle, but perhaps that doesn’t have to be the case!
This one is much more nebulous, but more important to me - How did you decide how to intersperse your code with the markdown narrative? In other words, the proper code-noncode ratio, + how to position the code relative to the prose. Whenever I create “story-telling” notebooks like yours, this is where I struggle the most. For example, at the very top of the notebook I provide an overview of the content, then load my libraries and raw data. But after that, I usually have lots of complex analyses and/or plotting that can sometimes require hundreds of lines of code. Because I’m afraid that huge, complex cells essentially in the middle of paragraphs decreases the narrative’s readability, I tend to put almost all my code in a single massive cell near the top (after the overview, imports, and data loading). This allows me to define lots of complex functions early on, then simply invoke them later as needed with minimal overhead or interruptions. But the trade-off is that my readers have to scroll past a lot of dense code at the top, so the notebook is both less user-friendly and less attractive. By contrast, your notebook is nice and tight from start to finish, with short code blocks that never really interrupt the reader’s flow. Can you offer any tips on how to hew more closely the way you’ve done it? I can already think of some (e.g., rely as much as possible on in-built pandas functions which can perform complex operations in just a few lines), but I’m curious to hear your thoughts.
Anyway, stellar work!