r/learnpython • u/pltnk • Jun 29 '19
I took my first Python project to the next level: from first ever GitHub repository to published package.
Around a month ago I posted here my first GitHub repository ever with simple command line application for selective file copying.
Couple of days ago I finally created the first release on GitHub and published it on PyPI!
This is how my code looked like a month ago.
You can check changes comparison between initial upload on GitHub and current version here by clicking 'Files changed' tab.
And now you can install my application via pip install slcp
As you can see my code went through significant changes. Firstly I implemented more effective logging system using logging module. Initially I used just a list where log was stored and saved to the file at the end of the program. But this way a log could be lost if the program crashes. Then I changed my error checking system and made it more precise. At first it checked only one case and moreover it did it wrong :) The most challenging task was to redesign the main functions for files copying. Since I have a progress bar I needed to know total number of files copied before actually copying them. This way os.walk function was called two times: when counting total and copying files. Also there were two copy functions one of them allowed to preserve source folder structure and the other doesn't. Thanks to /u/fernly from the comments of the previous thread I made one function that calls os.walk and make a to-do list for copying adding a sublist for each file with source and destiation paths. The len of this to-do list is the total number of files to copy that can be used in progress bar printing. Then I left only one copy function instead of two which takes that to-do list as an argument. Also I added some small features like adaptive progress bar that takes into account width of the terminal and couple of new arguments for command line parser. All of significant changes in the code can be seen as merged pull requests in the commit history.
Last time I received very valuable feedback regarding architecture of the program that really helped me to make my code better, so I want to thank this community and all the people who spend their own time to answer questions.
While working on this project I've learned how to use argparse, logging, os.path, os.walk and poetry. I made myself familiar with git and learned a lot about packaging, versioning, licensing and using markdown for readme files formatting.
Of course there is always a room for improvement so all feedback will be highly appreciated. As a next step I intend to finally write some tests, if somebody knows the best way to do it for such a program please let me know.
Working on your own project is the best way to learn indeed, good luck with your learning, have fun and don't ever hesitate to ask questions!
27
u/LlamaBananaJamma Jun 29 '19
Are you familliar with the book Clean Code? It could really help clean up your nested logic.
11
u/pltnk Jun 29 '19
I am not, thanks for the advice, will check this out!
7
u/FurSofa Jun 29 '19
I just learned about this book and have to wait two more weeks for my birthday because someone already bought it for me.
I can hardly wait!
6
1
Jul 01 '19
Uncle Bob rules!
I watched this video of his from start to end in spite of it being 70+ minutes https://www.youtube.com/watch?v=ecIWPzGEbFc
Super captivating speaker.
11
Jun 29 '19
This is a really nice post. We need to keep these before/after projects coming. Super helpful to see little detailed adjustments to the initial release.
Thanks for sharing!
2
u/pltnk Jun 29 '19
Moreover, it may be useful to those who write such overviews as well. It helps to analyze progress, decisions and to review chosen instruments and techniques once more.
6
u/the1whowalks Jun 29 '19
Amazing, man/woman!
I aspire to get something like this done.
2
u/pltnk Jun 29 '19
Thank you! I will try to collect all helpful articles I used during this and update the post.
3
3
3
u/polacoski Jun 29 '19
Why do you start this proyect?
7
u/pltnk Jun 29 '19
I made a function that copies files with given extension for a task from Chapter 9 of Automate the Boring Stuff. Then I became interested if it is possible to preserve source folder stricture. After finding a solution for that I decided to add command line interface with a couple of options to make running a script more convenient. Then I thought that it would be useful to see what exact files are copied. At first, program just printed every file copied in the terminal. But since each line was printed very fast I couldn't really read this in real time, so I decided to move it to a log file. But this way user can't track progress, so I implemented a progress bar and remaining files counter. During all this I ran code many many times and tested it for possible errors to find a way of handling them. After all of that I was just tweaking and improving existing code thinking that it would be great to learn how to create packages and to publish it on PyPI. So this became a project spontaneously and on every step I just thought what else could improve my program and at the same time teach me something interesting and useful.
3
3
u/nagarajan18 Jun 30 '19
Very nice project. The approach is very practical.
4
2
2
2
u/pktippa Jun 30 '19
Great work π, post description makes me to remember book: "Refactoring - Improving the design of existing code" by Martin Fowler. π
2
u/pltnk Jul 01 '19
Thank you! Do you recommend this book?
2
u/pktippa Jul 01 '19
Yes I do recommend this book, it is very well written for modifying existing code and ensuring design patterns are followed with out breaking functionality.!!
7
u/umboose Jun 29 '19
Pytest is very nice to use as a testing framework. Then write a few unit and integration tests and you should be good to go!
3
u/PinBot1138 Jun 29 '19
Do you have any specific articles that come to mind for best practices (and examples) of using Pytest?
3
u/yoptgyo Jun 30 '19
There's also a Coursera course named "Crafting Quality Code" which teaches unit testing too. It's quite good.
3
u/a1brit Jun 30 '19
Take a look into pathlib. It has some nice methods for file exists, make_dirs with parents and exists_ok options. Could clean up some of the checks and logic.
0
4
Jun 29 '19
[removed] β view removed comment
3
u/pltnk Jun 29 '19
For some reason I thought that 3.6 doesn't have tkinter in the standard library, but I just checked and it seems that I was wrong.
0
Jun 30 '19
[deleted]
1
u/pltnk Jun 30 '19
I use primarily Windows 10 and recently started to use Ubuntu 18.04. When I tried to run my application on Ubuntu with Python 3.6 I got ModuleNotFoundError when importing tkinter.
32
u/warhawk128 Jun 29 '19
Great job! This is truly an inspiration