r/learnpython • u/[deleted] • Jul 12 '20
Jupyter Notebooks!
Jupyter Notebooks are the tool that turned the tables for me learning python. I'm just here to evangelize them. You can run your code line by line and debug as you go in a comfortable web browser interface. It's an excellent tool, and it made it so much more efficient to learn. Check it out.
5
Jul 12 '20
I really didn't know you could use something else than jupyter. What are the other options to use to code in python?
2
8
6
u/catelemnis Jul 12 '20
You can code in a plain old text editor. I use SublimeText because it lets me run the code directly in it, but a python file is literally just text so you could code in Notepad if you wanted.
5
u/ThroughlyDruxy Jul 12 '20
I use pycharm. kinda heavy for the little coding l do but whatever
4
Jul 12 '20
Are the functions the same as jupyter? or do you use other things
4
u/ThroughlyDruxy Jul 12 '20
what do you mean the functions? isn't jupyter just an IDE?
5
Jul 12 '20
I mean just the general process of defining variables, working on data frames... The only time I used python was with jupyter so I don't really know
5
u/ThroughlyDruxy Jul 12 '20
I mean it's just python...you define variables as variables and write functions as python functions. from what l know it doesn't have special function making or variable defining processes except just writing them as you would in notepad++
3
Jul 12 '20
I understand, thank you for the explanation
2
u/ThroughlyDruxy Jul 12 '20
sorry, probably wasn't very helpful lol
2
Jul 12 '20
no no It was, I might try to find tutorials on pycharm and learn a little bit
3
u/Dewmeister14 Jul 12 '20
Honestly if your only prior experience is writing in Jupyter notebooks I recommend going right back to the beginning: install Python 3.whatever on your computer and NOTHING ELSE, write the code directly in text files that have the extension .py, and run them with "python3 your_code.py". (or whatever the command on your system is). Really strip away all the excess of an IDE so you're learning what Python is and nothing else.
→ More replies (0)2
u/ThroughlyDruxy Jul 12 '20
yeah it's pretty heavy in terms of ram usage and whatnot. But I've found it helpful and it seems to have plenty of helpful plug ins. I think it's probably better to learn with less handholding so you actually learn the universal way to do it, not the jupyter way. But I'm a amateur so what do l know lol.
3
u/ParanoydAndroid Jul 12 '20
You might find this article about professionalizing a jupyter notebook workflow with IDEs helpful:
https://florianwilhelm.info/2018/11/working_efficiently_with_jupyter_lab/
2
u/linuxqq Jul 12 '20
It's my opinion that if you are learning Python with the goal of getting a job where you write code, the "right way" to learn is through running scripts on the command line. I'm not a fan of notebooks unless you're purely doing data analysis work.
1
Jul 12 '20
I am.
I also think for students struggling with learning on "traditional" IDEs, Jupyter notebooks is a total game changer. It gives instant feedback and you can write a few lines at a time. Constant and specific feedback is the perfect way to learn any skill, and this tool gives it by design. Writing a bunch of code in an IDE only for it to not run is a non starter for an absolute beginner. Jupyter is an ideal tool for non-technical students interested in getting started.
1
u/linuxqq Jul 12 '20
I hear you. I acknowledge that I am biased because I taught myself Python exactly how I described it, by running scripts via the command line. If you do that you get the dual benefit of learning Python (you do get the immediate feedback when you do it this way) and also get comfortable with a more standard development environment. To learn Python in notebooks and then get on the job day 1 with expectations that you can set up your machine for development and start writing production code would be a nightmare. It is definitely a bit more of a learning curve at the start but I think you learn important things along the way.
2
Jul 12 '20
For programming enthusiasts or office workers trying to automate the boring parts of their job, Jupyter is a great place to start. You can export your code to a .py file no problem, and for many users there's no real reason to "upgrade" to a more traditional IDE. For an aspiring developer, learning exclusively on Jupyter notebooks would be like showing up to Isle of Mann with a bicycle. I wouldn't dream of recommending learning on Juptyer notebooks for a CS student hoping to land a dev job, but Python has a much bigger community than that. I think for lots of non-technical users that have struggled with other tutorials and workflows, this is the sort of tool that can make all the difference.
138
u/Rhjensen79 Jul 12 '20
If you are into that, then look at Google Colab. It’s a hosted Jupyter Notebook. Personly i mostly use it, when i’m playing aound with new things, or when i am on my iPad.
37
u/ropenni Jul 12 '20
Colab is my API playground, it is so convenient when I don’t want to create new venvs in my machine
13
u/cubinx Jul 12 '20
Can you schedule scripts in google colab?
6
u/Rhjensen79 Jul 12 '20
As far as i know then no. Imho i would create in colab, and then run it schelduled in some FAAS service.
7
u/Cdog536 Jul 12 '20
when i am on my iPad
You can do python on the ipad?
12
u/Rhjensen79 Jul 12 '20
Google colab is browser based, and works well on the ipad. So yes.
3
u/megaSalamenceXX Jul 12 '20
Doesnt typing code on ipad get tedious?
4
u/Rhjensen79 Jul 12 '20
I use it when i’m away, and have a little bit of time to try stuff out. On a 11” ipad, it’s not perfect. Don’t know about the 12” thou.
5
u/megaSalamenceXX Jul 12 '20
Ah I play games on my ipad when I got free time. You are a far better human than me sir!
0
1
1
4
u/bourne_again_coder Jul 12 '20
1
1
u/synthphreak Jul 12 '20
Without having done any research before asking: How does this app stack up Juno/Juno Connect?
3
u/warpingDragon Jul 12 '20
Yes, there are even Python and Jupyter notebooks apps on the AppStore. Pythonista 3 and Juno
1
2
8
u/monkiebars Jul 12 '20
Warning! Incoming dumb question:
Is Google Collab free? (If so, what are the limitations? If not, what is the pricing?)
8
u/Rhjensen79 Jul 12 '20
Yes but you can pay for extra ressources etc. But for a dev env it’s really great. As far as i understand, it spins up a container, for your code, when you launch it.
5
u/echo_awesomeness Jul 12 '20
Yes it's free. I guess the session expires after 12 hours not sure exactly how that works as I never did a coding session that long lol.
2
u/monkiebars Jul 12 '20
Wow that's awesome! Time to showcase and share some of my really low level analysis graphs in Colab rather than Excel! :)
2
u/alins_ir Jul 12 '20
Im coding with my mobile phone and i cannt run jupyter with my phone tnx a lot now i can coding better and it possible because you tnx man🙏💜
1
u/hmiemad Jul 12 '20
Check out azure notebooks. You get 1GO of free space and basically a virtual machine for free. You can create your own local DB and pile a bunch of py files.
3
Jul 12 '20
[removed] — view removed comment
8
u/Rhjensen79 Jul 12 '20
It’s free. But there is a pro option https://colab.research.google.com/signup
57
u/Kevstuf Jul 12 '20
I’ve always been more of a fan of Spyder but sadly it doesn’t seem to be popular anywhere
22
u/HeinzHeinzensen Jul 12 '20
You‘ll love the notebook interface in VS Code then! Same cell-based execution and you have the variable explorer that people like from Spyder.
9
u/Notchez Jul 12 '20
Is it possible to set up VS code to look just like spyder? By that I don’t mean the cell by cell execution like in Jupiter but rather working with scripts (that are executable line by line of you want) and a variable explorer.
11
u/HeyItsRaFromNZ Jul 12 '20
You get very similar behavior (line-by-line execution, graphic output and pretty-printing dataframes etc.) by selecting a line and hitting shift+enter. If you have the main Python extension installed, it should open a new pane to the right with the output.
It's very similar to what you get with hitting F9 in Spyder's editing pane. Instead of a dedicated pane, introspection in VS Code is taken care of by a hovering tool-tip.
7
6
3
5
u/Internet_employee Jul 12 '20
Oooh!! Thank you! I dropped Jupyter after discovering Spyder’s variable explorer, definitely giving VS Code a try now
4
u/mayankkaizen Jul 12 '20
Can you please point me to some resources about that? I use Jupyter notebooks and got VS Code on my machine. Although I don't use VS Code very much but would love to explore notebook interface in VS Code.
2
u/HeinzHeinzensen Jul 12 '20
Sure, this link should get you started.
3
u/miguelito_loveless Jul 12 '20
Thanks for the link. Still pretty beginner and 100% a happy VSC user and want to try this-- I couldn't get comfy w browser JN but get how useful that way of working could be.
10
u/Retinator99 Jul 12 '20
I wish there was an easy to update version independent of anaconda. Having to run it through another program makes it super clunky
13
u/ivosaurus Jul 12 '20
pip install spyderor you can use thepipxtool for similar but self-contained results.Also not sure if people know about it much but miniconda is great. It's anaconda python distribution but without the 500mb installer and every scientific package known to man cluttering up your root environment by default.
Anaconda don't seem to advertise it much, seems they want newbies always grabbing the whole pile to play with at once
7
u/kodyloki Jul 12 '20
I agree, I really enjoy Spyder. I can’t easily use Jupyter notebooks at work due to running on remote machines and shared networks, etc. so Spyder is really all I use.
1
1
u/Turd_Bucket Jul 12 '20
I learned R first, and use RStusio. Spyder makes sense to me, but I am not the biggest fan of Anaconda. I know you download Spyder free standing, but haven’t gotten around to it.
Didn’t install Anaconda either when I got my new PC.
-4
Jul 12 '20
[removed] — view removed comment
8
3
u/ParanoydAndroid Jul 12 '20 edited Jul 12 '20
Pycharm is my go-to and I looooove it. But its jupyter support is god awful.
2
Jul 12 '20
Agreed. VS Code doesn't hold a candle to it. At work, the code that gets committed with the most warnings almost always comes from a non-PyCharm editor.
57
Jul 12 '20
It’s great for experimenting. But beware of the way it’ll influence the way you code, especially if you’re a beginner. I’ve seen people overuse it to the point that they fail to write efficient code because their code just works well in their notebook. However, none of it becomes reproducible outside that notebook. In order to make it useful beyond that notebook, you do need to at least convert the code into more useful format using classes, functions, etc .
9
u/StillTop Jul 12 '20
glad someone mentioned this, part of the reason why i use pycharm with anaconda distribution
5
u/synthphreak Jul 12 '20 edited Jul 12 '20
Can you provide any concrete examples of someone’s coding style being influenced by the affordances of the notebook environment in this way?
9
Jul 12 '20
Yes. I once had a coworker who wrote mostly in notebooks. When it came to making production code, they struggled to get everything put together. They ran into a problem where they didn’t know how to encapsulate half their code in something reproducible because the notebook had become very messy and hard for them to even understand what was going on. I’m sure if they were given more time to figure it out, they would have. But we didn’t always have that time, so we had to sit down together and write it from scratch- this time in a script as a class from the start. They were so not used to the process that I ultimately had to do it myself. They still have similar habits now, but they’re trying to write more in scripts. It just causes ineffeciency for everyone if one team member is inefficient.
I use notebooks myself, but I actually write my classes and functions in notebook cells first and then immediately paste those into scripts in an appropriate manner. I then import those classes and functions instead of having them in my notebook. This is to ensure my notebook doesn’t become one giant hot mess.
5
u/MurderMelon Jul 12 '20
I actually write my classes and functions in notebook cells first and then immediately paste those into scripts in an appropriate manner. I then import those classes and functions instead of having them in my notebook.
This is exactly what I do and it's worked perfectly for me. I basically just use notebooks to prototype and debug. Then I package each discrete part into its own module, which I will then call from a master notebook (which usually ends up getting converted to a script, since all the dependencies are external anyway)
2
u/nomos Jul 12 '20
I do this too, but still havent found the sweet spot for when to move code over to modules. If that function you move over isnt quite debugged fully, at a minimum you have to restart the kernel for your changes to take place as opposed to rerunning a cell, which can take time. Even worse when you are executing code in a remote environment and you have to pull your local changes back into that environment after making changes. Agree that this is still better than having all code in notebook, though.
1
u/Socrato Jul 12 '20
For me, and maybe this isn't good practice, I try to encapsulate the notebook itself. I tend to create many directories, and my notebooks are "<function name> DEV" and I import what's necessary, usually my class I've developing along with the entire import list in those scripts, and then develop the new function in that. Once it's done, I save that notebook and move that over to my class main script. Then I go back to my "master" class testing notebook and make sure everything runs correctly. Then for the next function/process I start a new notebook.
Not sure how great that is but it's been pretty successful for all my projects which tend to be backend data processing and etl type things. I don't write any web code for production but I populate lots of tables that are used in prod.
4
u/synthphreak Jul 12 '20
I imagine that at a minimum, periodically restarting the kernel and running all cells will go some way toward fixing this problem.
As a notebook grows in size, complexity, and age, it becomes easy to lose track of what’s defined and where, what isn’t defined, and in what order cells need to run to work. But periodically rerunning everything and making sure it still runs will help keep your organization in check, forcing you to write your notebooks in a more “script-like” manner.
But TBH I really don’t use notebooks that much. So I’m curious what others think about this.
2
u/g0lem Jul 12 '20
You've pretty much hit the nail on the head. The problems you mention are quite hard or at least frustrating for a beginner to debug. Notebooks can be used correctly if you often refactor (groups of) cells in functions or simply delete them (if not useful for the end goal), but on the other hand, there are good debuggers in Pycharm and other IDEs. I don't think there are definitive arguments for and against one particular tool/environment for development. However, notebooks shouldn't grow too much in size/complexity/age, because at that point they need to be refactored/packaged in modules.
2
u/zylog413 Jul 12 '20
Most IDEs have the ability to jump to function and class definitions, and to have them open in a separate file in a separate tab/window. This can help encourage code reuse and abstractions. Without this, there can be a tendency to just dump the definitions right into the current notebook, and if the notebook becomes too long, do away with functions and classes entirely.
Also without functions, you probably don't end up with unit tests for them.
12
u/mr_bean__ Jul 12 '20
Good. Now try using jupyterlab. It's so much better
6
u/noonearya Jul 12 '20
I've gone back to notebook because the lab is missing quite a few things, it honestly depends on what you are doing.
With theming the notebook becomes acceptable
4
u/magestooge Jul 12 '20
I particularly miss the find and replace and easily customizable keyboard shortcuts. I don't know why they'd remove those.
3
u/noonearya Jul 12 '20 edited Jul 12 '20
Yes. And also tqdm, and extensions, and ctrl+tab to cycle through different notebooks, and output limitation.
1
u/jabroniiiii Jul 12 '20
I have been trying for a while but just got tqdm to work in jupyterlab a few days ago
1
2
9
u/Bocote Jul 12 '20
It's been great for Bioinformatics stuff I'm doing.
I did some tweaking to let it run R as well (although would still prefer Rstudio for the purpose).
16
1
u/apes-or-bust Jul 12 '20
How do you live without variable explorer? I really don’t think that I could.
2
3
1
u/pitkeys Jul 12 '20
If you use an iPad, also check out Juno for Jupyter. Pretty neat stuff!
1
Jul 12 '20
I've heard of developing for iPads, but developing on an iPad is some next level shit. I will not be joining you. Shine on you crazy diamond.
1
1
u/AverageDingbat Jul 12 '20
Is it, like, free or something, man? That would, like, be totally rad, you know? Free stuff.
1
1
u/24mitcher Jul 12 '20
Hi, Trying to install Jupyter Notebooks within multiple virtual environments (Python 3.6, 3.7)
After getting the virtual environments for each of the python3 versions, I have been trying to successfully run Jupyter Notebook to have the localhost window feature several different python3 versions. For some reason, each time I type 'Jupyter Notebook' in the command prompt I get an error that 'This command cannot be run due to the error: The system cannot find the file specified at line:1 char:1 +Start "file......./.local/share/jupyter/runtime/nbserver-3."'.
I was able to use pip install for Jupyter, ipykernel, and jupyter Notebook.
If anyone could point me to a common reason this error occurs it'd be awesome. The Jupyter notebook is fine in anaconda, but I would like to tie up all loose ends with making it functioning in the Windows console. Is it likely that a kernel spec is causing a problem?
2
Jul 12 '20
I think you're navigating to that file incorrectly. The file location shouldn't start with "file" it should start with something like C:/.../.local/share/jupyter/runtime/nbserver-3
Maybe find the specific file in file explorer and copy the exact location using the file explorer's gui?
Hope this helps. Let me know if it doesn't.
3
2
u/ursoevil Jul 12 '20
I use Jupyter Notebooks like an actual notebook for learning. I’ll run short lines of code and show the outputs including error messages if there was any. It’s so I can go back to them as a reference whilst I document my learning process.
When writing codes for bigger programs, I use PyCharm. IPython is integrated in PyCharm as it is with Jupyter so it’s also easy to experiment and test code out on PyCharm.
2
3
u/as9934 Jul 12 '20
Jupyter Notebook ftw. I started with it, then switched to VS Code and now I’m back on Jupyter. Haven’t looked back once.
9
u/ensign_paris Jul 12 '20
VS Code even Supports Notebooks!
3
1
u/kingbradley1297 Jul 12 '20
The cherry on top came when I realised it could be run on an android phone as well
1
u/TheBobPlus Jul 12 '20
How do you do this?
2
u/kingbradley1297 Jul 12 '20
There's a post on this very subreddit for the same one. I'll try to find it
1
u/Ke5han Jul 12 '20
I installed a Jupyter docker on my homeserver, so I can play with lines of code even with my phone or iPad.
1
1
u/kavitadrake Jul 12 '20
I’ve only recently started. My BIL hooked me up with Jupyter after a week of struggling with some other crappy apps like PyDroid. Jupyter is soooo much nicer, and the concept of the notebooks is really helpful.
1
Jul 12 '20
Check out Kaggle, it's incredible. Tons of wide open data-centric notebooks created by data-science folks.
1
u/unhott Jul 12 '20
Yes—- Jupiter notebooks are really great. But you can also royally Screw and confuse yourself by running cells out of order! So a traditional step by step may be better for absolute beginners.
I recommend python tutor for absolute beginners.
1
1
u/Sigg3net Jul 12 '20
Awesome that you enjoy it!
I never got it myself, but I am most at home on the command line (and use ipython3).
27
u/[deleted] Jul 12 '20
Agree to disagree. Beginners have enough pitfalls to avoid without having to mentally grapple with the concept of hidden state. They're great once you're past that beginner stage, though.