r/learnpython • u/EndlessTomes • Nov 24 '19
The basics so basic they don't even teach it
What is an environment, whats a configuration?
I keep seeing terms like these and a bunch of other stuff in Python courses I take, but no one takes the time to explain how Python even works on my computer. Are there any videos you can suggest that talk about not how to program with Python but what an environment is and why I have both a terminal and IDLE. Explaining Python even functions on my Mac.
thanks!
9
Nov 24 '19 edited Nov 24 '19
With deference to the knowledge and expertise of the other posters...I was a beginner (on Mac also) 2 years ago and hopefully can share a few tips. The advice given by Solonotix is great but really too much detail for what you need if you are a beginner.
The reason they don't teach you about environments in beginners courses is that they are not really a beginner's concept. Environments help you isolate different versions of python and sets of libraries ('packages'). You need to crawl, then walk, then run. If you are a beginner and they are telling you about environments, you are doing the wrong courses.
I can understand the confusion about terminal vs IDLE v both (and what is an IDE) etc. This is tricky for beginners and took me a little while to get straight in my head. Terminal is just how to interact wth your local mac. It is independent of python and is equivalent to cmd on windows. You'd have it even if you had no python on your mac. An IDE is just like a special text editor designed to help you program in a particular language. (Some handle multple langauges). You don't have to, but most people use an IDE because they give you nice features like checking your input, highlighting of errors etc. IDLE is a basic IDE for python. But because it is very basic most people don't use it. It's not even in the top 10 according to this poll. A watchout on your mac is that you probably have at least 2 versions of IDLE attached to different versions of python. Do a search using spotlight for "IDLE" to show what you have on your mac. But as said I woudn't recommend using such a basic IDE.
You should choose an IDE that works for you. Pycharm and VS Code are very popular for example. But they are very well featured and maybe also too much for a true beginner depending on your skill levels. If you do the extremely popular and very good course MIT Introduction to Computer Science and Programming Using Python they recommend you use Anaconda, a distribution which includes an IDE called Spyder. I know not everyone will agree if I recommend it (and it's also not in the top 10), but it's how I started and it solves multiple problems that beginners shouldn't have to worry about. I now use VSCode for main projects and Spyder for playing around but that's fine because I know now a lot more than I did 2 years ago. Why Anaconda and what problems does it solve for beginners?
You just download and install it, and straight away you can start using Spyder, no config or setup required
It sets up an environment for you automatically so you don't screw up the native python already installed on your mac.
It comes with dozens of many important libraries already installed and ready to go so you don't have have to go searching for them. You can upgrade any or all of them with a single command whenever you want. You can also install others if you need them.
At any time you can upgrade to another IDE like PyCharm or VSCode etc and keep using your existing envinronment, no worries
That's my 2 cents.
(edits: typos)
2
u/EndlessTomes Nov 24 '19
Thanks this is a super clear answer for me, I'll also check out the course!
1
Nov 24 '19
Please consider calling it out in your post comments - you got a lot of answers here that just weren't helpful.
3
u/stickerspls Nov 24 '19
I took a programming course and learned almost nothing because the explanations were always too overwhelming for me and there were too many starting assumptions. Your comment was the easiest for me to understand! Is the MIT course as easy to follow?
Ive always wanted to learn programming and incorporate it in my work but I keep getting stuck by not understanding foundational concepts.
4
Nov 24 '19 edited Nov 26 '19
Well, the MIT Edx course is awesome. There is a reason more than a million people have done it. The good thing about it is in the title: It's CompSci using Python, not "learn Python". But it assumes no prior knowledge of python, and automatically along the way you learn all the Python basics, and quite a lot more. But having said that there are plenty of awesome other courses / videos / materials. You can start here. https://www.reddit.com/r/learnpython/wiki/index. But I would recommend MIT Edx 100%.
6
u/SwellJoe Nov 24 '19
Nobody seems to have mentioned that "environment" has a non-python or even programming related meaning. It includes things like the path (PATH on UNIX and Linux, I don't know about Windows), which is the places where your shell will look for executable programs (like /bin, /usr/bin, and /usr/local/bin). You'll also possibly have a PYTHONPATH variable (if not, Python uses a compiled in default, so that's fine, too) that is where Python will look for library files to load (your own modules and any modules you're importing from the Python distribution or from pip or other source of third party Python packages).
Environment is the shell you're running, the way your environment variables are set (this is configured in ~/.bashrc, ~/.bash_profile and the system-wide variants of those files, generally), probably the terminal settings (though you rarely need to care about this unless you're doing something complex with e.g. curses). It may also mean your toolchain for working with Python, but I suspect you've seen someone mention setting something in your environment, and that's what you're asking about, and that's these environment variables.
PATH can be set with export PATH=/my/new/binary/path:$PATH (this adds /my/new/binary/path to the existing $PATH variable). Stick that in your .bashrc. Same for PYTHONPATH (though if you install a Python library with one of the most commonly used methods, like pip, you won't need to modify PYTHONPATH manually. Only if you want to add some development directory before you've started packaging/installing you're code.
You can see the contents of your current shell's environment with the env command.
1
u/EndlessTomes Nov 24 '19
Hi! So I know how to change my path on VS code but if my terminal is using a different one then how would I change that?
My current terminal is stuck using an older environment but changing that is proving difficult.
1
u/SwellJoe Nov 24 '19
If you're using UNIX/Linux, I mentioned how to do that above (
~/.bashrc, assuming bash is your shell of choice). I don't know about Windows, but you can search "how to set path in windows".And, it's not clear what you mean by "older environment" here. I think maybe you mean your Python version? If so, just say "Python version". Python is not your "environment" (despite what others in this thread have said), or at least it is not clear you're talking about Python version/path/whatever when you say "environment".
When modifying path, make sure the version of Python you want comes before any other versions in the path (or the other versions don't appear at all). Or, specify the version on the command line when calling python. e.g. on UNIX/Linux you'll likely have both a
python(for Python 2) andpython3(or apython2andpython, maybe on very new distributions where Python 3 is the default), and can run either without altering the path; both will have their own environment (PYTHONPATH) compiled in, so you don't need to worry about them being able to find their libraries.It sounds like maybe you have a somewhat messy Python installation or multiple Python installations. That might be causing confusion. On Linux, use your native package manager and the OS-provided Python packages until you know you need something else (which may never come up), so you don't need to worry about paths or other stuff like this. It's been a long time since I did anything with Python on Windows, but it can be tricky to have multiple installed versions (they end up installed in their own directories, which is good, but paths can be confusing). Linux is generally just more programmer-friendly, IMHO.
1
u/EndlessTomes Nov 24 '19
I'm on Mac, but you've got me perfectly pegged with the messy installation- as that's my exact problem. But I'd actually like you to expand on this "Python is not your "environment" (despite what others in this thread have said)". What exactly is an environment then? How do I create or delete one or even see which ones I have?
1
u/SwellJoe Nov 24 '19 edited Nov 24 '19
So, there's a tool called
virtualenvwhich creates a virtual Python environment, useful for testing or developing against a specific Python version, etc. A Python environment is a thing; and I suspect it's what people were getting at elsewhere in this thread. Some people even recommend every new application get its ownvirtualenv, so you know your dependencies are well-defined, etc. But, that's unnecessary complexity for you right now (probably). You just want to run the scripts you write while you learn, and those will usually be short single-file exercises rather than big full-blown applications.Environment isn't exactly any one thing. There is your Python environment (the version and paths, mostly), there is your development environment (you mentioned VS Code above, so that's your dev environment, which may or may not be configured to interact with your Python environment, and may interact with your shell environment, as well), and then there's your shell environment, which determines stuff like which Python version will run when you type
pythoninto the shell, if there are multiple programs namedpython.On OS X, your default shell is bash. So, you can modify
~/.bashrcto add stuff to the PATH. All the stuff I said about PATH above mostly applies to Mac OS X (it's mostly a UNIX, though a weird one and not at all standard in terms of where things get installed).How did you install Python? Mac OS X probably has a Python version that ships as part of its software development kit (or whatever? it's been 15+ years since I did any dev/packaging work on Mac). Use that one if you can, unless/until you know you need something else. Though, I guess if that one is Python 2, you need something else, as it makes sense to be learning Python 3 today, since 2 will be EOL very soon. I'm not an expert with Mac OS X...but if you know the path to the Python version you want, you just need to add the directory it is in to the front of your path, as I described above. It will have the rest of its environment configured into the Python binary; you won't need to set PYTHONPATH or anything else. You just need to make sure your shell (terminal is just the GUI displaying the shell, it has no bearing on PATH) calls the Python you want when you type
pythonon the command line. You can check the version (to confirm or deny you've got this right), by runningpython -VEdit: Also, you might be able to call the version you want with the version on the end (e.g.
python3orpython3.7, as those usually get setup as links during installation). This way you can have and call different versions of Python without altering your path. You may even need to do this if the binaries for the different python versions are installed in the same directory. If I want to run my code under Python 3, for example, I have to call Python withpython3(I'm on Fedora 30, which has Python 2.7.17 as its default Python version calledpython).1
u/EndlessTomes Nov 24 '19
I'll be looking into virtualenv, I've figured out that as long as I tell my terminal to open python3.8 specifically then it will be fine (as typing python3 will give me a much older version).
I downloaded Python by simply going online and doing so from the main webpage, if there are other ways I do not know them.
Thanks again, I appreciate you help.
1
u/SwellJoe Nov 24 '19
There aren't any great ways to install things on Mac OS X. Package management is disastrously bad there. So much so that Homebrew is wildly popular, and it's literally terrible (but, popular, so it's probably still a reasonable choice, since people will recommend it and often docs about working on Mac will start with "Install this thing with Homebrew").
4
u/NotTooDeep Nov 24 '19
Let's say you drive a Jeep. It has available 4 wheel drive.
The jeep is your program. The roads and trails are the environment. The configuration is some switch on the dashboard or gear shift knob that tells the jeep to run in either two wheel drive or four wheel drive. Changing the configuration changes the way the jeep behaves.
Let's say you manage a large hotel. This hotel has many conference rooms. This hotel has many storage rooms filled with different tables, chairs, PA systems. Some conference rooms have movable walls. The hotel is the environment. The setup of the conference rooms is the configuration. The program is the people giving and receiving presentations in the conference rooms. When you register at a conference, you are given a program which lists the times and places for all the events.
A python program may serve pages to a website. To accomplish this, it needs to access a database. The python program will have one or more configuration files which tell the program where and how to find that database. It may be on the same laptop as the program. It may be half way around the world in the AWS cloud. From the program's perspective,it does not matter. What matters is that the program can connect to the right database to do its work.
Notice that in the last example, the environment could be on a laptop, in which case the configuration would tell python this is the case. I think you should be getting the idea of how these words are used. In the hotel example, the environment is obvious. The hotel cannot pick up and move somewhere else. Notice the layers.
The conference rooms have their configuration. The hotel is configured permanently in a city. The parking for the conference, however, can vary; i.e. all conference attendees get free parking in parking structure A only. Could be a different parking structure next time.
So here are some good questions for you to consider. At what scale am I defining an environment? If I'm remodeling the hotel, the scale may be at the level of the utilities running throughout the building. If I'm turbo charging my jeep, it's at the scale of the engine bay and maybe the suspension. If I'm learning to write python programs on my laptop, where it the environment?
You could say it's the laptop, but that is very broad and not so useful if you need to change a configuration. That's your clue. Environments have configurations. Python installs and then needs to know things in order to work as expected. Most of this is probably preconfigured in some standard way, but where the data lives is typically not.
BTW when you download some program, you picked the version that runs on your Mac. At this scale, the environment is the Mac and the configuration is picking the Mac version. When you downloaded Python, you may have configured it to access a database. Python is the environment and the file you modified is the configuration.
To your last question about terminal (command line) vs. IDE. Everything you do easily in an IDE can also be done by editing a file at the command line. It takes longer to edit a file at the command line and is more error prone, which is why IDEs are favored, along with all the bells and whistles you get in an IDE.
You need to know both. The IDE is optimized for human interaction. However, the IDE is not how we typically automate recurring jobs. That is more easily done at the command line. The CL is optimized for human and machine. It has schedulers and controls and access to low level OS calls. It's more flexible than an IDE, but can be more difficult to use because of this flexibility. You need to know both because for some tasks (debugging code), an IDE wins every time, while for other tasks (modifying a configuration file) the command line is your go-to tool.
Words in software engineering are not always explicit in their meanings. Environment is one such word. Its meaning depends on the context and scale. Configuration has a consistent meaning, but also depends on context and scale to define its details for what can be configured.
Why do we have configuration files? Because if you put the database connection information in your program, then every time you wanted to connect to a different database, you'd have to change your program, which opens the door to introducing bugs into your code. By having this configuration information in its own tidy home, when you change the config, it either connects or it does not; there is no try. And you cannot break your application code by accidentally fat fingering the wrong key.
1
50
u/totallygeek bandarji Nov 24 '19
Constructing a decent development environment does not sound basic to me. In fact, I would have one heck of a time assembling a recreation of the one I use at work. At home, I have a simple setup with Docker containers, but everyone I know has a different type of system: variables, linting, directory structure, etc. A quick search on YouTube for 'python setting up environment' turned up quite a few videos. Even ones which do not line up exactly with your specifics might turn out helpful.
41
u/double_en10dre Nov 24 '19 edited Nov 24 '19
Hah, agreed 100%. Setting up your development environment is always a complete pain in the ass. And starting out at a new job will really drive this point home. Every time I've had to do that and set up my environment, I've asked a senior dev for help -- and without fail, they go "hmmm oh hum well...I set mine up years ago...can't really remember wtf I did to get it working...weird..uhhhh."
(Point being, people shouldn't feel bad if they're struggling with this. Like you said, it's not basic.)
18
Nov 24 '19
I realised this potential threat to my wellbeing early on, so I did what any well-combed schoolboy would do and wrote a quick-recipe in a notepad document which I always keep in that tiny pocket inside the pant pocket.
11
u/tomRyanXpress Nov 24 '19
I'm currently going through this - just moved to a new team who use very specific dev environments. I've spent the past few days going through very dated docs, spinning up VMs and asking senior devs for help. Thankfully our team communicates incredibly well, and everyone is quick to respond.
But yes OP, like others have said configs and environments require deeper knowledge that just takes time to pick up.
6
u/Kristaps_Porchingis Nov 24 '19
This makes me so happy to hear. I felt absolutely brain dead when I could get a few bits of example code working, but couldn’t get other major chunks of the environment functional
0
Nov 24 '19
[deleted]
1
u/L0neKitsune Nov 24 '19
I've worked as a dev for 6 years now and I don't feel like I have my setup perfect.
1
u/ScaredOfWorkMcGurk Nov 24 '19
I don't understand, what do you need for a setup? I just download python and whatever external modules I need then script from notepad++, should I be doing more?
0
Nov 24 '19
[deleted]
2
u/ScaredOfWorkMcGurk Nov 24 '19
I know I sholuld probably use a different editor. I tried to use PyCharm when I first began but it confused me more than I needed at the time so I just used N++. I will give Atom a try.
5
u/Scutterbum Nov 24 '19
I don't understand how this can be difficult. (Because I'm a n0ob). So setting up an environment to me is creating a new environment called "testing", installing your packages into it. Then start working. How much more difficult can it get?
2
u/SaltyEmotions Nov 24 '19
Its hard to get it exactly to streamline the dev process and make it easy for yourself.
First, you have to set up
anacondaorvenv. Then, you have to set up your IDE. This is especially difficult for a few reasons like personal preference. Then you've got all of your mixed steps in there like installing packages, your deployment system, your compile system etc.1
u/Scutterbum Nov 24 '19
Ok I'm definitely a n0ob. I don't know what a python compile system is. Better read up.
1
u/CompSciSelfLearning Nov 24 '19
The most difficult part is making it reproducible for yourself and others.
1
u/muy_picante Nov 24 '19
I have a git repo with all my tool configs, so when I get a new machine I just pull down that repo, install my text editor, and I’m good to go.
7
u/baetylbailey Nov 24 '19
The first chapter of the book Python in a Nutshell covers environments, etc.; at about the right level.
18
Nov 24 '19
The problem is that understanding environments requires a deeper understanding of topics more complex than programming, such as computer architecture and operating systems. Unfortunately, modern programming curriculum sees these topics as increasingly divergent.Truth be told, knowledge of circuits would actually be the best place to begin the fundamentals needed to learn about topics such as architecture and OS's, but that would be added curriculum, not necessarily curriculum that 'simplifies' programming (although it may actually help to simplify programming once you've been doing it for a while).
2
u/slick8086 Nov 24 '19 edited Nov 24 '19
I keep seeing terms like these and a bunch of other stuff in Python courses I take, but no one takes the time to explain how Python even works on my computer. Are there any videos you can suggest that talk about not how to program with Python but what an environment is and why I have both a terminal and IDLE. Explaining Python even functions on my Mac.
This is basic computer knowledge. A programming language is a method for communicating instructions to a computer. Effectively learning a programming language requires that you know the basics of how a computer works. In my opinion you would do well to learn the Linux operating system before you tried learning the Python programming language. Linux is similar to how your Mac works, so once learned a bit, that knowledge will transfer to your Mac for the most part.
If you don't have an extra computer (it can be a really old clunker) you can get a Raspberry Pi for $35. The Raspberry Pi is designed specifically to teach computer fundamentals and programming, it comes with Python as part of the operating system. All the folks here answering your specific questions is nice and all but this is just the tiniest tip of the iceberg, and will just lead you to more questions. The answer to all those questions is in learning the basics of how computers work, how operating systems work, etc.
here are some links about learning linux.
1
u/EndlessTomes Nov 24 '19
I've actually been thinking about playing around with raspberry pis, it's awesome to hear they come with Python in the OS. Are there any courses you would recommend on using them both together?
1
u/slick8086 Nov 24 '19 edited Nov 24 '19
Well I don't know any courses of the top of my head, but there is this book.
https://www.raspberrypi.org/blog/learning-python-with-raspberry-pi/
Learning Python with Raspberry Pi doesn’t presuppose any computing knowledge, and takes you from a standing start through variables, loops and functions, 3D graphical programming, building games, networking, scripting, interfacing with hardware…and, of course, Minecraft. There’s much more besides: if you work your way through the whole book you’ll be building robots and alarm systems; manipulating sound and video; and learning how to test and debug the Bradbury and Everard way.
EDIT: I found this:
https://www.youtube.com/watch?v=4z9trGhCbfk
This guy, Sentdex, is excellent, he has scads of great python instruction. Highly recommend.
1
5
Nov 24 '19
There is NOTHING "beginner" about these concepts. A lot of people who are great at programming are tired of setting them up—that's why DevOps is a thing—developer operations.
5
u/donteatkrust Nov 24 '19
The problem is even beginning Python takes 20+ years of computing at face value. Unless you grew up in the 80s/90s is unlikely you would have seen the evolution take place, even as an end user. Highly recommend the CS 50 course, it's free and does a good job of breaking it all down from the 0s and 1s, just started it myself!
5
u/Solonotix Nov 24 '19
Being that I grew up in that time, you make me wonder how much of my education in computers was environmental instead of actual schooling. Like I remember my elementary school's computer lab being a bunch of IBM's with nothing but CLI, and you were instructed what commands to type in to get to the program we were using that day. Then, by the time I was in middle school, Windows had become the de facto home computer, while Macs had become the standard computer in schools, going so far as having my high school computer lab decked out in the newest iMacs in all their clear plastic glory.
I may not have seen the birth of Pong, and the dawn of console gaming, but I definitely saw my fair share of computer history.
1
u/LostInTheAether304 Nov 24 '19
Your education vs environment statement. This is actually how I fell into my career. I was learning BASIC on a Commodore Vic 20 when I was about five. I took my first computer apart and rebuilt at 9 or 10 (an IBM PS/2 80386 with a 25mhz professor and 40 meg hard drive that I upgraded to 120M....). In 8th grade I convinced the computer science teacher in my school to get off token rings because they slowed us down too much and it was dying (...don’t see many token network around...)
Point is, due to other things in life I actually never attended high school (left school in 9th grade), and took whatever work I could until my son was born 9 years later. At that time I needed a real job to support him, and started shotgunning out my CV to anyone and everything tech related. Someone took a chance.
I did eventually go to college, but it was after five years of working as an IT tech and I went “I can do better then this...”, But it all came about because I don’t remember a time I didn’t have a computer in front of me.
1
2
u/soupie62 Nov 24 '19
Silly me - when I think of environment, I just think of the PATH variable.
There's obviously a bit more to it than that...
1
u/sojohnnysaid Nov 24 '19 edited Nov 24 '19
I wrote a big article going through really the basics of python with lots of code examples. It starts off talking about setting up an environment before starting a project and how it can be ported to other machines: https://johnyzaguirre.com/2018/11/13/cs50-week-6/
1
u/torkelspy Nov 24 '19
This class https://cs61a.org/ and the online book that goes with it https://composingprograms.com/
helped me a lot with those kind of questions. Particularly the early sections on environment diagrams and the later section on writing an interpreter.
0
Nov 24 '19
[deleted]
3
u/EndlessTomes Nov 24 '19
This here helps a ton, I kind of just assumed that it was more common knowledge. My problem is that I've downloaded Python 10 different times in the past few years to try learning it, again and again. Now that I've actually gotten a bit deeper I'm encountering a problem where I have way too many different environments (and I didn't even know what that meant).
18
u/ReversedEgo Nov 24 '19
Worth mentioning for anyone reading this comment: if you are on a linux distro, DO NOT uninstall existing python related stuff. That likely ends up nuking of a part of your operating system because it has its own python along with many specific OS related packages installed which cant be fixed easily. Trust me. I know. From experience.
3
u/nonotion Nov 24 '19
On the other hand, there's no better way to learn linux than breaking your os a few times. The Arch approach
0
Nov 24 '19
[deleted]
1
u/EndlessTomes Nov 24 '19
13 hours?! I'll be sure to save this for later, it doesn't seem to have any of the topics I was looking for specifically but I'm sure it's quite helpful.
0
350
u/Solonotix Nov 24 '19
Let's cover some terms you mentioned.
Environment: this is where your code runs
Environments occur at multiple scales
Your local machine might operate as a node in a cluster
The internet might be part of the domain if you are creating a web application that has more than one physical location (typically referred to as a domain)
Many domains might connect to forms parts of a whole (commonly referred to as a network)
Configuration: Metadata that your application uses to determine its operational state
This also occurs at many scales
When you run Python, you can pass configuration parameters as command-line arguments. This is a very localized scope to your current execution context
Your Operating System likely supports Environment Variables which are hosted by the OS and can be called on by anything within that environment. These are of a broader scope encompassing the machine that runs the code
You can import configurations using Configuration Builder classes within the language for predefined formats, or parse a custom made file with proprietary logic. These are a special scope, since they reside as a file reference, which could in theory live anywhere. If it's on the machine, it would be local, but a config file on a remote server might also be valid, in which case it is accessible in a far broader sense
Technically, the time could be considered a configuration, since it is a value controller by your computer, and updated by external sources, but universally available to your application context via the OS.
Terminal/CLI: a command-line interface
In theory, an OS can be run entirely by commands from the user. Realistically, there are some things that are easier to do in CLI, and others in GUI (graphical user interface; aka Point-and-Click).
The terminal will generally be where you call the Python interpreter to build your app, and/or run it. If you get to the professional stage of, say, deploying a website, you'll likely create a build configuration file to some software like TeamCity which will run command-line arguments or scripts for your to build on a remote machine.
IDLE: (I)ntegrated (D)eve(L)opment (E)nvironment
First off, that is a terrible acronym, but whatever
This is your first IDE for Python, but you should probably use...well, anything else. Like, it works, but there are so many better options out there, many of them free, that it's generally recommended not to use IDLE
Terminal vs IDLE
You will not develop in the terminal
You will not deploy from IDLE
Each of these serve drastically different purposes
As for how Python works on your computer, that's a matter of code. I can't speak to Python's lowest levels of code, but in principle what happens is you write the language in some library of code. Before you can write that code, however, you must have a common interface between the language code and the machine code. From this, they choose what functions they need from the lowest levels of the system and build a layer of abstraction that they can all reference.
When I say low-level code, we're talking stuff like Assembly, which is only a small step away from bare machine code. It reads like this:
Python:
print("Hello, world!")C: Source
Assembly: Source
So, what happens is low-level code, likely in Assembly or C gets written for deriving common functions in a higher level of abstraction, such as C or Python. From there, subsequent layers are added for commonly performed actions. These include simple things like loading values into a CPU register for computation, as well as complex actions like adding an item to a collection, which requires use of CPU registers, memory allocation, shuffling of pointers for elements in a linked list or otherwise rebuilding a new contiguous block of memory.
Eventually, you get to the layer in which Python can interact with a common interface, rather than constantly having to consider what platform it is running on. From there, you build the language on this foundation, and that is true for most any programming language.