r/Python • • Dec 18 '18

Python Virtual Environments: Extreme Advertising Edition

Post image
2.1k Upvotes

287 comments sorted by

View all comments

37

u/thisismyfavoritename Dec 18 '18

What do you think of Conda envs? Awesome post.

26

u/[deleted] Dec 18 '18

Conda is great for installing dependencies that require other system libraries. Great for scientific computing sort of stuff that deals with assorted file types.

22

u/travelinzac Dec 18 '18

They pretty much built a business deploying SciPy to windows.

4

u/wildcarde815 Dec 18 '18

They target financial HPC environments too to the tune of thousands of dollars a node for essentially a flat env space. (Or did when we talked to them a long while ago)

14

u/feindjesus Dec 18 '18

I have a strong distaste for anaconda. I have set up python on other peoples computers if they have had anaconda downloaded in the past it was a massive headache (maybe due to my inexperience). Some pip packages were not able to be found even when discoverable through pip3 list or figuring out how to change you bash.rc back to normal. As well people who always use anaconda may not develop the same unix skills. My recommendation is use pyenv with virtualenv and pyenv-virtualenv (the github tutorial is great and easy)

13

u/thisismyfavoritename Dec 18 '18

Yes, you have to check the Anaconda channels and sometimes packages are only available on pip (although they must not be standard because most of the usual dependencies are there).

However it is still possible to put everything under a env.yml and have Conda automatically call pip to install it.

Concerning UNIX skills, what do you mean? Conda can be controlled by the command line (I believe it's much better that way!).

9

u/wildcarde815 Dec 18 '18

If it's conda installable, use the conda version. If it's not, use the pip version. For the 99.99% user, this will resolve all of your problems. For the rest, congrats you are doing something both hard and arcane. Edit: and use the conda command line tool, it came before the GUI anyway and the GUI just calls the same stuff it does

2

u/s0v3r1gn Dec 19 '18

Or I just want a version of OpenCV built with certain extensions enabled...

2

u/wildcarde815 Dec 19 '18

You can rebuild it with the recipe to suit. But we've found the menpo build to be fairly good. I haven't had to dig into it in a while thankfully. I hate that god damn package.

→ More replies (5)

4

u/Spleeeee Dec 19 '18

Sounds like you are a budding Pythonista who has never had to deploy in windows worst of the worst snow-covered/dessert conditions. Don’t worry, you may never love (Ana)Conda and don’t have to, but at some point you will come to respect the Conda for its ease and simplicity in setting up (other peoples computers (not yours)).

→ More replies (3)

1

u/Verdris Dec 19 '18

What if I just need to do some math and make some graphs?

→ More replies (1)

2

u/[deleted] Dec 18 '18

It's fine if you're okay with running conda-only packages on a small list of platforms

2

u/rohit275 Dec 19 '18

l may not know what I'm talking about, but I don't get this. It seems very possible to install any pip package in a conda environment, is that not correct?

3

u/[deleted] Dec 18 '18 edited Dec 18 '18

If you need to run different versions of Python (e.g. Python 2.7 and Python 3.7) on same machine, Conda is a better choice, even so if you are on Mac

Otherwise virtualenv is simpler

(Edit: fixed typos)

1

u/RelevantToMyInterest Dec 18 '18

I'm still stuck with virtualenv but considering switching to conda. Been seeing some posts saying it's a better replacement but if also would like to know others' thoughts

5

u/xristiano Dec 18 '18

I'm a fan of conda. It does all three: pip, virtual environments, and manage python versions. The table at the bottom of this link summarizes all the pip, pyenv, etc. options.

https://www.andrey-melentyev.com/python-environments-and-where-to-find-them.html

7

u/thisismyfavoritename Dec 18 '18

Well I personally believe Conda is much better, as it can install tons of additional and system dependencies but I would like to hear OP's opinion on the matter.

4

u/ase1590 Dec 18 '18

Pipenv is the way to go if you're not using anaconda and if you want to do anything more advanced and need good package handling

See below

5

u/lifeofajenni Dec 18 '18

Wait, but conda also has environment handling and package management. So why should I switch to pipenv? (Not being snarky, legit curious.)

7

u/ase1590 Dec 18 '18

they both compete with eachother. conda is better geared for Anaconda.

Just use what you know.

Conda has a lot of overlap with the goals of pipenv, but it serves a different set of needs - in particular, the distribution of binaries. If the problem you’re trying solve is “how do I create reproducible environments, quickly, that include complex dependencies that are not written in pure Python”, conda is the tool of choice. If your problem is “How do I create and manage isolated environments for Python projects”, pipenv is the tool of choice.

3

u/lifeofajenni Dec 18 '18

Okay, this is a cool distinction. I'll have to read up on pipenv and see if it's more suitable for what I'm doing at work. Thanks for the info!

1

u/wildcarde815 Dec 18 '18

It's good to know how to use both because not everything like conda (older brew installs I'm looking at you)

1

u/pwang99 Dec 19 '18

This is a good place to start: https://www.anaconda.com/blog/developer-blog/understanding-conda-pip/

This is also a good blog post, by Jake Vanderplas: https://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/

PayPal engineering blog on why they use Anaconda & conda: https://www.paypal-engineering.com/tag/anaconda/

→ More replies (12)