r/Python • • May 27 '14

Python 3 is fine | Sealed Abstract

http://sealedabstract.com/rants/python-3-is-fine/
234 Upvotes

96 comments sorted by

57

u/Phild3v1ll3 May 27 '14

Couldn't agree more, I maintain multiple scientific software packages with a combined total of ~20k lines. Last week we decided to make the jump from 2 to 3. Within about a day we managed to port two of our projects with backwards compatibility. The third party libraries are there, 2to3 ported something like 80% of codebase correctly and the rest was trivial to fix. I feel for people who are stuck with large Python 2 codebases but if your employer can't, over the span of 5 years, devote some man-hours to make your system future proof, then blame them not Guido or the volunteers who have made the wonder that is Python in the first place.

4

u/[deleted] May 28 '14

Interesting example. I am in a similar boat (but with more LOC). The last time I tried to switch I was immediately hit with numpy / scipy bugs that made me think I must be the only one using those libraries with Python3.

How much of your stack is native code?

4

u/[deleted] May 28 '14

Example of bugs? I've never had any issues with numpy moving to python3.

4

u/[deleted] May 28 '14

Off the top of my head, crashes in the masked array module, crashes in the scipy optimization functions, going to and from pandas / numpy loss of precision (our unit tests failing).

0

u/[deleted] May 29 '14

Thanks, I will have to watch out for the precision bug.

4

u/BeatLeJuce May 28 '14

My numpy/scipy/scikit-learn stuff works fine on Python3.

2

u/AusIV Django, gevent May 28 '14

I would love to start using python 3, but there's no argument I could make that would convince my employer to let me convert our code base.

We develop against the RHEL supported python version, which currently means 2.6. We sell our software to our clients, who run a mix of our core software and custom software, some they developed and some we developed. There's no way we could force our clients to upgrade, so we would need to support two and three simultaneously.

Presently, none of our clients are even faintly interested in python 3, even if our own developers are. To move to python 3, we'd have to port at least a hundred thousand lines of code to maintain compatibility with Python 2 and 3. Maybe we can do that in a week, but developer time is awfully expensive to spend a week on something our customers aren't paying for. Since we have to support python 2 side by side, we can really take advantage of the new python 3 features in much of our code anyway.

We use a bunch of third party libraries. I'm sure most of them support python 3 by now, but a few won't, which will mean porting them ourselves.

We have pretty good test coverage, but it's not perfect, and frankly I'd be shocked if we could do this conversion without a few bugs slipping through to production. So then we've got clients calling about quality issues because we made a change they see zero benefit to.

From a business management perspective, I see a time consuming task with minimal benefit and quite a few risks. Until we have a contract with someone who wants python 3 and is willing to pay a premium for it, there's no way to convince the bean counters it's a good idea.

The way things are going at work, we're implementing more and more application logic in javascript on the browser, as it enables a more seamless user experience. Our python server applications are getting thinner, just doing data persistence and validation, and less of the work flow and presentation. I imagine that this trend will continue, and wouldn't be surprised if we start working in node.js for the server side, leaving python for a handful of legacy application that will slowly be superceded. It won't be a planned move from one language to another, rather a slow obsolescence.

Understand, I like python 3. I have a few projects on github that are 2 and 3 compatible, so I know it's not that hard to do. For personal projects, I tend to write python 3 without worrying about python 2 compatibility (which is really the only way to get the benefit of python 3).

1

u/GahMatar May 29 '14

RHEL compatiblity is the reason my employer packages a python interpreter as a custom RPM. In our case, it's currently packaging 2.5, 2.6.9 and 2.7.6 for RHEL 3 to 6 but the point stands and this could be done for python 3 as well. So you can ship your own if you need to. Since we had RHEL 3 as an official supported platform until recently, we had to do that to avoid 2.2 shudder

0

u/[deleted] May 28 '14

We develop against the RHEL supported python version, which currently means 2.6.

Sure, and if Red Hat users are your main customers, you will be stuck on Python 2 until Red Hat switches. Suck for you, but that is nothing anyone except Red Hat can do anything about. :-)

Presently, none of our clients are even faintly interested in python 3, even if our own developers are. To move to python 3, we'd have to port at least a hundred thousand lines of code to maintain compatibility with Python 2 and 3.

If I had access to a time machine, I'd stop people from calling it "porting". It's not porting. It's supporting. 90% of the time it really isn't that big of a deal. 10% of the time it's a royal pain in the ass.

0

u/[deleted] May 28 '14

[deleted]

6

u/Phild3v1ll3 May 28 '14

No it's an admission of lack of backwards compatibility, which never was a promised feature in the first place and is an issue which affects community and business driven projects alike. Blaming others for a decision, which was announced well ahead of time with an incredibly long grace period is a sign you're too slow to change rather than faulty decision making by the contributors.

0

u/[deleted] May 28 '14

[deleted]

3

u/[deleted] May 28 '14 edited May 30 '14

"Excuse it". You make it sound like it was broken lightly or by mistake.

Backwards incompatibilities were introduced because they made the language much better.

Also: Ten years of support and bugfixes apparently doesn't count as "LTS" for you.

0

u/[deleted] May 29 '14

[deleted]

2

u/[deleted] May 29 '14

You just said that it is bad to fix mistakes in a language.

That is a position that I do not find even worthy to spend time to argue against. I will just let you simmer in your own shame for a while.

Your example of apply() is particularly crappy, as it is a function that serves no purpose and has been deprecated for ages, and, whaddayouknow, it even throws a deprecation warning (if you enable them).

0

u/[deleted] May 29 '14

[deleted]

2

u/[deleted] May 29 '14 edited May 31 '14

Yes, you did. I quote:

"intentionally breaking code (e.g. "apply()") rather than throwing a warning into a log file about deprecated syntax is bad software engineering."

That means you can't fix mistakes in a language, because doing so would be intentionally break code. Many of the changes done can't warn about deprecated syntax, because the syntax is not deprecated.

(Edit: Although, as noted, the removal of apply() does have a deprecation warning, in direct contradiction to what you claimed).

0

u/[deleted] May 29 '14

[deleted]

→ More replies (0)

1

u/[deleted] May 28 '14

commercially supported languages

Python is commercially supported through a number of providers, continum, enthought, etc. What do you mean by this statement?

Other than than that - I agree with your position.

0

u/[deleted] May 29 '14

[deleted]

1

u/[deleted] May 29 '14

I think you are describing 'commercially developed' (i.e. the language).

And the rankings of the top 5 vary wildly depending on where you look.

0

u/[deleted] May 29 '14

[deleted]

3

u/[deleted] May 29 '14

My employer pays for commercial support from one of these small shops, and while they don't do language development they provide much more than "open source packages they're working on".

I work in science/engineering where MATLAB is #1 and Python is #2.

0

u/[deleted] May 29 '14

[deleted]

3

u/[deleted] May 29 '14

Sure, I never said Python wasn't a small section of the software space.

However, I would not attribute this to the language not being commercially developed - look at C, C++, javascript.

And honestly, PHP.

3

u/[deleted] May 29 '14

Also, WTF?

I get 78K for java and 29K for python, 35K for C#, 35K for C++

0

u/[deleted] May 28 '14

[deleted]

8

u/252003 May 28 '14

Internet explorer 6 works for pretty much everything my colleagues do. Yet they have upgraded to new versions of firefox/chrome.

You want to future proof your codebase. Python 2 has 5.5 years of support left and is not developing as a language, python 3 will be continued for decades and is evolving and improving.

0

u/[deleted] May 28 '14

[deleted]

14

u/dbaupp May 28 '14

Does Go have scientific libraries on the same scale as Python? (Numpy, scipy, pandas, ...)

21

u/AMorpork May 28 '14

No, it absolutely does not.

-1

u/[deleted] May 28 '14

take a look a Julia. It is still not on par with python, but it'll catch up pretty soon.

-3

u/[deleted] May 28 '14

maybe, only because the made interop with python possible.

-2

u/[deleted] May 28 '14

Of course. You can't rewrite every single python library in just a few years.

0

u/[deleted] May 28 '14

Yeah, but compare that strategy with PyPy and Python3.

1

u/[deleted] May 28 '14

What python3 strategy? python3 is just as slow as python2. And pypy is a very nice idea, but it also requires porting and rewriting every library that's written in c and also has that annoyin GIL... Don't get me wrong, I really like python, just not for technical computing.

-1

u/[deleted] May 28 '14 edited May 28 '14

Man you hates that GIL.

Anyway, my point was that Julia is being pragmatic and realises if they will convert python people they should have an interop story so codebases and be converted piecewise. PyPy basically says "C is lame, we will be faster, who uses numpy anyway" which is, lets say, a bold claim. Python3 patronizes us solid Py2 uses and made backwards incompatible changes such that you had to switch your stack over more or less at once.

I am a big fan of incremental optimization, incremental porting, and incremental interop.

Now, I really like python and use it for technical computing excessively. It is fast enough because the developer time I save building the entire stack I can, after profiling obviously, use to optimize (using any one of the many python strategies available) only the hot parts of the code. With all my remaining spare time I can bitch on reddit about how Python is slow.

1

u/rguillebert PyPy / NumPyPy May 28 '14

PyPy basically says "C is lame, we will be faster, who uses numpy anyway" which is, lets say, a bold claim.

That's not true, PyPy is just incompatible/slow with the C extension API, if the C API was more implementation independent, PyPy would be fine with it.

0

u/rguillebert PyPy / NumPyPy May 28 '14

Have you heard of PyPy STM ? http://pypy.org/tmdonate2.html

0

u/[deleted] May 28 '14

I have and sounds nice. But until pypy is actually working fine in that regard, I'll just use some other language that already works fine for what I need. I'll still use python for other things though.

15

u/ivosaurus pip'ing it up May 28 '14

python cannot make a significant progress over 5 years.

Uhhh... it has. It's people who have been whining this whole time about py2 who haven't.

38

u/doubleColJustified May 27 '14

I loved that it played an appropriate tune when one scrolls through the list of pages from what's new in Python 3.

I usually don't appreciate auto-playing music, but the way it was done here was great.

4

u/rtpg May 28 '14

In a way I feel like this is super representative of cool things you can do with the web, in terms of document presentation

2

u/SCombinator May 28 '14

The internet can shut the fuck up.

52

u/dtt-d May 27 '14 edited May 28 '14

"Python 3 is killing Python"

"Python 3 can revive Python"

"Python 3 is fine"

Well, guess there's an article for everyone! Edit: I also like how these titles seem to get progressively less passionate

46

u/ameoba May 28 '14

"Python 3 can guarantee full employment for mediocre tech bloggers"

15

u/iamtheLINAX May 28 '14

Coming up:

"Python 3 is."

"Python 3? idk"

10

u/MintyAnt May 28 '14

Coming up:

"Python 4 is killing Python"

8

u/dtt-d May 28 '14

Python 3 is meh.

4

u/iamtheLINAX May 28 '14

Eh, Python 3.

11

u/Sean1708 May 28 '14

I know this is just one man's opinion but in my experience people fight much harder against change than they do for change.

7

u/rtpg May 28 '14

"What Python 3 is"

"What Python 3 isn't"

With that we'll cover a good amount of ground

28

u/ThePurpleAlien May 28 '14

I'm glad this was posted in response to "python 3 is killing python". We've just decided to move our simulation software to Python 3 and the previous article had me a bit worried about Python's fate. So it's good to hear a robust defense of Python 3, but there's also a bit of attitude in this post that I don't like: the idea that Python 3 is just the product of a bunch of volunteers coders having fun, why should they care about you and your problems with Python 2/3? You do what you want to do (including forking Python 2 if that floats your boat); let the Python devs do whatever floats their boat.

But this attitude is actually not ok. Python is not just a fun experiment; it is a successful language as the Python.org website is eager to illustrate in the "Success Stories" section. Many businesses depend on it, a lot of money and innovation rides on it. And evidently, the python community thinks that's a good thing, it's maybe even a bit smug about it. So it's not acceptable to say that Python's devs can just hack away in a bubble of self interest. While Python 3 is hopefully not actually killing Python, ignoring the concerns of everyday Python users will kill Python. According to this author, that's ok because the Python devs are just volunteers having fun. But I don't think those devs would say that's ok. Python's success is a big part of what makes all their work relevant. So even if resistance to Python 3 is misguided, it is somewhat pervasive, and the Python community ought to have a good (i.e. not merely dismissive and patronizing) answer to those concerns.

12

u/shadowmint May 28 '14

yep; this /r/circlejerk approach to the issue (oh, I work in python 3, and so do all my friends, its great!) is really the problem.

I think its particularly telling that Guido himself works primarily in python 2 at Dropbox, and unsurprisingly we now have support for the 2.7 line in security patches to 2020. At least a few of the core developers must be awake.

The OP raises a valid point. It is an open source project. If you want change, you have to actually get involved, not just complain.

...that said, is encouraging the complainers to fork and start 2.8 really a good idea? (no, it's not; its a flat out terrible idea. It'll divide the community even more). Making python 3 compellingly better than the 2.x line is really the path forward.

4

u/[deleted] May 28 '14

that said, is encouraging the complainers to fork and start 2.8 really a good idea?

He is not actually encouraging them. The "Python 3 is killing Python" is. This response is just saying "If that's such a great idea, why don't you do it?"

The fact is that this has been discussed many times before. In January there was an actual call for this, and an IRC channel was created. Nobody have posted in it since January, despite the channel having several people claiming that a Python 2.8 fork should happen.

That's his point. He is not encouraging it, he said that if it was such a great idea it would have happened by now.

(The main problem with 2.8 is of course that nobody knows what should go in it).

0

u/djimbob May 28 '14

In 2020 when python2 is at EOL, it may make sense to fork to keep the python2 line with security patches (not to add new features) or add python support/binaries for some new OS/architecture that didn't exist in 2014.

Not to encourage new projects/work in it, but just for people who need to maintain legacy python2 programs. (E.g., need to re-run the old unmodified code in a long running experiment). There's absolutely no need to do it now, and while someone could try -- I doubt anyone would gain any traction doing so.

Granted then again, things like trollius that backport specific features from python3 into python2 seem like worthwhile endeavors if you are currently using python2 and want asyncio, but can't/don't want to go python3 yet. These projects are the result of forking and renaming the project. Granted it won't be python2.8, it will be python3's blah feature implemented as a library.

-1

u/realsw May 28 '14

By 2020 hopefully we'll have tools to convert any language to any other language, so we won't have to worry about all these divisions - programming will be universal. :-)

8

u/klbcr May 28 '14 edited May 28 '14

See, you're expecting volunteers to make a powerful free language which businesses use commercially, and then when they want to change it, you expect them to do it in a way that is commercially viable and easy for those businesses?

Why don't you expect the businesses that make money using free and open source software to donate money and always make sure to steadily port their code to newer versions of python? Because it's too expensive? Because it's cheaper to let the community of volunteers do it for them for free? I think that's ridiculous.

And when their code becomes old and unmaintanable because they didn't want to spend money earlier, they complain that it's the communities fault for their problems? Why should the volunteers care? They make what they want to use, they aren't employed by the businesses. Should they hold back their ideas and their project because it's too expensive for some business?

4

u/ThePurpleAlien May 28 '14

Why should the volunteers care? They make what they want to use, they aren't employed by the businesses. Should they hold back their ideas and their project because it's too expensive for some business?

You're right. The devs have every right to just do what they want. They're not beholden to my bottom line. But my point is, if they do that, that means they're ok with Python not being useful to other people, and they're ok with Python's user base dwindling away. But if the Python community was ok with that, they wouldn't brag about Python's success. Python.org wouldn't have a gigantic "Success Stories" section. Python wouldn't bother to promote itself at all, because it's just a bunch of volunteer devs having fun. But that's not the case. You can't have it both ways. If Python's success among the broader programming community is a good thing, then you can't just say "here's a phone, call someone who cares" when that community raises concerns.

3

u/klbcr May 28 '14

that means they're ok with Python not being useful to other people,

No, that just means they put the project and the vision of the project above what commercial business users want it to be to suit their own commercial and financial means. Python remains useful to other people, it's just that those people aren't the ones that are tied up with legacy code, and are willing to change and make better, new things. And it's not like the core devs are forbidding the use of older python versions. It's not like 2x is suddenly going to become unavailable and apps all over the world are going to crash. Older versions are available and they can be forked, and modified and hacked. It's just that they, the core devs, want to move onto other things, better things. If bussineses have a financial interest in maintaining old code then they should be willining to spend money on it and either do it themselves or pay someone to else to do it. They should be proactive about it instead of expecting the core developers and the community to do it for them.

and they're ok with Python's user base dwindling away.

Is there any evidence of this trend?

5

u/ThePurpleAlien May 28 '14

I'm not disagreeing. If I were a Python dev, I would want to make a clean break from Python 2 and develop new innovations in Python 3. Backward compatibility sucks for keeping a language clean and efficient. And that's what they did, and now the core Python community is talking about all the great things Python 3 can do. But that is naturally going to put off folks who manage production code bases in Python 2. Those people, who otherwise love Python, get the shaft. It's very hard for them to justify the risk and cost of upgrading working code. So if you're going to promote Python 3, you have to be a bit sensitive to the fact that a lot of Python users are going to say "that's great, but it doesn't benefit me at all."

My take on this is that Python 3 proponents have to stop pushing Python 3 on people. And pushy is how it comes across. Everyone has to weigh the costs/benefits of moving to Python 3 for their own purposes. Some speaker at pycon, no matter how authoritative, does not have a valid say over whether I should port my code from 2 to 3. If you want people to do something, don't tell them to do it, make it as easy as possible (e.g. 2to3.py), and then let them decide if/when to do it.

0

u/Veedrac Jun 01 '14 edited Jun 01 '14

My take on this is that Python 3 proponents have to stop pushing Python 3 on people.

From the article:

Not everybody can forget about Python 2 completely right now, but not everybody needs to move to Python 3 at all, or to use any particular programming language for that matter. Go do what makes you happy.

What makes me happy is using Python 3.


The only reason people seem to do this is because people attack Python 3 with so much fervour.

Move when you want. There are masses of support for people using Python 2. Nobody sane is arguing with that. We're just asking you to allow people not tied down to be able to use version 3 without having a hundred "Python 3 sucks", "Python 3 can't do bytes", "Python 3 is killing Python", "Nobody uses Python 3" posts thrown at us.

6

u/brechtm May 28 '14 edited May 28 '14

In 2008 I started a Python 3-only project. Now, as the project is finally maturing, I feel the need for it to run also on Python 2, as I'd like to reach an audience that is as large as possible. Note that next to CPython 2, there's also Jython and IronPython that both seem to be stuck in Python 2.x land (but PyPy3 is in fairly good shape, it seems).

I did manage to backport the code to Python 2 some time ago, making sure the same codebase still runs on Python 3. Unfortunately, there are some things that make this unpleasant at the least. For example, metaclass definitions need to be specified in different ways in 2/3. It would be great if Python 2 would also support the Python 3 syntax. Similar to the re-addition of the unicode literal u"" to Python 3 and "from future import unicode_literals" in Python 2, this would be a minor change that could greatly simplify having a unified 2/3 codebase. Also, the Py2 str / Py3 unicode thing is very tricky to get right, but I'm not sure how this could be simplified.

There are also some Python 3 features that are not present in Python 2 which cannot be remedied by a third-party library. This stops me from adopting these Python 3 features. One example is Python 3's function annotation capability (which I'd like to use for type checking). Again, having support for these in Python 2 (even if they are simply parsed and ignored) would be great.

I think that the move to the (backward-incompatible) Python 3 was a good one, but in hindsight, perhaps more thought/effort should have been put into making the upgrade path as smooth as possible. Some tools have been given to the Python developer over the years (from future import <py3_thing>, u"") to help with this, but I think they should have been available from the start (yes, easy to say now). Something to remember for Python 4 or 5?

2

u/[deleted] May 28 '14

Supporting the new metaclass syntax is indeed one of the few additions to Python 2 that would make sense. Backporting the bytesclass is another.

Backporting the new features is highly unlikely to happen, simply for the reason that nobody is interested in doing it, partly because that would lessen the incentive to move to Python 3.

2

u/moor-GAYZ May 28 '14

partly because that would lessen the incentive to move to Python 3.

Well, they are shooting themselves in the foot if that's what they are thinking.

Python is a platform, which means that it derives its value not directly from the applications to which it gives value, but through libraries. Which makes migration pretty awkward, since libraries must support both versions for quite a while until most applications have migrated.

Which, in turn, results in a counter-intuitive need for Python3 compatibility in Python2 (on the language level, as libraries, as 3to2.py) as a requirement for smooth migration to Python3. Because you want people's libraries to be written in Python3 and backported to Python2, not remaining a Python2 codebase on which 2to3.py and a bunch of custom patches is run once a month.

On the other hand, when the time comes, nobody will say "I can't be bothered to install Python3 in order to remove all those from future import's and compatibility libraries and stuff that I added to be compatible with Python3, I'm just fine with awkwardly using Python3 features in Python2". That's a complete non-concern compared to the very real hurdles that delay Python3 adoption.

0

u/[deleted] May 28 '14

Well, they are shooting themselves in the foot if that's what they are thinking.

A lot of people do claim they stay on Python 2 because Python 3 doesn't offer any incentive to move, so that's why they are thinking like that, I'd guess.

Python is a platform, which means that it derives its value not directly from the applications to which it gives value, but through libraries. Which makes migration pretty awkward, since libraries must support both versions for quite a while until most applications have migrated.

Right. And having more new features in Python 2 is not going to change that.

Because you want people's libraries to be written in Python3 and backported to Python2, not remaining a Python2 codebase on which 2to3.py and a bunch of custom patches is run once a month.

That's really only relevant if you still want to support Python 2.5. Otherwise you don't use 2to3 anymore, but use a common codebase.

Yes, if you want Python 2 support, then you can't use the Python 3 only features. But that is only a problem for people who are already mainly using Python 3, and then afterwards discover that Python 2 might be nice as well. And that is currently a very small group of people.

I can't be bothered to install Python3 in order to remove all those from future import's and compatibility libraries and stuff that I added to be compatible with Python3, I'm just fine with awkwardly using Python3 features in Python2.

Huh? I'm sorry, that makes no sense.

  1. You don't need to remove the things you added to be compatible with Python 3. I'm not sure what you are trying to say with that.

  2. If you added things to be compatible with Python 3, you presumably installed Python 3. How else would you know that you are compatible?

3

u/moor-GAYZ May 28 '14

A lot of people do claim they stay on Python 2 because Python 3 doesn't offer any incentive to move, so that's why they are thinking like that, I'd guess.

Nope, most people have no problem installing the latest 2.x version, outside of rare corporate environments. I've never seen anyone discussing whether Python2.7 is killing Python, compared to the Python3 migration this is a complete non-issue, a red herring.

What people who say that Python 3 doesn't offer any incentive to move mean is that it doesn't offer enough incentives to overcome the significant hurdles. Remove the hurdles and this objection would disappear.

libraries must support both versions for quite a while until most applications have migrated.

Right. And having more new features in Python 2 is not going to change that.

This is going to make supporting both versions much less painful.

That's really only relevant if you still want to support Python 2.5. Otherwise you don't use 2to3 anymore, but use a common codebase.

Which is painful, otherwise we wouldn't even have this discussion.

I'm not talking about backporting all new features, I'm talking about backporting all backward incompatible changes and changes required to write basic Python3 code, to be enabled with from future import, ideally including renames and everything that six.py takes care of.

The idea is that a library maintainer should be able to port his library to Python3, then insert "from future import py3_compatibility" and have it running on Python2.7 and Python2.6, whatever they used before, with no further code changes. If that were possible then the whole Python3 migration conundrum simply would have not existed.

The only choices that library maintainers would have to make is whether to use actual new Python3 features, same they have to make re: collections.namedtuple wrt Python2.5 compatibility etc. That would be their private business and have no effect on Python3 adoption.

"I can't be bothered to install Python3 in order to remove all those from future import's and compatibility libraries and stuff that I added to be compatible with Python3, I'm just fine with awkwardly using Python3 features in Python2."

Huh? I'm sorry, that makes no sense.

Well, you said that backporting Python3 features to Python2.8 where they are enabled with from __future__ import and compatibility shims "would lessen the incentive to move to Python 3". So there's this guy who writes de-facto Python3 compatible code using all that stuff but refuses to move to actual Python3. That makes no sense, my point exactly.

0

u/[deleted] May 28 '14

A lot of people do claim they stay on Python 2 because Python 3 doesn't offer any incentive to move, so that's why they are thinking like that, I'd guess. Nope, most people have no problem installing the latest 2.x version, outside of rare corporate environments. I've never seen anyone discussing whether Python2.7 is killing Python, compared to the Python3 migration this is a complete non-issue, a red herring.

What? I don't understand the relation to what you and me said before.

That's really only relevant if you still want to support Python 2.5. Otherwise you don't use 2to3 anymore, but use a common codebase. Which is painful, otherwise we wouldn't even have this discussion.

Going up in the morning is painful. It's all relative. Using the common codebase for Python 2.7 and Python 3.3 is not really very painful.

I wrote Pyroma for Python 2. Then I added Python 3 support with 2to3, with almost no changes. Later I decided using 2to3 was silly, and went onto a common codebase. Here's the changeset:

https://bitbucket.org/regebro/pyroma/commits/b81cbe0085ff0bb71fd048b011808a5aee616867

When I wrote Hovercraft, I wrote that for Python 3. There are reports that it works under Python 2 as well, if you just add argparse as a dependency. So it seems to work with no changes. I just wrote in a "common codebase" automatically.

It's really not very painful. Especially if you use six. The common codebase is pretty much Python 3 coding, with some future imports, and lacking some of the new Python 3 features like "yield from".

I'm not talking about backporting all new features, I'm talking about backporting all backward incompatible changes and changes required to write basic Python3 code, to be enabled with from future import

So in fact, what you are talking about when you say "new features" is "not new features". Yeah. That was clear. I have previously answered you as if you actually meant "new features", which is where much of the confusion is.

The question then becomes exactly what incompatible changes you want to backport.

ideally including renames and everything that six.py takes care of.

Sure, but then again, you can just use six for that. So that's not an actual hurdle, and you mentioned hurdles. ;-)

1

u/moor-GAYZ May 28 '14

Sure, but then again, you can just use six for that.

It sucks to use six.iteritems(d) and the like.

0

u/[deleted] May 28 '14

Well, it's ugly, sure. "Sucks", maybe not.

And that is an interesting example, because what do you want Python 2.8 to do about it, specifically?

1

u/moor-GAYZ May 28 '14

Well, it's ugly, sure. "Sucks", maybe not.

So the problems with Python3 adoption are caused by something else or don't exist?

And that is an interesting example, because what do you want Python 2.8 to do about it, specifically?

That's an interesting question, now that I think about it. I guess a dirty hack to make such objects present different methods to code compiled with that from __future__ import. It can break inheriting from them, but so would a milder method (implement .itemview() etc). Still, it would work for 99.99% of the code and more or less cleanly break for the rest.

Or maybe for that using something like "six" was inevitable.

Anyway it would have been even nicer if this stuff were implemented in 2.6.0 five years ago, before the 3.0 release actually. Then we'd all be using 3.x by now. Unfortunately, the Python core developers apparently believed that they could achieve a clean jump from 2 to 3 somehow, the idea that people would have to make code that works in both never crossed their minds. Thus the dances with unicode literals etc.

0

u/[deleted] May 29 '14

So the problems with Python3 adoption are caused by something else or don't exist?

The slow Python 3 adaptation is NOT because dict.iteritems() is gone, no.

That's an interesting question, now that I think about it. I guess a dirty hack to make such objects present different methods to code compiled with that from future import.

Right, what is the future import supposed to do? Have different dict types? Make the type behave differently in that specific module? (That's tricky at best.)

The only option there was really to change the behavior of dict values, and hence break backwards compatibility. This is one of the changes that can't be done in a good backwards compatible way.

3

u/[deleted] May 28 '14 edited May 28 '14

A lot of people do claim they stay on Python 2 because Python 3 doesn't offer any incentive to move, so that's why they are thinking like that, I'd guess.

On the contrary: businesses cannot afford to stop development on new features to move their code base to a new version of python. To do so would erode their market edge. Businesses cannot afford the upgrade path. Python 3's upgrade path really needs to improve.

The only other thing that would compel businesses to upgrade is a 'gee-wiz' game changer feature that wasn't in python 2: for example a JITed GIL-less python with performance nearing c or an extremely easy-to-use threading model the works on multiple cores that isn't a multiprocessor hack. This makes the upgrade cost affordable because there is a high value in moving.

Right now, python 3 feels more like the developers decided to refactor the code and have added some bug fixes on top of it. And they are holding those bug fixes hostage from python 2. In addition, they're spreading propaganda (in some cases it feels like they're beating our heads with how much better python 3 is) about how much better python 3 is but not highlighting much value difference between the two.

Of course businesses are not going to jump on a new version. Why would they when the cost to move is so high compared to little value? The choice is: add 6 months of features vs refactor code for 6 months with no real difference (except possibly a performance slow-down) and allow the competition to catch up. Why would any business switch?

Edit: And here's the part that I think is really scary. While python 3 developers have been refactoring python 2, other languages have highlighted the major issues with python (e.g. threading model, GIL, multi-core processing, etc.). They've come up with performant solutions for these problems, and people are starting to move away from python. Even the python developers are eyeing these new languages and looking at them as something interesting to develop.

As much as I enjoy python and have over the past decade, even my personal projects are starting to move away from python and into something more performant. The python development has basically stalled over the past 5 years and its no wonder businesses are now starting to move away from it.

0

u/[deleted] May 28 '14

A lot of people do claim they stay on Python 2 because Python 3 doesn't offer any incentive to move, so that's why they are thinking like that, I'd guess. On the contrary: businesses cannot afford to stop development on new features to move their code base to a new version of python.

I don't see how that relates to what I said.

To do so would erode their market edge. Businesses cannot afford the upgrade path. Python 3's upgrade path really needs to improve.

OK, so exactly what is the problems with the upgrade path and how can we improve it?

The only other thing that would compel businesses to upgrade is a 'gee-wiz' game changer feature that wasn't in python 2: for example a JITed GIL-less python with performance nearing c or an extremely easy-to-use threading model the works on multiple cores that isn't a multiprocessor hack. This makes the upgrade cost affordable because there is a high value in moving.

You are now arguing against my claim that backporting new features to Python 2 would lessen the incentive to move to Python 3, with saying that there are not enough features unique to Python 3 so there is not enough incentive.

Which to me seems like you are in fact agreeing with what I said.

Right now, python 3 feels more like the developers decided to refactor the code and have added some bug fixes on top of it.

Well, Python 3.0 did yes. That's pretty much exactly what it was. But new features are being added to Python 3, you know. ;-)

And they are holding those bug fixes hostage from python 2.

No, that's absolute bullshit. Please do not discuss this issue if you don't know what you are talking about.

In addition, they're spreading propaganda (in some cases it feels like they're beating our heads with how much better python 3 is) about how much better python 3 is but not highlighting much value difference between the two.

Again you are saying there are not enough difference as an incentive. As a response to me saying that backporting features to Python 2 would lessen the incentive.

2

u/[deleted] May 28 '14

Maybe I can break this down differently. There are two basic pain points:

  1. There is a real cost associated with moving a large code base from python 2 to python 3. This cost is measured in months of refactoring.

  2. There is little value / incentive for a business to move from python 2 to python 3.

These are both non-negligible, real problems that a developer could dismiss, but a business simply cannot.

Fixing either pain point would improve Python 3 adoption rate. However, improving the upgrade path would be far easier to develop than creating a wiz-bang feature that pulls companies to python 3. The wiz-bang feature would likely need to be something to vastly improve multi-core processing -- both in development time as well as maintenance. For example, the tulip rehash of twisted didn't make python easier to use. The mental gyrations needed to use either library are still ridiculous and error-prone.

The best way to improve the upgrade path would be to create a version of python (2.8) that straddled the new features/bug fixes of python 3 but was backward compatible with python 2.7. This would allow businesses to systematically improve code over time and take the cost associated with moving from python 2.7 to python 3.x incrementally rather than in one-shot.

2

u/[deleted] May 28 '14

There is a real cost associated with moving a large code base from python 2 to python 3. This cost is measured in months of refactoring.

"Months".

There is little value / incentive for a business to move from python 2 to python 3.

Right.

Fixing either pain point would improve Python 3 adoption rate.

Sure.

However, improving the upgrade path would be far easier to develop than creating a wiz-bang feature that pulls companies to python 3.

Sure.

the tulip rehash of twisted

:rolleyes:

The mental gyrations needed to use either library are still ridiculous and error-prone.

multi-core processing is not a trivial problem. Nothing will magically appear that makes it so.

The best way to improve the upgrade path would be to create a version of python (2.8) that straddled the new features/bug fixes of python 3 but was backward compatible with python 2.7.

Please note that backporting new features contradicts your second point. It would make the incentive to move to Python 3 even less. Also, it would not reduce the pain, as it does not help the upgrade path.

Bugfixes are already backported, so that's a moot point.

If you are saying that you want to backport some of the incompatible changes in a compatible way, I will ask you to state exactly which changes you want to backport.

1

u/[deleted] May 28 '14

Clearly wasted effort on my part.

0

u/[deleted] May 29 '14

Yes, unless you can actually say exactly what you want included in a Python 2.8, calling for it is a wasted effort.

People who call for a Python 2.8 often seem to just assume that not only can they demand that somebody else does the job, but also that these people haven't already considered a 2.8 seriously.

0

u/[deleted] May 29 '14

Some tools have been given to the Python developer over the years (from future import <py3_thing>, u"") to help with this, but I think they should have been available from the start (yes, easy to say now).

Just FYI: Almost all of them were. u'' were not, though.

4

u/[deleted] May 28 '14

Why are people surprised/angry that Python 3 is not fully backwards-compatible? Breaking API changes are what major versions represent according to semver. angular.js 2.0 is coming out later this year and I expect I'll have to modify my projects to switch.

Using __future__ now can save you a lot of time later when you decide to move to Python 3.

6

u/erewok May 28 '14

I couldn't agree more and I am glad to see an appropriate response to that other poorly argued piece.

4

u/vacuu May 28 '14

As far as the language itself, both 2 and 3 are good in my opinion.

Where I think python 3 really missed the mark was not taking the opprotunity to fix the import system and distribution system.

2

u/[deleted] May 28 '14

The distribution system is independent of the Python version, and is being fixed. Attempts have been ongoing since before Python 3. It's a difficult issue.

3

u/[deleted] May 28 '14

Python 3 might be fine, but I see no compelling reason to commit time and effort into moving to it. Sure, it has plenty of "exciting" features, but I'm a professional developer who writes boring code meant to last for years, if not decades. "Exciting" new features look fragile and unproven to me.

The OSes I develop for (and on) have Python 2 distributed with them, and provide hundreds of pre-compiled packages for them. Going through the effort of compiling my own packages and having to fight against the OS' version of Python just for a few new language features sounds like tilting at windmills to me.

As we're fond of saying in defense of our language... Python is good enough. I'll take that and extend it a bit: Python 2 is good enough for now.

0

u/Veedrac Jun 01 '14

"Exciting" new features look fragile and unproven to me.

If Python 3's features are "unproven" (maybe other than asyncio) to you, you better not release any code until millions of people have used it rigorously for five years.

1

u/mipadi May 27 '14

Well, at least that was appropriately filed under "Rants" on the author's website.

1

u/ayanami_rei EVA pilot May 28 '14

Great post, I strongly agree.

People who spout nonsense about Python 3 mostly don't write useful code (if at all), so their opinion should be discarded in a most humiliating way, and they should feel bad.

I've never seen actual developers complain that "bawww Python is dying because Python 3".

-2

u/[deleted] May 28 '14

The fact that there are a number of changes from Python 2 to 3 does not imply that there are a number of compelling changes from Python 2 to 3. It also does not imply that there were not steps backward.

It was a nice touch to autoplay music while making such a shit attempt at an argument. It ensured that people who weren't annoyed by the argumentation itself were still annoyed by the presentation.

With all the shit-talking about other bloggers I expected a little more. Glass houses.

Really though, this article misses multiple points. Just use Python 2? We do. The problem is that the future of the language is in the hands of people in whom we have no confidence to steer the language well. Volunteers or not, that's an issue.

I agree that it's easy to fork and make a Python 2.8. Or if not easy at least doable with the right set of resources behind. But how does this fix the direction of Python? How do we choose one of the 14 random attempts at Python 2.8 that will pop up next week? The only way that doesn't fragment the Python community further is if the people controlling the project do it, and they have no interest.

0

u/lykwydchykyn May 28 '14

This rant pretty much applies to any open source project that people get upset about.

-8

u/drive0 May 28 '14

If you have to keep convincing yourself that its fine, well good for you.

Despite what it may seem like, I want python 3 to win. But since I'm a programmer, I can move to golang, or objective c, or java, or pretty much any language without much trouble.

edit: the real elephant in the room is that python3 should have either been a new language, or it should have been an iteration. It is neither and has suffered.

1

u/chasecaleb May 28 '14

It is an iteration. The version number iterated from two to three. I'm not sure what you mean.

-4

u/drive0 May 28 '14

Being pedantic doesn't actually make python3 better. Python3 should have had real concurrency and probably a real http library (though that is easily solved with a 3rd party library.)

1

u/LightShadow 3.13-dev in prod May 28 '14

But we're asynchronous now like everyone else. /s

0

u/[deleted] May 28 '14

The point of Python 3 is to fix problems in the language itself. That's also what causes the problems. New libraries are not a problem and not a solution. :-)

2

u/drive0 May 29 '14

So, GIL?

0

u/[deleted] May 29 '14

Not a problem in the language, but in the implementation. Also not a big problem. Also crazy hard to solve.

-1

u/staz May 28 '14

One of the great strengths of Python 2 is the long tail of 3rd party libraries to do, well, just about anything. Python 3 doesn’t have this.

It is time for this talking point to die. Of the top 200 3rd party libraries, 80% have Python 3 support at the time of this writing. Of the remaining packages, many are things like supervisor and Fabric that are stand-alone applications where nobody cares what version of Python they run. To say that Python 3 “doesn’t have” 3rd party libraries is not just wrong, it’s completely absurd.

The author doesn't have any fucking clue what a Long Tail is; because the Long Tail is not the Top 200, it's the exact opposit, it's the metric tons of other packages which are only used by a few people. And most of thoses don't support Python 3 yet.

0

u/[deleted] May 28 '14

I noticed this as well. I think shows that the author is in a defensive mood, and is not really considering the points the article is making, but rather rushing to refute the claims whatever they are without reading them properly.

0

u/[deleted] May 27 '14

[deleted]

11

u/[deleted] May 28 '14

Or they have no incentive to upgrade. That's not laziness.

14

u/john_m_camara May 28 '14

You assume everyone who has not ported over to Python 3 is lazy and i'll assume everyone who makes that statement is naive and can't be taken seriously.

5

u/codekoala May 28 '14

We're programmers. We're all lazy, by necessity. Automate all teh things!

1

u/LightShadow 3.13-dev in prod May 28 '14

Automation Engineer checking in. I love my job.

0

u/[deleted] May 28 '14

i'll assume everyone who makes that statement is naive and can't be taken seriously.

ie: hobbyist, picked up programming last month, wrote a web scraper and now fancies themselves an expert. :P

-1

u/[deleted] May 28 '14

Not the only reason. Some depends on libraries that are hard to make support Python 3.