In 2008 I started a Python 3-only project. Now, as the project is finally maturing, I feel the need for it to run also on Python 2, as I'd like to reach an audience that is as large as possible. Note that next to CPython 2, there's also Jython and IronPython that both seem to be stuck in Python 2.x land (but PyPy3 is in fairly good shape, it seems).
I did manage to backport the code to Python 2 some time ago, making sure the same codebase still runs on Python 3. Unfortunately, there are some things that make this unpleasant at the least. For example, metaclass definitions need to be specified in different ways in 2/3. It would be great if Python 2 would also support the Python 3 syntax. Similar to the re-addition of the unicode literal u"" to Python 3 and "from future import unicode_literals" in Python 2, this would be a minor change that could greatly simplify having a unified 2/3 codebase. Also, the Py2 str / Py3 unicode thing is very tricky to get right, but I'm not sure how this could be simplified.
There are also some Python 3 features that are not present in Python 2 which cannot be remedied by a third-party library. This stops me from adopting these Python 3 features. One example is Python 3's function annotation capability (which I'd like to use for type checking). Again, having support for these in Python 2 (even if they are simply parsed and ignored) would be great.
I think that the move to the (backward-incompatible) Python 3 was a good one, but in hindsight, perhaps more thought/effort should have been put into making the upgrade path as smooth as possible. Some tools have been given to the Python developer over the years (from future import <py3_thing>, u"") to help with this, but I think they should have been available from the start (yes, easy to say now). Something to remember for Python 4 or 5?
Supporting the new metaclass syntax is indeed one of the few additions to Python 2 that would make sense. Backporting the bytesclass is another.
Backporting the new features is highly unlikely to happen, simply for the reason that nobody is interested in doing it, partly because that would lessen the incentive to move to Python 3.
partly because that would lessen the incentive to move to Python 3.
Well, they are shooting themselves in the foot if that's what they are thinking.
Python is a platform, which means that it derives its value not directly from the applications to which it gives value, but through libraries. Which makes migration pretty awkward, since libraries must support both versions for quite a while until most applications have migrated.
Which, in turn, results in a counter-intuitive need for Python3 compatibility in Python2 (on the language level, as libraries, as 3to2.py) as a requirement for smooth migration to Python3. Because you want people's libraries to be written in Python3 and backported to Python2, not remaining a Python2 codebase on which 2to3.py and a bunch of custom patches is run once a month.
On the other hand, when the time comes, nobody will say "I can't be bothered to install Python3 in order to remove all those from future import's and compatibility libraries and stuff that I added to be compatible with Python3, I'm just fine with awkwardly using Python3 features in Python2". That's a complete non-concern compared to the very real hurdles that delay Python3 adoption.
Well, they are shooting themselves in the foot if that's what they are thinking.
A lot of people do claim they stay on Python 2 because Python 3 doesn't offer any incentive to move, so that's why they are thinking like that, I'd guess.
Python is a platform, which means that it derives its value not directly from the applications to which it gives value, but through libraries. Which makes migration pretty awkward, since libraries must support both versions for quite a while until most applications have migrated.
Right. And having more new features in Python 2 is not going to change that.
Because you want people's libraries to be written in Python3 and backported to Python2, not remaining a Python2 codebase on which 2to3.py and a bunch of custom patches is run once a month.
That's really only relevant if you still want to support Python 2.5. Otherwise you don't use 2to3 anymore, but use a common codebase.
Yes, if you want Python 2 support, then you can't use the Python 3 only features. But that is only a problem for people who are already mainly using Python 3, and then afterwards discover that Python 2 might be nice as well. And that is currently a very small group of people.
I can't be bothered to install Python3 in order to remove all those from future import's and compatibility libraries and stuff that I added to be compatible with Python3, I'm just fine with awkwardly using Python3 features in Python2.
Huh? I'm sorry, that makes no sense.
You don't need to remove the things you added to be compatible with Python 3. I'm not sure what you are trying to say with that.
If you added things to be compatible with Python 3, you presumably installed Python 3. How else would you know that you are compatible?
A lot of people do claim they stay on Python 2 because Python 3 doesn't offer any incentive to move, so that's why they are thinking like that, I'd guess.
On the contrary: businesses cannot afford to stop development on new features to move their code base to a new version of python. To do so would erode their market edge. Businesses cannot afford the upgrade path. Python 3's upgrade path really needs to improve.
The only other thing that would compel businesses to upgrade is a 'gee-wiz' game changer feature that wasn't in python 2: for example a JITed GIL-less python with performance nearing c or an extremely easy-to-use threading model the works on multiple cores that isn't a multiprocessor hack. This makes the upgrade cost affordable because there is a high value in moving.
Right now, python 3 feels more like the developers decided to refactor the code and have added some bug fixes on top of it. And they are holding those bug fixes hostage from python 2. In addition, they're spreading propaganda (in some cases it feels like they're beating our heads with how much better python 3 is) about how much better python 3 is but not highlighting much value difference between the two.
Of course businesses are not going to jump on a new version. Why would they when the cost to move is so high compared to little value? The choice is: add 6 months of features vs refactor code for 6 months with no real difference (except possibly a performance slow-down) and allow the competition to catch up. Why would any business switch?
Edit: And here's the part that I think is really scary. While python 3 developers have been refactoring python 2, other languages have highlighted the major issues with python (e.g. threading model, GIL, multi-core processing, etc.). They've come up with performant solutions for these problems, and people are starting to move away from python. Even the python developers are eyeing these new languages and looking at them as something interesting to develop.
As much as I enjoy python and have over the past decade, even my personal projects are starting to move away from python and into something more performant. The python development has basically stalled over the past 5 years and its no wonder businesses are now starting to move away from it.
A lot of people do claim they stay on Python 2 because Python 3 doesn't offer any incentive to move, so that's why they are thinking like that, I'd guess. On the contrary: businesses cannot afford to stop development on new features to move their code base to a new version of python.
I don't see how that relates to what I said.
To do so would erode their market edge. Businesses cannot afford the upgrade path. Python 3's upgrade path really needs to improve.
OK, so exactly what is the problems with the upgrade path and how can we improve it?
The only other thing that would compel businesses to upgrade is a 'gee-wiz' game changer feature that wasn't in python 2: for example a JITed GIL-less python with performance nearing c or an extremely easy-to-use threading model the works on multiple cores that isn't a multiprocessor hack. This makes the upgrade cost affordable because there is a high value in moving.
You are now arguing against my claim that backporting new features to Python 2 would lessen the incentive to move to Python 3, with saying that there are not enough features unique to Python 3 so there is not enough incentive.
Which to me seems like you are in fact agreeing with what I said.
Right now, python 3 feels more like the developers decided to refactor the code and have added some bug fixes on top of it.
Well, Python 3.0 did yes. That's pretty much exactly what it was. But new features are being added to Python 3, you know. ;-)
And they are holding those bug fixes hostage from python 2.
No, that's absolute bullshit. Please do not discuss this issue if you don't know what you are talking about.
In addition, they're spreading propaganda (in some cases it feels like they're beating our heads with how much better python 3 is) about how much better python 3 is but not highlighting much value difference between the two.
Again you are saying there are not enough difference as an incentive. As a response to me saying that backporting features to Python 2 would lessen the incentive.
Maybe I can break this down differently. There are two basic pain points:
There is a real cost associated with moving a large code base from python 2 to python 3. This cost is measured in months of refactoring.
There is little value / incentive for a business to move from python 2 to python 3.
These are both non-negligible, real problems that a developer could dismiss, but a business simply cannot.
Fixing either pain point would improve Python 3 adoption rate. However, improving the upgrade path would be far easier to develop than creating a wiz-bang feature that pulls companies to python 3. The wiz-bang feature would likely need to be something to vastly improve multi-core processing -- both in development time as well as maintenance. For example, the tulip rehash of twisted didn't make python easier to use. The mental gyrations needed to use either library are still ridiculous and error-prone.
The best way to improve the upgrade path would be to create a version of python (2.8) that straddled the new features/bug fixes of python 3 but was backward compatible with python 2.7. This would allow businesses to systematically improve code over time and take the cost associated with moving from python 2.7 to python 3.x incrementally rather than in one-shot.
There is a real cost associated with moving a large code base from python 2 to python 3. This cost is measured in months of refactoring.
"Months".
There is little value / incentive for a business to move from python 2 to python 3.
Right.
Fixing either pain point would improve Python 3 adoption rate.
Sure.
However, improving the upgrade path would be far easier to develop than creating a wiz-bang feature that pulls companies to python 3.
Sure.
the tulip rehash of twisted
:rolleyes:
The mental gyrations needed to use either library are still ridiculous and error-prone.
multi-core processing is not a trivial problem. Nothing will magically appear that makes it so.
The best way to improve the upgrade path would be to create a version of python (2.8) that straddled the new features/bug fixes of python 3 but was backward compatible with python 2.7.
Please note that backporting new features contradicts your second point. It would make the incentive to move to Python 3 even less. Also, it would not reduce the pain, as it does not help the upgrade path.
Bugfixes are already backported, so that's a moot point.
If you are saying that you want to backport some of the incompatible changes in a compatible way, I will ask you to state exactly which changes you want to backport.
Yes, unless you can actually say exactly what you want included in a Python 2.8, calling for it is a wasted effort.
People who call for a Python 2.8 often seem to just assume that not only can they demand that somebody else does the job, but also that these people haven't already considered a 2.8 seriously.
6
u/brechtm May 28 '14 edited May 28 '14
In 2008 I started a Python 3-only project. Now, as the project is finally maturing, I feel the need for it to run also on Python 2, as I'd like to reach an audience that is as large as possible. Note that next to CPython 2, there's also Jython and IronPython that both seem to be stuck in Python 2.x land (but PyPy3 is in fairly good shape, it seems).
I did manage to backport the code to Python 2 some time ago, making sure the same codebase still runs on Python 3. Unfortunately, there are some things that make this unpleasant at the least. For example, metaclass definitions need to be specified in different ways in 2/3. It would be great if Python 2 would also support the Python 3 syntax. Similar to the re-addition of the unicode literal u"" to Python 3 and "from future import unicode_literals" in Python 2, this would be a minor change that could greatly simplify having a unified 2/3 codebase. Also, the Py2 str / Py3 unicode thing is very tricky to get right, but I'm not sure how this could be simplified.
There are also some Python 3 features that are not present in Python 2 which cannot be remedied by a third-party library. This stops me from adopting these Python 3 features. One example is Python 3's function annotation capability (which I'd like to use for type checking). Again, having support for these in Python 2 (even if they are simply parsed and ignored) would be great.
I think that the move to the (backward-incompatible) Python 3 was a good one, but in hindsight, perhaps more thought/effort should have been put into making the upgrade path as smooth as possible. Some tools have been given to the Python developer over the years (from future import <py3_thing>, u"") to help with this, but I think they should have been available from the start (yes, easy to say now). Something to remember for Python 4 or 5?