r/Python • • Nov 14 '17

Senior Python Programmers, what tricks do you want to impart to us young guns?

Like basic looping, performance improvement, etc.

1.3k Upvotes

629 comments sorted by

View all comments

Show parent comments

1

u/NoLemurs Nov 16 '17 edited Nov 16 '17

We do often write code that accepts either sequences or generators, but the right way to do that is to immediately call iter on the argument

This doesn't help at all. An empty iterator is still truthy. In fact, this makes things worse:

>>> a = []
>>> b = iter(a)
>>> if b: print('non empty')
... 
non empty

It's hard to keep track of the intricacies of all these different approaches. But the behavior of the if myCollection check, when applied to things that aren't actually lists, tuples, or dictionaries, requires you to be thinking about these things and to know the behavior backwards and forwards. To me that makes the code much less readable.

Duck typing is good as long as the thing you're duck typing on is logically the thing you care about. Once you start duck-typing on something that's just a proxy for the thing you care about you start moving into territory where your code may look nice, but it has a bunch of hidden gotcha's that you have to be aware of - all the cases where the thing you're duck-typing on isn't actually the thing you care about. The result is bug prone, and despite looking simple, much more complex and difficult to understand than a more explicit version.

EDIT: I'll add that the reason I knew how iter would behave is that iterators don't have a concept of length - they can't because they have to handle potentially infinite items. Since python doesn't really have a distinct idea of a collection being 'empty', a collection (like an iterator) with no length is always going to be truthy because there's no relevant specific idea of truthiness, and Object is truthy by default. This is why the if len(myCollection) > 0 check makes sense. It is checking the condition that is most closely tied to the idea of 'non-emptiness'. If python collections had an is_empty() method - that would be the thing to check.

1

u/stevenjd Nov 17 '17

This doesn't help at all. An empty iterator is still truthy. In fact, this makes things worse:

Of course it helps. If your dealing with iterators, you don't test for truthiness at all: don't Look Before You Leap, instead it is Easier to Ask Forgiveness than Permission. When you are dealing with iterators, including generators, you shouldn't write:

if spam:  # len(spam) != 0 won't work either
    # handle non-empty case
else:
    # handle empty case

because it doesn't work, regardless of whether you test for truthiness or directly compare the length to 0. Instead:

spam = iter(spam)  # make sure we have an iterator
try:
    # handle non-empty case
except StopIteration:
    # handle empty case

works for any iterable, including iterators, generators, sequences and collections of all sorts.

But the behavior of the if myCollection check, when applied to things that aren't actually lists, tuples, or dictionaries, requires you to be thinking about these things and to know the behavior backwards and forwards.

Apart from iterators, can you think of any other built-in type or standard library type that doesn't obey the rule that "nothing/empty" values are falsey, and "something/non-empty" values are truthy? I can't, and I've been using python since before iterators even existed. So there's really only one common exception to the rule, and if you're accepting generators and other iterators as well as collections, you can't use the if len(x) == 0 test because iterators don't have a well-defined length.

So in practical terms, there's no need to think about this "backwards and forwards" -- there's just two cases to deal with. If you are dealing with iterators, use the EAFP idiom and catch StopIteration; if you're using sequences or mappings or other collections, you can LBYL by checking the truthiness of the collection. In neither case is there any point in first calculating the length of the collection if all you want to know is whether it is empty. Even if len is cheap for the builtins, that's an implementation detail, not a language promise, and you never know when somebody will pass you a linked list with a billion items.

If python collections had an is_empty() method - that would be the thing to check.

They do have an is_empty method. It is spelled bool(collection), and you can leave out the call to bool in many contexts.

a collection (like an iterator) with no length is always going to be truthy because there's no relevant specific idea of truthiness

In practice, you're right, but that's only because the iterator protocol is deliberately minimalist. There's no reason why iterators cannot support a more useful bool even if they don't support len. Here's a quick sketch that shows one possible way to make it work:

class MyIterator:
    def __iter__(self):
        return self
     def __next__(self):
         sentinel = object()
         x = getattr(self, "_saved", sentinel)  # look for a saved value, if it exists
         if x is sentinel:
             # calculate the next value the usual way...
             return "something"
         else:
              del self._saved
              return x
    def __bool__(self):
         sentinel = object()
          x = next(self, sentinel)
          if x is sentinel:
              return False
          else:
              self._saved = x
              return True

You can see why Guido doesn't want to require that for all iterators. Its a pain. But you can certainly make your own iterators support a more useful concept of truthiness, if you can be bothered.

But if you think that's hard, well, that's nothing compared to having arbitrary iterators support len. It is not just the the length might be infinite -- it can also be indeterminate.

def generator():
    while random.random() < 0.5:
         yield 1

Short of caching the entire sequence of values, which defeats the purpose of using a generator, there's no way of knowing what the length will even be until you reach the end.

The bottom line is, the idea of emptiness (i.e. truthiness) is more fundamental than the idea of having a well-defined length that can be compared to zero. I don't know exactly how many grains of sand there are on the beach, nor do I care, and I certainly don't want to have to enumerate each and every one of them, just to decide whether the beach is empty or not. I don't care about the difference between a beach with 67 trillion grains of sand and one with only 57 trillion grains. All I care about it whether or not there are any grains of sand, and the canonical way to write that using LBYL is if myCollection.