r/Python • • Nov 14 '17

Senior Python Programmers, what tricks do you want to impart to us young guns?

Like basic looping, performance improvement, etc.

1.3k Upvotes

629 comments sorted by

View all comments

Show parent comments

3

u/RaionTategami Nov 14 '17

OOP in my opinion is just bad. But that's my opinion. Never writing classes is obviously going overboard so I'm not really saying you should never do it. A good rule of thumb is only writing a class if you have a new data type. If you want to bundle data then things like dicts and tuples (esp named tuples) are better. I see people using them when they are not needed and their code would be much cleaner without them.

Here's a recent example a saw from a neural network lib. They wanted to create layers which you could then call;

class Layer(object):

    def __init__(self, params):
       self.params = params

    def __call__(self, x):
        # do something with x and self.params

People seem to love this pattern but this is maddening to see because this is a closure.

def layer(params):
    def f(x):
        # do something with x and params
    return f

Not only is that shorter its less prone to bugs.

So what I should really say is try not to write classes, there's probably a better way.

19

u/[deleted] Nov 14 '17

God no. You can't test that closure. You just made that whole function a black box.

5

u/Hellios Nov 14 '17

I think closures can be tested just fine.

import unittest

def foo(n):
    def add_one():
        return n + 1

    return add_one

class AddOneTest(unittest.TestCase):

    def test_add_one_increases_value_given(self):
        add_one = foo(1)

        self.assertGreater(add_one(), 1)

if __name__ == "__main__":
    unittest.main()

You have the same control over the code under test as you would in the class example. Pass in a mock object to __init__ and get an object with a mock inside it or pass a mock object to a function and get a closure with a mock inside it. Either way there's dependency injection and visibility into what's happening under the hood. Far from black box testing.

4

u/[deleted] Nov 14 '17

Definitely still not a unit test on functionf.

0

u/[deleted] Nov 14 '17

[deleted]

4

u/RaionTategami Nov 14 '17

Not sure what you are asking. Are you commenting on the fact that you no longer have access to params once you get f back? I see that more as a feature, it's now state that you can't touch you know that f won't change because it can't, the state of the object in the other hand can change and cause bugs. Though admittedly this closure is harder to unit test.

0

u/[deleted] Nov 14 '17

[deleted]

5

u/peedubyaeff Nov 14 '17 edited Nov 14 '17

Turns out the locals are stored in 'cells', and referenced in the bytecode: https://docs.python.org/2/c-api/cell.html

In [56]: def layer(params):
   ....:     def f(x):
   ....:         print params, x
   ....:     return f
   ....:

In [57]: foo = layer('these are my params')

In [58]: foo.__closure__[0]
Out[58]: <cell at 0x7fac61825830: str object at 0x7fac6186bc70>

In [59]: foo.__closure__[0].cell_contents
Out[59]: 'these are my params'

I don't see how you'd tie the cell back to the variable name though, so if you have more than one and need to know which is which you might be out of luck.

Edit - Of course, if you're actually using a debugger it will know the variable name from the frame you're in:

In [1]: import ipdb

In [2]: def layer(params):
   ...:     def f(x):
   ...:         print params, x
   ...:         ipdb.set_trace()
   ...:     return f
   ...:

In [3]: foo = layer('params')

In [4]: foo('x')
params x
--Return--
None
> <ipython-input-2-80f4294737a0>(4)f()
      3         print params, x
----> 4         ipdb.set_trace()
      5     return f

ipdb> params
'params'

1

u/patrickchen1994 Nov 14 '17

Are you talking about pytorch? In fact, I found this kind of OOP suitable in the context of neural network programming. It will lose generalizability if it's implemented using functions.

1

u/Arancaytar Nov 14 '17
    def f(x):
        # do something with x and params
    return f

I'm kind of surprised python hasn't added anonymous non-lambda functions yet.

1

u/[deleted] Nov 14 '17

Can you give a reasonable explanation for why you would need an anonymous function?

2

u/Arancaytar Nov 14 '17

Well, in this case it would save you a redundant temp variable (f).