27
u/NFriik 20d ago
Python evaluates default arguments when the function is defined. That makes keeping track of the default values pretty straightforward. In your example, they're stored as part of the function object in the dictionary add.__defaults__. Every subsequent call can then just look-up the default value in this dictionary, if necessary.
Evaluating the default argument each time the function is called would be much more difficult. A function is essentially just yet another object, so you'd somehow have to store arbitrary expressions as an object inside the __defaults__ dict and then decide at runtime what to do with them. Sure, you could implement a builtin type at the CPython level to accomplish this. But then when you encounter such an ArbitaryExpression object inside the __defaults__ dictionary, how do you know that this time, you want to actually execute the underlying code and not use the object "as-is"? The whole thing would quickly become a convoluted, expensive mess.
By performing evaluation at definition time, function calls remain consistently fast and the mechanism to insert default positional arguments becomes incredibly simple. Python is already not the fastest language when it comes to function calls, so these things matter.
12
5
u/BuriedStPatrick 18d ago edited 18d ago
It often surprises me how much of Python rubs me the wrong way. I'm not even saying it's bad or whatever and I'm happy it helps less experienced people get into basic programming. It's just every time I come into contact with it it explodes on me and I want to get as far away as possible from it. I hate the syntax and the strange design decisions with a passion. I've never used a language where default function arguments worked this way. I would've just opted to not support this feature if I had to make it work this way.
I don't write Python myself but you tend to come across it in a lot of smaller open source CLI tools. And it's always a big sigh for me when I look at the source and know the install/build process has a 50/50 chance of being a nightmare.
1
u/NFriik 17d ago
A lot comes down to personal taste. I've been working with Python for almost 15 years now and I actually enjoy it a lot. Yes, a lot of things work a bit different, especially if you're coming from the C world.
If you're having problems with the build process, you should have a look at uv. Pip and poetry have become obsolete to me.
3
u/gundam1945 19d ago
The explanation makes sense but the behavior is probably not expected. Though if I need to add to a list I usually make the variable outside first and extend in the function.
Never encounter such a use case in this example.
1
u/Bob_Dieter 19d ago
Counter point, Julia (the language) does just that, reevaluate the default expression if no value is provided, and it is neither slow nor messy.
Admittedly, Julia gets to recompile each function for different arities, which does not fit to pythons execution model. Still, I feel like there is a clean, performant solution for this that the python community could implement. But that would be a breaking change, and that is probably the real reason this well known foot gun is going to stay in the language.
-3
u/ZylonBane 20d ago
You make it sound like it would be absurdly complex or slow to just clone any non-primitive default value.
16
6
u/NFriik 20d ago edited 20d ago
Well, it kinda is, because that's not how Python works. A function is an ordinary object, it's default argument values are stored in a dict inside this object, so you can read and manipulate them at runtime. That wouldn't work with your idea, not without huge overhead.
0
u/ZylonBane 20d ago
"That wouldn't work because that's not how it works" is a circular argument. The implementers could have made default values evaluate at call time, like JavaScript does, but instead they chose violence.
5
u/NFriik 20d ago
Implementing functions the way Python does is a design decision with advantages and disadvantages. One of the advantages is that it allows to adhere to one of the language's main principles, which is that everything is an object. In the end, this behavior is just a consequence of this. Having default arguments treated differently to other function attributes (such as the name, doc string and code) would be weird. The Python devs didn't choose violence, they chose consistency.
2
u/Loading_M_ 19d ago
A more important point: default values don't have to be stored on the function object at all. The only information actually needed is whether the keyword argument has a default value, not the specific value. Then the code to instantiate the default values (if needed) can simply be inserted into the function body before the user code. This is basically how most other interpreted languages handle it.
Alternatively, the default value could always be stored as an Expression object or similar.
115
u/_killer1869_ 20d ago
Any IDE will warn not to use mutables as default parameter values. And for good reason. Such code would never appear in actual code intended to be used.
35
u/Double_Cost4865 20d ago
Unfortunately, I’ve seen monstrosities like this in production in big finance corps. That’s what happens when statisticians are expected to create software.
20
u/marinuso 19d ago
I've seen it used in production code to memoize functions in this manner:
def fib(n, cache = {0: 1, 1: 1}): if not n in cache: cache[n] = fib(n-1) + fib(n-2) return cache[n]Except with much more complicated code rather than this simple example.
2
0
u/bigorangemachine 19d ago
default params is the classic case of IDGAF with code.
"I don't which array... just add it to an array"
7
u/CagoSuiFornelli 20d ago
Such code would never appear in actual code intended to be used.
You should take a tour in my company codebase
12
3
u/scheimong 19d ago
Such code would never appear in actual code intended to be used.
Tell me you're a junior dev without telling me you're a junior dev.
19
u/mothzilla 20d ago
It's a well known python gotcha that people ask about in interviews all the time.
6
u/Beregolas 20d ago
This also happens in JavaScript btw, as well as several smaller scripting languages in game engines, it's a pretty normal feature in many interpreted languages.
In my last backend job, this was actually the source of our longest running and most annoying bug, where a specific page would display the wrong sorting, but never in testing. Because someone *caugh* used a mutable argument to give it default sorting parameters, and they got mutated by a very specific query, which caused it to be wrong for all subsequent requests.
So yeah, this is a real thing, you should know it, and just in case, you should have linting and/or your IDE setup so this kind of thing is rejected out of hand, and cannot be merged to prod/main.
2
u/4xe1 20d ago
just in case, you should have linting and/or your IDE setup so this kind of thing is rejected out of hand
So what kind of linting have you adopted? Default parameters can be quite handy. Does it reject them, or does it manage to detect and reject their mutations?
4
u/Beregolas 20d ago
I get a warning when I use anything mutable. Instead of an empty list, or a list with presets for example, I have to put None, and set it in code. This is basically also the recommended pattern.
2
u/tehdog 16d ago edited 16d ago
This also happens in JavaScript
No it doesn't? JS has it's own dumbness, but this particular ridicularity is Python-only. e.g.
function test(x=[]) { x.push(1); console.log(x) }does what you'd expect.And this is not the only place where this happens, it also happens in the very-innocent looking.
class Foo: x = [] a = Foo() a.x.append('wat') b = Foo() print(b.x) ['wat']It's the kind of thing that makes you think how could this possibly be a serious programming language if they didn't consider this basic code?
Python is supposed to be simple and intuitive. Nothing is intuitive about having to do
def __init__(self): self.x = []instead of the above.1
u/Beregolas 16d ago
you actually seem to be correct... I must have misremembered. I vividly remember we had a bug in our Node-JS backend (not my choice) which worked exactly the same as this, but then it was probably passed in or a const...
Sorry for the confusion / misinformation ^^
4
u/RngdZed 20d ago
The list is stored directly in the parameter without actually giving it an argument? I'm baffled, I didn't know this was a behavior, so weird
Is this something that someone could ELI5 to my smooth brain?
6
u/bigorangemachine 20d ago
Default param is an object reference. So every default param is the same object
2
u/lolcrunchy 20d ago
Here's a single change to the posted code that kinda shows what actually happens.
my_list = [] def add(v, items=my_list): items.append(v) return items print(add(1)) print(add(2))1
u/bigorangemachine 19d ago
Ya I test in JS and JS will do this.. oddly enough they are so similar JS didn't do the thing OP posted about.
Surprised about that one... but it makes sense.
4
3
u/GoddammitDontShootMe [ $[ $RANDOM % 6 ] == 0 ] && rm -rf / || echo “You live” 19d ago
So each call reuses the same default list? Yeah, I think I would've expected it to get passed an empty list by default each time.
3
u/awidesky 20d ago
Is that a documented behavior, or not?
If it is, that doesn't make sense.
If not, it's technically a bug, right?
1
u/CoachSevere5365 17d ago
Yes, yes it does, no it isn't.
2
u/awidesky 17d ago
So you're telling me, "default argument is []" is NOT supposed to mean "if you omit this argument, the default value of [] will be passed",
But "if you omit this argument, it'll be substituted to a hidden static storage global object whose initial value is [], so that you have absolute zero information what the default argument would be, and your code will suffer from side effects of other code"It does not make a sense at all.
Please name one programming language that works the same way.0
u/CoachSevere5365 17d ago
Read the docs, I didn't design it.
1
u/awidesky 17d ago
I never asked you to redesign it.
Also, python documentation is quite far from informative nor useful; I wasn't surprised when I read the docs and couldn't find a single bit of info about why they chose that nonsense design.
1
u/bigorangemachine 20d ago edited 20d ago
I bet this happens in JS as well
Edit: nope python
I should check gdscript
1
u/Crafty-Detail-3788 20d ago
Issue here is that the default parameters are initialized ONCE. If you want an empty list every time you need to define it inside.
1
u/itemluminouswadison 20d ago
It's a quirk of the language and something you should have learned and read about. The default arg gets created once upon func definition, not per invocation
1
u/jmooroof2 20d ago
I can't imagine a compelling use for this feature. it really should just error
4
u/NFriik 20d ago
There are some use cases for this behavior, but it's reeeaaally limited and you have to know exactly what you're doing. For example, you could use it to cache relatively expensive function calls as default values:
import re def parse(text, pattern=re.compile(r"\d+")): return pattern.findall(text)This way, instead of re-compilingthe regex every time the function is called, you can just reuse it.
6
u/simonsmine 20d ago
Its not really a feature more a consequence of pythons reference handling. You can do fun things with it though, memosation comes to mind.
1
u/Double_Cost4865 20d ago
It has it’s uses, I just wouldn’t provide default argument like that since it’s wrong. But it’s quite common to modify a list in place and have the changes persist outside the function scope
1
u/4xe1 20d ago
default function parameters are useful, shared function default parameters are efficient, and mutable function parameter are convenient (saves the interpreter the check for mutability, and and saves the user the boiler plate to copy it in a local variable).
What you're seeing is merely a combination of the three.
But shared mutable function parameter is also uniquely useful for memoisation. It's not where you'd put the table of previous results in an other language, but it makes perfect sense to do so in python, as it hides a side effect which is not a real side effect.
1
0
u/OhItsJustJosh 20d ago
This is why I stick to C#, I've written in Pascal, Java, JavaScript, Python, handful of others, and they all have their weird quirks, but C# seems to be the only one that works as expected
-3
u/CaptainFoyle 20d ago
Everyone knows: don't use mutable default arguments.
You just outed yourself as uninformed.
5
u/nakurtag 20d ago
Who you name "everyone"? Python developers? I'm not python developer, I'm from C#/Go/Rust world and this behavior is ridiculous for me. This is highly counterintuitive and illogical.
3
u/squabzilla 19d ago
My main language is Python, and still think it’s ridiculous how much unintuitive behaviour there is because Python’s array-equivalents, by default, copy the object-pointer instead of the object itself.
And then people tell you why it works like that, confusing an explanation for an excuse.
-1
u/CaptainFoyle 20d ago
I mean how do you learn a new language? Does it just pop into your head? Or do you look up documentation and syntax? Then that's usually one of the first things you read
4
u/nakurtag 20d ago
Eventually, I consulted the documentation when I encountered this behavior. But initially, I tried to write the code based on my experience as a programmer and by using deduction.
175
u/drislands 20d ago
I would have expected this function to create a new empty list on any call where
itemsis not given. Doesn't this behavior make the function into a closure?