The best answers to the question “"Least Astonishment" and the Mutable Default Argument” in the category Dev.
Anyone tinkering with Python long enough has been bitten (or torn to pieces) by the following issue:
def foo(a=): a.append(5) return a
Python novices would expect this function to always return a list with only one element:
. The result is instead very different, and very astonishing (for a novice):
>>> foo()  >>> foo() [5, 5] >>> foo() [5, 5, 5] >>> foo() [5, 5, 5, 5] >>> foo()
A manager of mine once had his first encounter with this feature, and called it “a dramatic design flaw” of the language. I replied that the behavior had an underlying explanation, and it is indeed very puzzling and unexpected if you don’t understand the internals. However, I was not able to answer (to myself) the following question: what is the reason for binding the default argument at function definition, and not at function execution? I doubt the experienced behavior has a practical use (who really used static variables in C, without breeding bugs?)
Baczek made an interesting example. Together with most of your comments and Utaal’s in particular, I elaborated further:
>>> def a(): ... print("a executed") ... return  ... >>> >>> def b(x=a()): ... x.append(5) ... print(x) ... a executed >>> b()  >>> b() [5, 5]
To me, it seems that the design decision was relative to where to put the scope of parameters: inside the function, or “together” with it?
Doing the binding inside the function would mean that
x is effectively bound to the specified default when the function is called, not defined, something that would present a deep flaw: the
def line would be “hybrid” in the sense that part of the binding (of the function object) would happen at definition, and part (assignment of default parameters) at function invocation time.
The actual behavior is more consistent: everything of that line gets evaluated when that line is executed, meaning at function definition.
Suppose you have the following code
fruits = ("apples", "bananas", "loganberries") def eat(food=fruits): ...
When I see the declaration of eat, the least astonishing thing is to think that if the first parameter is not given, that it will be equal to the tuple
("apples", "bananas", "loganberries")
However, suppose later on in the code, I do something like
def some_random_function(): global fruits fruits = ("blueberries", "mangos")
then if default parameters were bound at function execution rather than function declaration, I would be astonished (in a very bad way) to discover that fruits had been changed. This would be more astonishing IMO than discovering that your
foo function above was mutating the list.
The real problem lies with mutable variables, and all languages have this problem to some extent. Here’s a question: suppose in Java I have the following code:
StringBuffer s = new StringBuffer("Hello World!"); Map<StringBuffer,Integer> counts = new HashMap<StringBuffer,Integer>(); counts.put(s, 5); s.append("!!!!"); System.out.println( counts.get(s) ); // does this work?
Now, does my map use the value of the
StringBuffer key when it was placed into the map, or does it store the key by reference? Either way, someone is astonished; either the person who tried to get the object out of the
Map using a value identical to the one they put it in with, or the person who can’t seem to retrieve their object even though the key they’re using is literally the same object that was used to put it into the map (this is actually why Python doesn’t allow its mutable built-in data types to be used as dictionary keys).
Your example is a good one of a case where Python newcomers will be surprised and bitten. But I’d argue that if we “fixed” this, then that would only create a different situation where they’d be bitten instead, and that one would be even less intuitive. Moreover, this is always the case when dealing with mutable variables; you always run into cases where someone could intuitively expect one or the opposite behavior depending on what code they’re writing.
I personally like Python’s current approach: default function arguments are evaluated when the function is defined and that object is always the default. I suppose they could special-case using an empty list, but that kind of special casing would cause even more astonishment, not to mention be backwards incompatible.
Actually, this is not a design flaw, and it is not because of internals or performance.
It comes simply from the fact that functions in Python are first-class objects, and not only a piece of code.
As soon as you think of it this way, then it completely makes sense: a function is an object being evaluated on its definition; default parameters are kind of “member data” and therefore their state may change from one call to the other – exactly as in any other object.
In any case, Effbot has a very nice explanation of the reasons for this behavior in Default Parameter Values in Python.
I found it very clear, and I really suggest reading it for a better knowledge of how function objects work.
I know nothing about the Python interpreter inner workings (and I’m not an expert in compilers and interpreters either) so don’t blame me if I propose anything unsensible or impossible.
Provided that python objects are mutable I think that this should be taken into account when designing the default arguments stuff.
When you instantiate a list:
a = 
you expect to get a new list referenced by
Why should the
instantiate a new list on function definition and not on invocation?
It’s just like you’re asking “if the user doesn’t provide the argument then instantiate a new list and use it as if it was produced by the caller”.
I think this is ambiguous instead:
user, do you want
a to default to the datetime corresponding to when you’re defining or executing
In this case, as in the previous one, I’ll keep the same behaviour as if the default argument “assignment” was the first instruction of the function (
datetime.now() called on function invocation).
On the other hand, if the user wanted the definition-time mapping he could write:
b = datetime.datetime.now() def x(a=b):
I know, I know: that’s a closure. Alternatively Python might provide a keyword to force definition-time binding:
def x(static a=b):
The relevant part of the documentation:
Default parameter values are evaluated from left to right when the function definition is executed. This means that the expression is evaluated once, when the function is defined, and that the same “pre-computed” value is used for each call. This is especially important to understand when a default parameter is a mutable object, such as a list or a dictionary: if the function modifies the object (e.g. by appending an item to a list), the default value is in effect modified. This is generally not what was intended. A way around this is to use
Noneas the default, and explicitly test for it in the body of the function, e.g.:
def whats_on_the_telly(penguin=None): if penguin is None: penguin =  penguin.append("property of the zoo") return penguin