To get started, I’d just like to say “Hello”. This is my first real blog post (excluding the geocities and xanga sites I started back in high school).
I’ve been playing with Python recently, and a few things have been bugging me. Before I get started, I do have to admit Python does a lot of things right; for all its flaws, I’d much rather work in Python than Java or C++.
That being said, there are a few things I’ve encountered that seem very inconsistent, and are very inconvenient. I had hoped that they were just legacy artifacts and would be fixed in Python 3000 but, for the most part, they seem to be sticking around.
Scoping
First, to show you the expected behavior of lexically scoped variables in Python:
a = "foo" def f(): print a #a is free, so it refers to the global binding "foo" def g(): a = "bar" f() #Python creates a closure around the function f with a reference to the global a print a #Here the global a is shadowed by the local a, which has a value of "bar" >>>g() foo bar
As expected, calling the function g prints “foo” and then “bar”.
Now, when we try a slightly more interesting example:
a = "foo" def f(): print a #Should refer to the global a, declared above a = "bar" #Should shadow the global (statically scoped) a print a #Should print bar >>>f() UnboundLocalError: local variable 'a' referenced before assignment
There is no good reason for a language to exhibit this behavior: it’s completely inconsistent with what was demonstrated in the first example. Since ‘a’ is free in the function f the first time we try to print it, it should refer to the global ‘a’ with the value of “foo”.
Rounding errors
Now, maybe I’m just spoiled coming from doing Common Lisp work, but I’m used to programming languages handling numbers right. In Lisp, I don’t have to tell the compiler how big or small my numbers are going to be (although, I optionally can for better performance). They just work. Lisp transparently uses fractions or floating points or complex numbers or integers automatically - and it never makes a rounding error. To me 
, etc.
Technically, I understand that it is rather difficult to perfectly represent decimal numbers in binary machines, but Lisp (and many other languages) solved these problems 30 years ago. There’s no reason that we should still be struggling with them today - especially in a very high level language like Python.
>>>a = 0.1 >>>a 0.10000000000000001 >>>a - 1/10 0.10000000000000001
’s
Python’s lambdas have been emasculated. I know that lambas are never strictly necessary - anything I can do with a lambda I can do with a named function. And I also know that python provides powerful list comprehensions which serve the purpose of map/filter and lambda more often than not. But, there are often times when I just want a lambda.
The problem is, python’s lambda doesn’t work. You can’t ‘print’ inside a lambda (but you can call another function that can?!) and they can’t span multiple lines. I don’t know if these were purposeful design decisions meant to discourage the use of lambdas (maybe, especially in light of the talk I’ve seen of eventually deprecating reduce, filter, and map) but, in any case, they are annoying.
>>> a = lambda x: print x SyntaxError: invalid syntax def myPrint(x): print x a = lambda x: myPrint(x) >>>a("Hello World") Hello World
And here ends my first, relatively mild, rant.
Comments 19
Your second example, with the rounding errors, is wrong. You should do a - 1.0/10, then you get the expected result. I agree that floats can be a bit tricky in python, especially when treating them as numbers, but this has been somewhat corrected with the introduction of the Decimal class.
Posted 06 Sep 2008 at 2:58 am ¶The reason for the ‘global’ semantics is to avoid thrashing things that are supposed to have translation unit scope. If you ever worked with non-strict Perl or even Visual Basic, you’d know how useful that is in a dynamic language.
Posted 06 Sep 2008 at 3:01 am ¶Oh man, you really should have posted this in python-list. You still can, but now your response won’t be as friendly because you come across as an arrogant Lisper, and if there is one thing the Python community can’t stand, it is a Lisper complaining about how Python isn’t an incarnation of The One True Language. Get over it.
1) Scoping. Functions are compiled, and references to names are converted to lexical addresses. We can’t have the same name in a function point to a local in one place and a global in another unless we break the one-to-one correspondence between names and lexical addresses. This would probably be a bad thing. It would certainly make compilation more difficult, and it might break other things too. On top of that, this problem never arises in practice, so this is basically a nitpick.
2) Rounding Errors. You will find that Python is very much in RPG’s “worse is better” category vs. “the right thing”. In other words, simplicity and practically *almost always* beat purity. The situation for floats in Python is actually not bad - there is no such thing as a single precision float in Python, so you don’t have to worry about converting between singles and doubles. (There is an exception carved out for numpy and/or ctypes, I can’t remember which). Again, 99.99% of the uses of floats do not demand arbitrary precision.
3) Lambda. Hah! You should be glad you still have it in 3. I think it was on the chopping block at one point (although that might have just been the higher order functions). Yes, you are *strongly discouraged* against using lambdas. Just look through the standard library. Lambdas barely do *anything*. But they are handy to whip out if you need them. Generally speaking, prefer generator expressions, list comprehensions, str.join, and the built-in function sum().
Here is an example.
IDENT_PAT = re.compile(’^ *([a-zA-Z][^ ]*).*’)
Posted 06 Sep 2008 at 3:05 am ¶[len(match.groups()[0]) for match in (IDENT_PAT.match(line) for line in open(filename)) if match]
Tiberiu Ichim:
I know that 1.0/10.0 would have worked. The point I was trying to make (and maybe I did a poor job of explaining this) is that Python and its ilk are made to make development quick and easy. It would just be easier and more intuitive to have 1/10 = 1.0/10.0 = 0.1. One less thing to distract you from important stuff.
Klaus:
Sorry, I don’t think I understand what you mean … Python is statically scoped, so it’s always pretty easy to check what free variables are bound to in a particular function. I think I’m misunderstanding you, would you mind clarifying?
David:
Sorry, didn’t mean for this to come off too critically. Python is one of my favorite mainstream languages - these are just a few little inconsistencies that bother me. Common Lisp has its own weird idiosyncrasies (odd function naming scheme, CLOS being tacked on, threading/sockets/etc not being part of the spec).
No language is the ‘One True Language,’ of course. But I think all languages should strive to improve - and these are some areas I think Python could improve in.
1) I understand *why* Python has this problem now. I just think that it should be fixed (and it obviously can be fixed, since most other languages that support closures don’t have this problem). And I have had this come up in practice - that’s how I discovered it. There are of course easy work arounds, but it is inconsistent and should be fixed (albeit, it may not be the highest priority). I just think Python3k is a good time to do so, since it’s probably the one time this decade Guido will be able to break backwards compatibility.
2) Python’s number handling is neither simple nor practical. The simplest solution (for the developer/end-user, not the language implementer) is to have numbers in the computer work just like actual numbers. Of course, Python is a step up from C++/Java, but I think it can do better. I think language developers should always strive to follow the principle of least astonishment (http://en.wikipedia.org/wiki/Principle_of_least_astonishment). And most people would agree that having 1/10 = 0, and 1.0/10.0 = 0.10000001 is far more astonishing than 1/10=1.0/10.0=0.1
3) Again, I don’t ever need lambdas. But, every so often a situation arises where lambdas are the best tool for the job - why not make them powerful enough to do that job well? As I mentioned in the main post, I love list comprehensions - 90% of the time they take the place of lambdas. But I don’t see that as any excuse for purposefully crippling lambdas.
Posted 06 Sep 2008 at 4:43 am ¶2) This will change in 3.0, and is already an option in 2.5:
>>> from __future__ import division
>>> 1 / 10
0.10000000000000001
3) print() is a function in 3.0, so you can use it in lambdas there. Still, list compressions FTW
Posted 06 Sep 2008 at 6:46 am ¶*comprehensions
Posted 06 Sep 2008 at 6:46 am ¶On your first example :
Javascript behaves the exact same way
var a = “bar”;
function foo() {
print(a);
var a = “baz”;
print(a);
}
foo();
>>> undefined
>>> “baz”
There’s no local -> global lookup, the names belongs to the block in which they are declared.
Posted 06 Sep 2008 at 7:20 am ¶1) in you your first example to make it working add “global a”. Be happy, be very very happy that your example have not worked. You will understand that in future;
2) In python floats are real machine floats, not decimal numbers. Check http://docs.python.org/lib/module-decimal.html - while I’m pretty sure there are more options (check pynum);
3) lambda’s will be removed from python 3000. You don’t need them at all. Find polish mathematician presentation from EuroPython 2008 about functional programming - you will enjoy that;
Posted 06 Sep 2008 at 7:22 am ¶What I always missed the most were let expressions, and temporary variable assignments in list comprehensions. In both cases it would let us name potentially expensive computations once and for all before using the value in later parts of the computation.
Posted 06 Sep 2008 at 8:32 am ¶> lambda’s will be removed from python 3000. > You don’t need them at all.
Sorry, but that’s a bit thick. There are plenty of things that one doesn’t literally “need” that are nevertheless desirable. I think lambdas are pretty useful myself…
Posted 06 Sep 2008 at 8:34 am ¶You really want the same identifier to refer to a local variable in one part of a function and a global in another? That make things very confusing for the reader and leads to confusing bugs when lines within the function are moved.
Posted 06 Sep 2008 at 8:48 am ¶Dalius is correct on #1: You can add “global a” as the first line in your function in your second scoping example to have it work as you intended.
(Also, in Py3000, there’s going to be a new “nonlocal” keyword to deal with nonglobal variables outside the current function in the case of nested functions.)
Re lambda: The last I read, lambdas are not being removed from Py3000 after all (but they will still just be for expressions and not statements).
Posted 06 Sep 2008 at 10:02 am ¶No, “global a” won’t do what he intended — then when f() assigned “bar” to a it would change the value of the global a. This is because Python doesn’t distinguish between creation and assignment, and is a primary reason why Python’s closures are usually seen to be broken.
(Not that I wholeheartedly agree.)
Posted 06 Sep 2008 at 1:05 pm ¶I love that the python weenies can’t cope with someone who likes python pointing out some of the leaky abstractions and corner cases of the python language.
The fact is that python seems to be a pretty solid language, with huge base of programmers who don’t know what they’ve got, who try to have it made more retarded (e.g. by crippling lambda), and a small number who do know what they’ve got, and push to make it better (the newer object system), but no-one who is committed to stability of implementation, or even to keeping the community up to date with what the current state of play (look at all the out of date information floating around that is now completely and fatally wrong about python).
Posted 07 Sep 2008 at 3:46 am ¶I agree that the number system in Common Lisp is great (and better that Python’s), but it does not solve the problem of rounding errors unless you stick to rationals. Equality testing of floats is as big a problem in Lisp as it is in other languages.
For instance, 0.1 and 1/10 is not necessarily equal. Try evaluating (= 0.1 1/10). I think the result is implementation-dependent, but it returns false both in clisp and sbcl, at least. However, (= 0.5 1/2) returns true.
Posted 07 Sep 2008 at 9:09 am ¶two examples with solutions, first a re-statement of one of yours:
>>> def mkcounter(start):
… def counter():
… start += 1
… return start
… return counter
…
>>> c = mkcounter(5)
>>> c()
Traceback (most recent call last):
File “”, line 1, in
File “”, line 3, in counter
UnboundLocalError: local variable ’start’ referenced before assignment
and the “correct” (ha ha) way of writing that:
>>> def mkcounter2(start):
… mutable = [start]
… def counter():
… mutable[0] += 1
… return mutable[0]
… return counter
…
>>> d = mkcounter2(7)
>>> d()
8
and second, one you haven’t mentioned, that drives me crazy:
>>> lambdas = [lambda: i for i in range(3)]
>>> for l in lambdas: print l
…
>>> for l in lambdas: print l()
…
2
2
2
and the “correct” (tee hee) way to write that:
>>> lambdas2 = [lambda x=i: x for i in range(3)]
>>> for l in lambdas2: print l()
…
0
1
2
despite those (and occasional confusion over module names), i have to say i do love python. it sneaks enough good things under the corporate radar to make my job enjoyable :o)
Posted 07 Sep 2008 at 8:34 pm ¶ugh. insert spaces as appropriate above.
Posted 07 Sep 2008 at 8:36 pm ¶Another gotcha - surprisingly many things evaluate to False. My favourite one (and one I really hope they fix for Python 3.0) is midnight being False, while all other times are True. It caused nasty bugs in real production code, as time could be None for unspecified and I tested if time was specified using if time: else:,
what worked for all time except for midnight.
>>> midnight = datetime.time(0, 0, 0)
>>> bool(midnight)
False
There are other things evaluating to False.
Another really ugly one is XML element containing no XML children (but possibly some text children) is False, while XML element with XML children is True. It could be argued that XML elements acts as a list here, but I think it’s really stupid and counterintuitive.
>>> bool(xml.etree.ElementTree.fromstring(”Hello”))
False
>>> bool(xml.etree.ElementTree.fromstring(”"))
Posted 10 Sep 2008 at 7:37 pm ¶True
I meant this before XSS stripper removed tags instead of escaping them (showing once more that all post forms need preview function):
>>> bool(xml.etree.ElementTree.fromstring(”<foo>Hello<foo”))
False
>>> bool(xml.etree.ElementTree.fromstring(”<foo><x /><foo>”))
Posted 10 Sep 2008 at 7:39 pm ¶True
Post a Comment