The best answers to the question “Why is there no xrange function in Python3?” in the category Dev.
Recently I started using Python3 and it’s lack of
from time import time as t def count(): st = t() [x for x in xrange(10000000) if x%4 == 0] et = t() print et-st count()
from time import time as t def xrange(x): return iter(range(x)) def count(): st = t() [x for x in xrange(10000000) if x%4 == 0] et = t() print (et-st) count()
The results are, respectively:
Why is that? I mean, why
xrange has been removed? It’s such a great tool to learn. For the beginners, just like myself, like we all were at some point. Why remove it? Can somebody point me to the proper PEP, I can’t find it.
Python3’s range is Python2’s xrange. There’s no need to wrap an iter around it. To get an actual list in Python3, you need to use
If you want something that works with Python2 and Python3, try this
try: xrange except NameError: xrange = range
Some performance measurements, using
timeit instead of trying to do it manually with
First, Apple 2.7.2 64-bit:
In : %timeit collections.deque((x for x in xrange(10000000) if x%4 == 0), maxlen=0) 1 loops, best of 3: 1.05 s per loop
Now, python.org 3.3.0 64-bit:
In : %timeit collections.deque((x for x in range(10000000) if x%4 == 0), maxlen=0) 1 loops, best of 3: 1.32 s per loop In : %timeit collections.deque((x for x in xrange(10000000) if x%4 == 0), maxlen=0) 1 loops, best of 3: 1.31 s per loop In : %timeit collections.deque((x for x in iter(range(10000000)) if x%4 == 0), maxlen=0) 1 loops, best of 3: 1.33 s per loop
range really is a bit slower than 2.x
xrange. And the OP’s
xrange function has nothing to do with it. (Not surprising, as a one-time call to the
__iter__ slot isn’t likely to be visible among 10000000 calls to whatever happens in the loop, but someone brought it up as a possibility.)
But it’s only 30% slower. How did the OP get 2x as slow? Well, if I repeat the same tests with 32-bit Python, I get 1.58 vs. 3.12. So my guess is that this is yet another of those cases where 3.x has been optimized for 64-bit performance in ways that hurt 32-bit.
But does it really matter? Check this out, with 3.3.0 64-bit again:
In : %timeit [x for x in range(10000000) if x%4 == 0] 1 loops, best of 3: 3.65 s per loop
So, building the
list takes more than twice as long than the entire iteration.
And as for “consumes much more resources than Python 2.6+”, from my tests, it looks like a 3.x
range is exactly the same size as a 2.x
xrange—and, even if it were 10x as big, building the unnecessary list is still about 10000000x more of a problem than anything the range iteration could possibly do.
And what about an explicit
for loop instead of the C loop inside
In : def consume(x): ....: for i in x: ....: pass In : %timeit consume(x for x in range(10000000) if x%4 == 0) 1 loops, best of 3: 1.85 s per loop
So, almost as much time wasted in the
for statement as in the actual work of iterating the
If you’re worried about optimizing the iteration of a range object, you’re probably looking in the wrong place.
Meanwhile, you keep asking why
xrange was removed, no matter how many times people tell you the same thing, but I’ll repeat it again: It was not removed: it was renamed to
range, and the 2.x
range is what was removed.
Here’s some proof that the 3.3
range object is a direct descendant of the 2.x
xrange object (and not of the 2.x
range function): the source to 3.3
range and 2.7
xrange. You can even see the change history (linked to, I believe, the change that replaced the last instance of the string “xrange” anywhere in the file).
So, why is it slower?
Well, for one, they’ve added a lot of new features. For another, they’ve done all kinds of changes all over the place (especially inside iteration) that have minor side effects. And there’d been a lot of work to dramatically optimize various important cases, even if it sometimes slightly pessimizes less important cases. Add this all up, and I’m not surprised that iterating a
range as fast as possible is now a bit slower. It’s one of those less-important cases that nobody would ever care enough to focus on. No one is likely to ever have a real-life use case where this performance difference is the hotspot in their code.
One way to fix up your python2 code is:
import sys if sys.version_info >= (3, 0): def xrange(*args, **kwargs): return iter(range(*args, **kwargs))
range type works just like Python 2’s
xrange. I’m not sure why you’re seeing a slowdown, since the iterator returned by your
xrange function is exactly what you’d get if you iterated over
I’m not able to reproduce the slowdown on my system. Here’s how I tested:
Python 2, with
Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)] on win32 Type "copyright", "credits" or "license()" for more information. >>> import timeit >>> timeit.timeit("[x for x in xrange(1000000) if x%4]",number=100) 18.631936646865853
Python 3, with
range is a tiny bit faster:
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:57:17) [MSC v.1600 64 bit (AMD64)] on win32 Type "copyright", "credits" or "license()" for more information. >>> import timeit >>> timeit.timeit("[x for x in range(1000000) if x%4]",number=100) 17.31399508687869
I recently learned that Python 3’s
range type has some other neat features, such as support for slicing:
range(20, 60, 10)!