Comparing Types in Python 3
Say you're set to do some Python metaprogramming and find yourself in a need of sorting a sequence of types—not objects, types. Read more to find out how Python 2 and 3 compare.
Join the DZone community and get the full member experience.
Join For Free Let's fire up a Python 2 prompt to try it:
$ python2.7
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
>>> class Foo(object): pass
...
>>> class Bar(object): pass
...
>>> class Baz(object): pass
...
>>> sorted([Bar, Baz, Foo])
[<class '__main__.Foo'>, <class '__main__.Bar'>, <class '__main__.Baz'>]
Looks good. How about Python 3?
$ python3.4
Python 3.4.3+ (3.4:4255ca2f5314, Apr 2 2015, 05:00:06)
[GCC 4.8.2] on linux
>>> class Foo: pass
...
>>> class Bar: pass
...
>>> class Baz: pass
...
>>> sorted([Bar, Baz, Foo])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unorderable types: type() < type()
Oops, what's going on? A bit of digging quickly reveals that comparisons for type objects were removed in Python 3. In Python 2, the type of type
has the tp_richcompare
slot populated with:
static PyObject*
type_richcompare(PyObject *v, PyObject *w, int op)
{
PyObject *result;
Py_uintptr_t vv, ww;
int c;
/* Make sure both arguments are types. */
if (!PyType_Check(v) || !PyType_Check(w) ||
/* If there is a __cmp__ method defined, let it be called instead
of our dumb function designed merely to warn. See bug
#7491. */
Py_TYPE(v)->tp_compare || Py_TYPE(w)->tp_compare) {
result = Py_NotImplemented;
goto out;
}
/* Py3K warning if comparison isn't == or != */
if (Py_Py3kWarningFlag && op != Py_EQ && op != Py_NE &&
PyErr_WarnEx(PyExc_DeprecationWarning,
"type inequality comparisons not supported "
"in 3.x", 1) < 0) {
return NULL;
}
/* Compare addresses */
vv = (Py_uintptr_t)v;
ww = (Py_uintptr_t)w;
switch (op) {
case Py_LT: c = vv < ww; break;
case Py_LE: c = vv <= ww; break;
case Py_EQ: c = vv == ww; break;
case Py_NE: c = vv != ww; break;
case Py_GT: c = vv > ww; break;
case Py_GE: c = vv >= ww; break;
default:
result = Py_NotImplemented;
goto out;
}
result = c ? Py_True : Py_False;
/* incref and return */
out:
Py_INCREF(result);
return result;
}
Note that type objects are compared by numerically comparing their PyObject
pointers. Also note the deprecation warning about type inequalities going away in Python 3. And indeed, if we peek into typeobject.c
in Python 3, the tp_richcompare
slot is empty.
But what is a poor meta-programmer to do? Use metaclasses, what else!
The solution is very simple, actually. All Python needs to generate full rich comparisons is for the types to have __lt__
defined. Again, not the objects, types. Defining __lt__
in Foo
, Bar
and friends will make objects of these types comparable. We're not concerned with that right now. We want the types themselves to be comparable. Therefore, __lt__
has to be defined in the type of Foo
, which is type
by default; we can use a metaclass to amend that.
Here's one possible solution:
>>> class _ComparableTypeMeta(type):
... def __lt__(self, other):
... return id(self) < id(other)
...
>>> class Foo(metaclass=_ComparableTypeMeta): pass
...
>>> class Bar(metaclass=_ComparableTypeMeta): pass
...
>>> class Baz(metaclass=_ComparableTypeMeta): pass
...
>>> sorted([Bar, Baz, Foo])
[<class '__main__.Foo'>, <class '__main__.Bar'>, <class '__main__.Baz'>]
Ah, much better. To make some family of types comparable, we provide a common metaclass that defines __lt__
. When Python invokes Foo
< Bar
(which is called in the process of sorting), _ComparableTypeMeta.__lt__ fires
, and compares the types based on their id
—similarly to the address comparison in Python 2.
As with anything involving metaclasses, this is a "power feature" that should be used sparingly.
To be fair, there's a much simpler solution for this particular problem. We can just pass a key argument to sorted as follows:
sorted([Bar, Baz, Foo], key=id)
It will then work without special metaclasses. However, sorted is not the only place where we may want to sort and compare types, and moreover, it may be buried in some generic code that handles all kinds of objects; the metaclass-based approach is more general.
For more information see:
- PEP 207 that introduced rich comparisons back in Python 2.1.
- Documentation of rich comparison methods in Python 3.
- Type objects in the Python 3 C API.
- Python objects, types, classes and instances - a glossary.
- The fundamental types of Python - a diagram.
- Python object creation sequence.
- Python metaclasses by example.
Finally, please let me know if you manage to dig up the original discussion (on python-dev perhaps, or an issue) where removing tp_richcompare
was decided upon. I'm curious to see what led to the decision.
Published at DZone with permission of Eli Bendersky. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments