2006-09-26

Faster type checks in Python

I've been fiddling around a bit with metaclasses in order to speed up type checks for the Alpha Network (and specifically the Object Type Node). I noticed previously that type checks take a lot more time than simple attribute checks. In a dynamically typed language such as Python that's not a big deal because if you're writing type checks you're probably doing something wrong anyway. The most pythonic way to go about it is to make sure that all required methods and/or attributes are present and go about your business (this is known as duck typing).

In a Rule Engine OTOH where you've given up control over the execution, duck typing might not be the best idea, even though there are ways to make it work.

Here's an interactive session cut-and-pasted from IDLE which I hope clarifies how I can implement an efficient ObjectTypeNode. This requires that all Fact objects inherit from pyRete.Fact, but I can live with that.

>>> class Meta(type):
... def __init__(cls, *args, **kwargs):
... print "# using Meta to create", cls.__name__
... setattr(cls, '__classname__', cls.__name__)
...

>>> class Fact(object):
... __metaclass__ = Meta
...

# using Meta to create Fact

>>> class Foo(Fact):
... pass

# using Meta to create Foo

>>> class Bar(Fact):
... pass

# using Meta to create Bar

>>> foo = Foo()
>>> bar = Bar()
>>> foo.__classname__, bar.__classname__
('Foo', 'Bar')

>>> import datetime
>>> def test1(obj):
... start = datetime.datetime.now()
... for i in range(1000000):
... if obj.__classname__ == 'Foo':
... pass
... stop = datetime.datetime.now()
... print stop - start
...

>>> test1(bar)
0:00:00.581000

>>> test1(foo)
0:00:00.591000

>>> def test2(obj):
... start = datetime.datetime.now()
... for i in range(1000000):
... if obj.__class__.__name__ == 'Foo':
... pass
... stop = datetime.datetime.now()
... print stop - start
...

>>> test2(bar)
0:00:01.051000

>>> test2(foo)
0:00:01.062000
Quite a difference in speed... did someone say premature optimization? ;-)

[2006-09-27] I may have posted too soon. Having tried the above in the "real" environment. It shows no increase in speed. Apparently I'm missing something... Damn, I was soo happy.

When I run a simplified test case I get the behaviour I expect.
It's faster (roughly 2 times faster) to check for the attribute __classname__ than __class__.__name__.
It's faster when the if statement fails (when asserting a Bar object) than when it has to execute the then statement(s) (when asserting a Foo object).

[2006-09-27] I've built and executed a number of more complex tests now (by actually more or less rewriting most of the Alpha Network).
I've changed __classname__ to _classname because that seems to speed things up a little.
I've also made sure that the attribute is set on each instance because that also seems to speed things up a little.

All in all though, things like method calls, try-except and if-else statements seem to have more impact on speed than my type-checking does. I guess my journey ends here, it was good though because now I've got a better way of registering property change listeners in Fact objects (using metaclasses).

Inga kommentarer: