More profiling

Turns out I was right about the error in the current pyRete version. I hand made a Rete network by connecting nodes from the refactored Rete implementation and it turned out to be as fast as (and sometimes faster than) the one based on dictionaries/functions.

Whilst the one based on dicts and functions is in some was easier to handle and deal with it also has some serious drawbacks. First and foremost it saves state at *each* node. This can probably be dealt with but then I'd have to write some boiler-plate code to handle when and when not to store tokens and that was the whole point of the implementation, to reduce the boiler-plate code. Secondly, it can be harder to wrap your head around since you'd have to look at both where the function is defined and where it is called in order to find out the exact behaviour of a node. It's not magic or anything but it may be confusing at first.

Given these two drawbacks of the dicts and functions implementation I decided that I wasn't going to use it. At all... unless I could find a way to perform function inlining and that the inlined version was faster than an optimized OO-Rete network.

So, now I've got no less than six implementations of a small Rete network[1]:
1) the current pyRete implementation (which is quite obviously just plain wrong).
2) the experimental dicts and functions implementation
3) the refactored OO-implementation
4) an optimized version of 3[2]
5) an implementation using function inlining based on 2
6) an optimized version of 5[2]

Here are the results (I've run these about ten times and the numbers below are typical) for 2 to 6 using 4000 facts:

2) took 6.58899998665 s => 79735 facts
3) took 5.98900008202 s => 79735 facts
4) took 5.70799994469 s => 79735 facts
5) took 5.63800001144 s => 79735 facts
6) took 5.52800011635 s => 79735 facts

Inlining apparently speeds up the matching but not with much and I'm not sure it's worth the effort. It would be cool though to generate a large blurb of code and see if it can be optimized by some other program like psyco or possibly one of the bytecodehacks or some such thing. I haven't looked for optimizers but I believe I will have to if I am going to get something out of the inlining implementation.

All in all. This week's been good fun looking at different implementations but it's time to get back to work and make sure that the pyRete code in the subversion repository works as expected.

[1] Here is the Rete network I'm using. Memory nodes are not shown.

#                       TYPE
#                       
#   TOP         ALPHA {--+     TYPE
#                            
#    +--} JOIN {--+    ALPHA {--+
#                      
#           +--} JOIN {--+
#                 
#                  +--} BETA
#                        
#                         +--} PROD

[2] I have removed a few "unneccessary" memory nodes. But since I believe that that sentence can easily be misunderstood I'll try to explain a bit more.
I have, for example removed storage of tokens at nodes which are never "read" in other parts of the network. For example the Beta Node places Tokens directly into the Production Node instead of going via a Beta Memory Node. It is "last" in the network and the Beta Memory would have fed the Production Node. Also, the first AlphaMemory has been converted into a LeftInputAdapter and sends Tokens directly to the second Join instead of going thru a DummyJoin with a DummyTopNode.