Creating an .exe for a CLIPS application

Every now and then someone (on CLIPSESG) asks about how to create a standalone binary (usually a Windows .exe) for a CLIPS application they wrote. The process for doing so is described in Section 5 of the CLIPS Advanced Programming Guide. But even though that may be the correct answer to such questions it isn't always what the poster wants to hear. At least that's the impression I get.

The problem is (probably, I'm guessing here), that the description is kind of brief, contains little or no examples and is written for an experienced (or at least decent) C programmer. Which, unfortunately, kind of misses the mark since it seems that most people that ask about how to create .exe files for their applications are (relatively) new programmers and probably feel a bit lost reading that section. I know I do.

Though I can see several reasons why distributing your CLIPS application as a binary file (in any format) is a bad idea. I also know that sometimes it's the only way, so I'm not going to question your motives or try to talk you out of it. Instead. Here is a step-for-step demonstration on how to do it using PyCLIPS.

Step 0. Know the difference.
I'm not sure whether or not the steps described in CLIPS APG Section 5 results in a binary that includes the CLIPS IDE (the Dialog window and other useful things). But what I am describing here does not! So if you rely on user input at runtime you're also going to have to create a GUI for your application. I've written about that before and all of those examples are also available as PyCLIPS downloads.

Step 1: Install the tools.
The first thing you need to to do is to get all of the tools needed. You're going to have to download and install Python, PyCLIPS and py2exe. If you also need a GUI or want to try either of the wxPython CLIPS demos you should get wxPython. Make sure to get the latest version of Python (2.5.2 as I'm writing this) and releases matching it (2.5) for the other projects. Installation should be very simple since all projects provide binaries and the installers pretty much take care of themselves.

When you're done with all of the installations you can make sure that everything went alright by opening up IDLE (it should be in your start menu) and try the following:

>>> import clips
>>> import py2exe
>>> import wx

If either of those raises an exception you're going to have to re-install. Installation instructions and trouble-shooting guidance can be found on the projects' websites and in their docs.

Step 2: Wrap your application in PyCLIPS.
Once you've got all the tools in place it's time to write a wrapper for your CLIPS application. I'll use the sudoku demo (sudoku.clp and solve.clp) as base for my example application.

I'll wrap it in a Python program that reads a file as input (containing an unsolved sudoku puzzle), triggers the CLIPS program to solve the puzzle, queries CLIPS for the answer and writes the solution to stdout.

The program is in essence a copy of the on_openfile and on_solve functions from the wxPython demo so I've copy-pasted most of it. You can download a .zip with the code here.

Step 3: Write a setup.py script.
The py2exe website has tutorials and examples on how to best use it so I'll be quite brief here.

In order to compile a Python program/script into a standalone Windows executable you're going to have write a small configuration file (setup.py) which specifies the options py2exe should work with. Py2exe is an extension to distutils so you should read those docs as well.

The setup.py which I'll use looks like this:

from distutils.core import setup
import py2exe

setup(console=['sudoku.py'], data_files=[('', ['sudoku.clp', 'solve.clp'])])

there are a lot more options available but since this is more of a teaser (or a shameless plug for Python and PyCLIPS depending on how you look at it) I'll let you look into that yourself.

Step 4: Compile the application.
All we need to do to generate the .exe is to run the setup.py script in a command prompt:

C:\Program\...\py2exe-demo>setup.py py2exe
running py2exe
...
*** copy data files ***
copying C:\Python25\lib\site-packages\py2exe\run.exe -> C:\Program\...\py2exe-demo\dist\sudoku.exe

*** binary dependencies ***
Your executable(s) also depend on these dlls which are not included,
you may or may not need to distribute them.

Make sure you have the license if you distribute any of them, and
make sure you don't distribute files belonging to the operating system.

ADVAPI32.dll - C:\WINDOWS\system32\ADVAPI32.dll
USER32.dll - C:\WINDOWS\system32\USER32.dll
SHELL32.dll - C:\WINDOWS\system32\SHELL32.dll
KERNEL32.dll - C:\WINDOWS\system32\KERNEL32.dll
msvcrt.dll - C:\WINDOWS\system32\msvcrt.dll

C:\Program\...\py2exe-demo>cd dist
C:\Program\...\py2exe-demo\dist>sudoku.exe ..\Game03.txt
5 9 7 8 3 4 1 2 6
2 6 8 9 1 5 7 3 4
3 1 4 7 2 6 5 8 9
8 7 3 2 4 1 9 6 5
9 2 5 6 7 3 4 1 8
1 4 6 5 8 9 2 7 3
4 5 2 1 6 8 3 9 7
7 8 9 3 5 2 6 4 1
6 3 1 4 9 7 8 5 2

Seems to work. But listing the contents of the dist directory might make you a bit disappointed:

C:\Program\...\py2exe-demo\dist>dir
...
2006-09-19  09:52            77 824 bz2.pyd
2008-06-04  19:42         1 034 699 library.zip
2006-03-16  16:19           348 160 MSVCR71.dll
2006-09-19  09:52         2 109 440 python25.dll
2008-01-18  21:08            39 997 solve.clp
2008-01-18  21:08             5 036 sudoku.clp
2008-06-04  19:42            18 944 sudoku.exe
2006-09-19  09:52           475 136 unicodedata.pyd
2006-09-19  09:52             4 608 w9xpopen.exe
2008-03-09  04:29         1 009 152 _clips.pyd
...

Neither of the .clp files have been compiled!

If the purpose of generating the .exe is to hide implementation details, such as the CLIPS program, then you're going to have to convert the .clp files into PyCLIPS programs. It's easy enough to do, but it's boring.

You have to go through each .clp file and
1) Either remove all comments or convert them into Python comments (use # instead of ;).
2) Indent all lines in the file with 4 spaces.
3) Insert import clips as the first line.
4) Insert def load(): as the second line.
5) Wrap all definitions with a clips.Build(""" ... """).
6) Rename the file to [filename]_clp.py.

After doing that you can remove the data_files option in setup.py and modify a few lines in sudoku.py. If we change:

clips.Load("sudoku.clp")
clips.Load("solve.clp")

to

import sudoku_clp, solve_clp
sudoku_clp.load()
solve_clp.load()

and run the compilation again, the problem is "fixed". Since all of the CLIPS constructs are now embedded in Python files they will be sucked into the .exe. The "fixed" code is available here.

The steps outlined above might at least be an alternative for those of us that aren't that well-versed in C/C++. I'm sure that the same, or at least something similar, can be done with other tools and languages - but that's for someone else to write about.

The code I've used as the example ought to be pretty simple to understand, but if you have any questions or if you think I've been too brief at some point, leave a comment or send me an e-mail and we'll sort it out.