Awasu » Embedding Python: Compiling the Python interpreter
Sunday 30th November 2014 7:44 AM []

Python is a very cool scripting language, and one of the neat things it offers is a very easy way to embed it into other programs, either because you want to have a scripting interface, or because it's just easier to do some things in Python.

In this series of tutorials, we'll take a look at how to embed Python into your program, how to communicate with the Python interpreter, then a few other useful tips and tricks that most programs will need to use.

We'll be embedding Python 2.7.5 into a C++ program, using Visual Studio 2012 on Windows, although most of what we cover will also apply to Python 3 and other operating systems.

Do I need to compile Python?

Most of Python's functionality lives in a DLL (e.g. python27.dll) and when you run the Python interpreter using python.exe, it's actually just a thin wrapper around the DLL, accepting commands from the interactive console, or reading a script from a file, then passing them on to the DLL to do the real work (see right). Normally, in this kind of situation, you would be able to just load the DLL in your own program and call the various functions in it to do all the Python-y things you want it to do, but unfortunately, the API offered by the DLL has a flaw that prevents us from doing this.

Some of the public functions take a pointer to a FILE structure[1]This is the FILE* returned by fopen()., that the C runtime library uses to manage open files, but the contents of this structure varies from compiler to compiler. This means that you would need to use the same compiler for your program as the one that was used to compile the Python DLL[2]Otherwise your program would open a file and set up a FILE structure using one layout, while the Python DLL would then try to interpret it using a different layout, with the expected crashiness happening shortly thereafter.. But even if you used the same compiler, you would need to use the same version of that compiler, since the layout of the FILE structure can change from version to version. Even worse (!), even if you use the same version of the same compiler, the layout of the FILE structure can differ between debug and release builds.

The TL;DR is this: you have to compile Python yourself, using the same compiler that you use for your own program[3]Namely, the program you want to embed Python into., and you have to use a debug build of Python for the debug build of your program, and a release build for your release build.

This is tedious, but fortunately, compiling Python is quite straight-forward.

Compiling Python

Download the version of Python you want to embed from python.org and unpack it somewhere. There are several .sln files in the unpacked files, but the one we want is pcbuild.sln in the PCBuild directory[4]The other ones are for older versions of Visual Studio..

Open it in Visual Studio[5]It's a VS 2008 solution file, so VS 2012 will need to upgrade it. and the project we want is pythoncore, to build the Python DLL. It would be a good idea to also build the python project as well, to get the wrapper python.exe program, in case you want to test scripts using your version of Python.


Once the build has finished, you'll find the EXE and DLL files in the PCBuild directory.

You can see how small the EXE's are, compared to the DLL's.

Testing the new build

To check the new build of Python, we'll get it to run a quick script. Save the following Python script in a file somewhere, then use your newly-compiled python.exe to run it.

import sys
print sys.version

Your mileage will obviously vary, but note the build timestamp reported by Python, which should confirm that you are running the build of Python that you just compiled.


We now have our own build of Python, that we compiled ourselves, ready to be linked in to our own program.


   [ + ]

1. This is the FILE* returned by fopen().
2. Otherwise your program would open a file and set up a FILE structure using one layout, while the Python DLL would then try to interpret it using a different layout, with the expected crashiness happening shortly thereafter.
3. Namely, the program you want to embed Python into.
4. The other ones are for older versions of Visual Studio.
5. It's a VS 2008 solution file, so VS 2012 will need to upgrade it.
Have your say