Getting started with XBOX 360 Kinect on OSX

Jun 21, 2012

A recent project of mine involves research and development with an XBOX 360 Kinect Sensor. Being a python guy, I started searching for python bindings to some OSX-supported framework. When you just get started looking into this area it can be a little confusing. There are a number of layers to the software stack to enable one to accomplish anything meaningful. This is just a short and general blog post outlining the basics of what I have discovered thus far, to help anyone else that might also be getting started.

At the lowest level, you need a driver. Something that can talk to the USB device that is the Kinect sensor. When you purchase the XBOX Kinect for Windows version of the sensor, and you are going to be developing on windows, much of this whole stack is provided to you by way of the Kinect SDK. But for the open source folks with the standard XBOX 360 sensor, you need to piece together your own solution.

Two drivers that I have discovered thus far:

I had started OpenKinect (libfreenect) because it comes with a python wrapper included. There were a few dependencies (I will talk about specific build steps in just a moment), but once I got this installed I was able to fire up the included glview app and see both depth and rgb data streaming in from my sensor. The role of these drivers is to provide simply the basic streams. That is, the depth, rgb, audio, and a few other sensor data streams. If your goal is to start tracking players, seeing skeletons, and registering gestures, the drivers are not enough. You would be required to make your own solution from this raw data at this phase in the game.

You would now want to look into middleware that can take the raw data and provide to you an API with higher level information. This would include finding users in the scene for you, tracking their body features, and giving you various events to watch for as the data streams.

Being that my goal was to have python bindings, I found my options to be much more limited than if I were going to be developing in C++. Wrappers have to exist for the framework you want. This is where my research really started ramping up. I spent a few days dealing wtih compiling issues, as well as having an actual bad power adapter that had to be exchanged. But all said and done, here is what I have settled on thus far…

Driver: PrimeSense Sensor
OpenNI Framework
A recent project of mine involves research and development with an XBOX 360 Kinect Sensor. Being a python guy, I started searching for python bindings to some OSX-supported framework. When you just get started looking into this area it can be a little confusing. There are a number of layers to the software stack to enable one to accomplish anything meaningful. This is just a short and general blog post outlining the basics of what I have discovered thus far, to help anyone else that might also be getting started.

Two drivers that I have discovered thus far:

Driver: PrimeSense Sensor
OpenNI Framework
OpenNI Modules for OpenNI
PyOpenNI python bindings

Install Details

Install homebrew (package manager)

http://mxcl.github.com/homebrew/

Install build tools

brew install cmake
brew install boost

Install python2.7

brew install python --framework

Suggestion: virtualenv Environment

This is not a requirement. But I recommend using virtualenv to set up an environment that specifically uses python2.7 so that you don’t have to fight with mixed dependencies and versions.

Create a virtualenv called “kinect”

pip install virtualenv
virtualenv --no-site-packages -p python2.7 kinect
cd kinect
source bin/activate

Install libusb (patched version)

There is a special patched version of the libusb library, in the form of a homebrew formula.

git clone https://github.com/OpenKinect/libfreenect.git

Now copy platform/osx/homebrew/libusb-freenect.rb -> /usr/local/Library/Formula/

brew install libusb-freenect

Install SensorKinect drivers

git clone https://github.com/avin2/SensorKinect.git

Then uncompress Bin/SensorKinect093-Bin-MacOSX-v*tar.bz2

sudo ./install.sh

Install OpenNI framework

Go here: http://www.openni.org/Downloads/OpenNIModules.aspx
Download Unstable Binary for MacOSX
sudo ./install.sh

Install NITE middleware (for OpenNI)

Go here: http://www.openni.org/Downloads/OpenNIModules.aspx
Download Unstable MIDDLEWARE of NITE for OSX
sudo ./install.sh

Install PyOpenNI

Be aware that on OSX, PyOpenNI requires a framework build of python 2.7+ and that you must build it for x86_64 specifically. Also, I was having major problems with cmake properly finding the python includes location. I had to suggest a fix, so please see here for the necessary corrections. I have referenced a patched fork of the repository below.

export CPPFLAGS="-arch x86_64"
git clone git://github.com/justinfx/PyOpenNI.git
mkdir PyOpenNI-build
cd PyOpenNI-build
cmake -D PYTHON_INCLUDE_DIR=/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Headers ../PyOpenNI
make

copy the lib/openni.so module to the python2.7 site-packages

Examples

Once you have everything installed, you can try out the examples that are included both in the NITE source location that you downloaded and also in the PyOpenNI source location:

NITE/Samples
PyOpenNI/examples

I also tried out ofxKinect (github.com/ofTheo/ofxKinect) on the side, which is an addon for OpenFrameworks. This is kind of a separate path than the OpenNI stack. I would say its more like an advanced offering of libfreenect. Using the included example, I recorded a 3D point cloud that is built on the fly from the RGB and depth data:

JustinFX.com