No devices found error with OpenNI2

Problem

I installed OpenNI2 on a fresh Ubuntu system as described here. I plugged in a Primesense RD1.09 camera. I tried the default NIViewer app and it failed with this error:

openDevice failed:
DeviceOpen using default: no devices found

Solution

This error was quite a hard nut to crack. I could see that the Primesense camera was detected without errors in /var/log/syslog. I could see it being listed as ASUS when I run the lsusb command. And yet any OpenNI2 program would quit with the above error!

I could only solve this cause I remembered a similar error with SoftKinetic cameras. I tried the same solution as described here: creating a symbolic link to the libudev file.

After this, OpenNI2 programs began working immediately!

Tried with: OpenNI 2.2 x64, Primesense RD1.09 and Ubuntu 14.04

Advertisements

How to log in OpenNI 2

I use OpenNI 2 to read depth and color (RGB) images from a Primesense camera. Sometimes when my program does not work as expected, I like to enable the logging that is built into OpenNI 2.

  • To change the logging settings, the file that needs to be edited is Redist/OpenNI.ini.

  • Log messages can be printed to the console by setting LogToConsole to 1. If you run your program after this is enabled, it prints log messages on the console.

  • Log messages can be written to a log file by setting LogToFile to 1. If you run your program after this is enabled, it prints log messages to files in a Log subdirectory. One log file is created for every invocation of your program. The log filename is a timestamp, for example: 2015_08_18__20_27_53_16182.log

  • Four levels of verbosity is available for log messages. In increasing verbosity these are: error (3), warning (2), info (1) and verbose (0).

Tried with: OpenNI 2.2, Primesense RD1.09 and Ubuntu 14.04

How to update Primesense firmware

My Primesense RD1.09 cameras were having lots of trouble working under Linux. They had no problems working under Windows.

One of the suggestions I found online was to update their firmware. Note that the firmware update can only be done from a Windows computer. I followed these steps to update:

  1. Remove all OpenNI 1.x or 2.x SDK installed on Windows.
  2. Remove the driver for Primesense camera as described here.
  3. Install the latest OpenNI 2.x SDK by downloading it from here.
  4. Plug in the Primesense camera and ensure that it is detected in Device Manager. Strangely my camera is detected as a Primesense 1.08x device! I have no idea why this happens.
  5. Get the firmware update software from here. There are two versions of firmware, which one to pick? See below.
  6. Unzip the firmware and run the EXE file. It should change some settings and show a SUCCESS in the console. If you get a FAILURE, then make sure you followed the OpenNI removal and driver removal steps above.

Which firmware version to pick? There are two versions: one for RD108 and another for RD109 devices. My camera says RD1.09 on the back, but shows up in Windows as a 1.08x device! Which version of firmware should I use? I tried both and here is my experience. When I used RD109 firmware, the camera would give this error on Linux:

One or more of the following nodes could not be enumerated:
Device: PrimeSense/SensorV2/5.1.0.41: The device is not connected!

I went back and tried the RD108 firmware and the camera worked under Linux after that πŸ™‚

Tried with: OpenNI 2.2.0.33 and Windows 7 x64

Primesense on Linux

My Primesense RD1.09 camera that causes many troubles
My Primesense RD1.09 camera that causes many troubles

I have a Primesense RD1.09 camera that works without any problem under Windows using OpenNI 2.2.0.33. Under Linux though, it is a world of pain!

Investigation

After a lot of debugging, I have discovered that most of the problems arise if you have a computer which has USB 3.0 controllers or hubs. It does not matter if you plug in the Primesense into a USB 3.0 or USB 2.0 port. Just the existence of USB 3.0 on your computer means that the Linux kernel behaves differently.

After endless hours of plugging and unplugging into USB ports and poring over the errors in /var/log/syslog and the behavior of lsusb on 3 different computers, I believe it is the USB subsystem of Linux that is causing the problem. Linux kernel uses the XHCI module and on computers which have USB 3.0 controllers, it seems to be causing problems. For an excellent example see this scanner problem.

Additionally, if you have played around with different versions of OpenNI, that can also add another layer of bugs. Make sure you apt-get remove --purge when you remove OpenNI 1.x. Also make sure you manually remove the files installed by OpenNI 2.x (which does not have an uninstaller).

Symptoms

Check if you see of these symptoms when using Primesense in Linux:

  • You plug in Primesense and type lsusb. It should enumerate the USB devices in a second. If it takes longer, you have a computer with USB 3.0 ports and the buggy Linux kernel USB modules are having trouble with it.

  • Primesense is detected in lsusb and it appears as ASUS. But, running any sample viewer program returns in error.

  • Primesense is detected only half the times it is plugged in.

  • When the camera is detected, I run a program to read depth and color streams. Half the time, the color stream does not provide any frames!

  • Even when both depth and color frames are provided, the color frames have lower FPS and even freeze sometimes!

Solution

Both of these solutions worked for me:

  • On computer with USB 3.0, go to BIOS setup and disable Intel xHCI and EHCI. If your BIOS does not have these options, you are out of luck! 😦

  • Switch to an older computer which has only USB 2.0 controllers and ports.

Primesense worked flawlessly with the OpenNI 1.x packages (from Ubuntu) on the computers after I tried the above solutions! πŸ™‚

USB interface error with OpenNI and Primesense

Problem

I wanted to use a Primesense RD1.09 depth camera on my Ubuntu computer. I installed the OpenNI packages available in Ubuntu as described here. I plugged in the camera to a USB 2.0 port and tried the default OpenNI viewer and got this error:

$ Sample-NiSimpleViewer 
Open failed: Failed to set USB interface!

Solution

The camera seemed to be detected since it appears in lsusb listing. The solutions provided online did not work. In the end, I removed the installed OpenNI packages and installed the latest OpenNI2 package as described here. This worked! πŸ™‚

Tried with: Primesense RD1.09 and Ubuntu 14.04

How to install and use OpenNI2

I use OpenNI to read from a Primesense RD1.09 depth camera. Ubuntu has OpenNI library packages in its archive, but this is the older OpenNI 1.x. I find its API and device support lacking and so prefer to use OpenNI2. Also, there is no need to build the newer OpenNI2 from source since pre-built packages are available.

Installing and using OpenNI2 with Primesense is easy:

  • Download: The pre-built SDK file for your Linux can be downloaded from the OpenNI homepage. It is available for x86, x64 and ARM. I downloaded the x64 SDK file and unzipped it.

  • OpenNI directory: Your OpenNI code and programs will look for header and library files from this unzipped directory. So, move it to a suitable location. I like to keep it in ~/lib

  • Install: Run the installer from the unzipped directory:

$ sudo ./install.sh

All this does is to place the UDEV rules for the Primesense camera in /etc/udev/rules.d. Your camera is now ready to be used!

  • Camera detection: Plug in your depth camera to a USB port on your computer. Check if the camera is detected using lsusb command. My camera is listed as ASUS in this listing when it is detected.

  • Simple viewer: Run the simple viewer provided in OpenNI. You should be able to view depth and color (RGB) images now:

$ cd Tools
$ ./NiViewer
  • Device error: If NiViewer failed with a no devices found error, then a solution I discovered is described here.

  • Color image error: If NiViewer only shows depth image, but no color image: this sometimes happens. Unplug and plug in the camera and try again.

  • Header and libraries: You will find a OpenNIDevEnvironment file containing the paths for two environment variables: OPENNI2_INCLUDE and OPENNI2_REDIST. Use these include and library paths when you build your own C++ code that uses OpenNI. To link with OpenNI libraries add the -lOpenNI2 directive.

  • Sample C++ project: If you need help starting off, I have a CMake project with code for reading from Primesense camera here.

Tried with: OpenNI 2.2.0.33, Primesense RD1.09 and Ubuntu 14.04

Depth to color image registration fails in OpenNI

Problem

I wrote C++ code to read depth and color images from a Primesense camera using OpenNI 2, running on Windows. After initializing OpenNI and opening the Primesense device, I check if depth-to-color image registration mode is supported by calling psense_device.isImageRegistrationModeSupported(openni::IMAGE_REGISTRATION_DEPTH_TO_COLOR). This returns openni::STATUS_OK. So, next I set this image registration mode by calling psense_device.setImageRegistrationMode(openni::IMAGE_REGISTRATION_DEPTH_TO_COLOR). However, this fails!

Solution

I could not actually figure out why OpenNI reports that the device supports this mode, but fails setting it. I suspected that maybe the camera needs to see depth and color video streams being created before this mode is set. So, I set this mode as late as possible, after the depth-color streams are opened and configured. That seemed to work!

Tried with: Primesense camera, OpenNI 2, Visual Studio 2013 and Windows 7 Professional 64-bit

How to build OpenCV with OpenNI on Ubuntu

OpenNI is a library that provides support to capture color and depth images from cameras like Kinect and Primesense.

To build OpenCV with OpenNI support:

  • Install the OpenNI libraries and Primesense module:
$ sudo apt install libopenni-dev
$ sudo apt install libopenni-sensor-primesense-dev
  • Build OpenCV as described here. But, addΒ -D WITH_OPENNI=ON to the CMake compilation string used in OpenCV compilation.

Tried with:Β OpenCV 2.4.9,Β OpenNI unstable 1.5.4.0 and Ubuntu 14.04 LTS

How to get started with Kinect for Windows on Ubuntu using OpenNI

The Kinect can be connected to a computer and used for a lot of fun projects. OpenNI is a popular library that is used by many applications and libraries to use the Kinect. You can also use the OpenNI API in your programs to access the output of the Kinect. This post describes the steps I used to get OpenNI working with the Kinect for Windows sensor on Ubuntu:

Preliminary checks

  1. This post is about the Kinect for Windows sensor. It is not about the Kinect for Xbox 360 sensor. If you don’t know which type of Kinect you are using, read about the difference between the two types of Kinect.
  2. Plug the power plug of the Kinect to a power source. Plug the USB plug of the Kinect into your computer running Ubuntu. You should see the light on the Kinect blink slowly in green color.
  3. Ensure that the Kinect has been identified by the Linux kernel. You can check this by listing all the USB devices as described in this post. Check if the 3 devices described in that post are listed with the correct product and vendor IDs.

Install unstable branch of OpenNI 1.x

By default, OpenNI does not ship with the drivers for Kinect. Also, OpenNI 2.x does not seem to work with Kinect for Windows. To use the Kinect for Windows sensor on Ubuntu, it seems that the unstable version of OpenNI 1.x should be used.

Get the unstable branch of OpenNI 1.x code:

$ git clone https://github.com/OpenNI/OpenNI.git
$ cd OpenNI/
$ git checkout -b unstable origin/unstable

Compile the OpenNI code:

$ cd Platform/Linux/CreateRedist/
$ ./RedistMaker

Install the OpenNI headers, libraries and binaries on the system:

$ cd ../Redist/
$ cd OpenNI-Bin-Dev-Linux-x64-v1.5.4.0/
$ sudo ./install.sh

You now have OpenNI installed. However, it cannot yet communicate with your Kinect for Windows sensor. To get that working, we need to install the Kinect drivers.

Install SensorKinect drivers

These are drivers that have been modified to work with OpenNI 1.x (not OpenNI 2.x) on Linux. We need to compile and install this to use the Kinect. Compilation of these drivers needs OpenNI to be installed, which is why we did that step first.

Get the code of SensorKinect:

$ git clone https://github.com/avin2/SensorKinect.git
$ cd SensorKinect/

Compile the SensorKinect driver code:

$ cd Platform/Linux/CreateRedist/
$ ./RedistMaker

Install the SensorKinect drivers:

$ cd ../Redist/
$ cd Sensor-Bin-Linux-x64-v5.1.2.1/
$ sudo ./install.sh

You now have the drivers installed on your system.

Start playing with Kinect

It is time to check out if your Kinect for Windows sensor works on Ubuntu. The OpenNI directory has some tools that can be used to start playing with your Kinect. Go to OpenNI/Platform/Linux/Bin/x64-Release and try them out. For example, the NiViewer tool displays both the RGB and depth output from your Kinect.

Tried with: Kinect for Windows, SensorKinect 5.1.0.25, OpenNI unstable 1.5.4.0, Linux 3.2.0-48-generic and Ubuntu 12.04 LTS