So in order to do some serious image processing with the Intel PCSDK, you need OpenCV, and to do that, you need to convert your pictures from Intel PCSDK (PXCImage::ImageData) to OpenCV (IplImage and cv::Mat)and maybe to OpenGL textures, if you want to display them.
I’ve seen all kinds of conversions over the internet, and all of the included nasty pitches and borders and flipping images – but I think I’ve found something logical and useable.
First, let’s work on color pictures.
You need your usual elements:
PXCImage::ImageData mCImageData; //Color Image data..)
Where your acquiring (in CINDER, this is probably in your update()), call this:
IplImage* colorimg = cvCreateImageHeader(cvSize(640, 480), 8, 3);
cvSetData(colorimg, (uchar*)mCImageData.planes, 640*3*sizeof(uchar));
/*Now you can whatever you want with OpenCV*/
mLabelMap =gl::Texture(fromOcv(image)); //Here's how to get to OpenGL
Displaying this (in the draw() function if you are using CINDER), is easy as pie:
This was inspired by this forum post: http://software.intel.com/en-us/forums/topic/375707
where you can also read the parameters for the depth picture and the UV image:
IplImage* depthimg = cvCreateImageHeader(cvSize(320, 240), 16, 1);
IplImage* UVimg = cvCreateImageHeader(cvSize(320, 240), 32, 2);
PXCImage *colorImage = mCapture->QueryImage(images, PXCImage::IMAGE_TYPE_COLOR);
PXCImage *depthImage = mCapture->QueryImage(images, PXCImage::IMAGE_TYPE_DEPTH);
cvSetData(depthimg, (short*)depthImageData.planes, depthsize.width*sizeof(short)*1);
cvSetData(UVimg, (float*)depthImageData.planes, depthsize.width*2*sizeof(float));
The rest is the same.
With a bit of trial/API studying, I also found a solution for the blob image, use this:
if(mGesture->QueryBlobImage(PXCGesture::Blob::LABEL_SCENE, 0, &mImage)>=PXC_STATUS_NO_ERROR)
IplImage* blobimg = cvCreateImageHeader(cvSize(320, 240), 8, 1);
cvSetData(blobimg, (uchar*)mImageData.planes, 320*sizeof(uchar));
/*Do whatever you want with OpenCV*/
(By the way, if you want to access the labels for your hand (for example for thresholding), you need the following code:)
Edit: 6.3.2014, Updated a line for depthimage, see discussion below. You can also use the sample code there.