10-21-2004

Poking around some more with Lucas-Kanade feature tracking – adjusted it to draw points instead of stripes, and played around with some other values. The input image the algorithm is working off of is based on the background subtraction method shown in a few other examples, so I tweaked the threshold levels to match what I’ve been using in the other files (more on this in a bit).: 200 levels, minimum at 0, maximum at 200.

This seems to be as clean as I can get the background subtraction for now. These settings seem to filter out most of the noise, but don’t distinguish between screen black and shadow black. From a graphic standpoint, I think grays darker than 60% are a bad idea, because they are too easily mistaken for shadows. We may see different results on switching to a rear-capture model; all in due time.

Since all but one of my testing files uses some sort of test pattern on the screen that I have since been unable to filter out, I have been using the remaining “clean” test: my hand’s silhouette waving around on the screen. The feature tracker plots a set of points on the screen and attempts to track them in successive frames of input. At intervals, or if the image changes too much, it assigns new points.

Advantages:

“state-neutral” – doesn’t need a beginning reference frame to function, meaning that it can run continuously without having to stop. The tracking points seem to conform to the edges of the shadow pretty well at the right settings (more on this in a bit), though they don’t work out to a perfect edge.

Disadvantages:

Need to think about this some more – I’m concerned that I can’t get enough tracking points running simultaneously to calculate more complex shadows.

To do: put in some more complex shadows. See about extracting the tracking point position data to a set of coordinates. See about skewing and stretching the input frame to match the projector output proportions (is this called normalization in eyesWeb?)

Notes on settings:

For background subtraction: on the subtraction patch: 200 levels, minimum at 0, maximum at 200.

For point tracking, setting the threshold to somewhere between .22 and .3 seems to give fairly clean results without any too many jittery artifacts, while maintaining a good number of tracking points. Have to look up what this is actually adjusting, though.