11-1-2004

Bit more work on the Lucas-Kanade tracking patch, with some findings.

Threshold, as noted before, seems to deal with the threshold necessary to do a real-time reset of the tracking system and assign new points. Lowering the threshold basically increases the sensitivity of whatever mechanism causes a reset. Lowering the value, then, means that new tracking points are assigned more frequently to the test loop, resulting in a more accurate, more up-to-date edge definition.

This increased definition comes with the price of an increased computational load, though. Dropping the threshold all the way to .05 definitely results in sluggish playback, and that’s just with a loop with my hand in it. I’m running the system at around .2 now, with reasonable response times and fairly accurate tracking.

Ultimately, though, I don’t think the LK tracking algorithm, at least in this implementation, will ever be really good at tracking overly rapid motion. For the slower stuff, like where a user moves their hands with spread fingers, it seems good enough to isolate the gaps between the fingers.

Moving right along, I dug up a promising post on the discussion newsgroup, originally posted back in April 2004:

From: Anders Bo Pedersen To: eyesweb.discussion
Re: LKtracker problem

Use the MoCap.Item.Extract and then the Math.Matrix.Extract

Best
Anders

"trimard_" wrote in message
news:MDU4PpG$DHA.1828@lab0.infomus.dist.unige.it...
> Is there a way to extract the coordinates of the points made by the
> LKtracker? so that i can assign a move image action?
> thnx
>
>

-strapped this configuration into the LK Tracking example, the MoCap.Item.Extract fishing its input right off of the LK block’s output stream (it’s a copy of what is going on to the display stream).

-spliced Output.MoCap.Draw2DItems into the system near the end of the output stream, taking the output of the LK Tracker and the dot-drawing “view” block, then dropping in a second display block to show the visual output of that sucker.

The result: the dots in this output are replaced with little crosses and named labels. The implications:

1) The dots are tracked as discrete data objects, and aren’t the result of some creative visual filtering (kind of obvious in retrospect, but I had to be sure).

2) Since they are labeled, there may be a means of tracking semi-persistent data on the Flash side of things. We could, for example, somehow buffer previous frames in Flash, and look at where, say, point_6 was three frames back, and calculate some rudimentary motion data from it. We don’t have to, but it is another option there if we want it (which is A Good Thing).

3) There is no point three.

So, the data is there. MoCap.Item.Extract only seems to want to let me extract the data for one point at a time; I’ll have to go in and look at a way of pulling the whole list for every frame.