Saturday, October 19, 2019

Tracking by detection based on ORB

The search of correspondence between corners using ORB:
Let say, in frame f1 we have a corner associated with some salient point. In the next frame f2 we find ORB (Oriented fast and Rotated Brief) keypoints in the whole picture. Monoslam predicts position of a salient point in f2 and hence we know a small search rectangle, where to search for corresponding ORB keypoints. We select all keypoints in this search rect as the candiates for correspondence. This candidates may be different corners and placed very closely to original keypoint in f1. Hence we can't rely on Euclidean distance between two corners and have to compute 256 bit ORB descriptors for all these candidates, which is very slow. Then we select the candidate with the ORB descriptor, most similar to ORB descriptor of original corner. If the similarity between these descriptors is weak (more than 64 out of 256 bits are different), we treat the original corner as having no correspondence.
The key problem of this technique is that for some keypoint in f1 there could be no match in f2. Then such corner is lost. Or we may find a wrong match: some keypoint may be close to original keypoint and has a quite similar descriptor. In this case the corner is unstable and occasionally jumps to a different close keypoint.
Watch demo.