Cross-correlation can only find translations in the image pairs, whereas LSM can match image patterns that have been undergone a full affine transformation. This is one of the main advantages of applying LSM the way we do it. However, since we are applying LSM and the cross-correlation approach on the same scale (pixel-wise), their results are closely linked: If cross-correlation did not provide a good initial estimate, LSM won't fix this, but if the initial guess from the cross-correlation is good, LSM won't change it significantly. Thus, in our case, applying LSM does not provide fundamental improvement of the tracking (it would, however, if issue #35 is implemented).
In general, affine transformations will in many cases look like translations on a small scale (this is especially true for scaling and shearing, less so for rotations). Thus, it may be interesting to apply the cross-correlation tracking for the initial estimate on a much smaller scale than the LSM. Like this, we could improve matching in images, where cross-correlation does not provide correct tracking (because of too large distortions), but features in general can still be identified.
A possible approach could even be to track multiple very small image patches via cross-correlation, estimate an affine transformation from their respective translations (as is done for the image alignment) and provide this transformation as an initial guess for a larger-scale LSM matching.
Cross-correlation can only find translations in the image pairs, whereas LSM can match image patterns that have been undergone a full affine transformation. This is one of the main advantages of applying LSM the way we do it. However, since we are applying LSM and the cross-correlation approach on the same scale (pixel-wise), their results are closely linked: If cross-correlation did not provide a good initial estimate, LSM won't fix this, but if the initial guess from the cross-correlation is good, LSM won't change it significantly. Thus, in our case, applying LSM does not provide fundamental improvement of the tracking (it would, however, if issue #35 is implemented).
In general, affine transformations will in many cases look like translations on a small scale (this is especially true for scaling and shearing, less so for rotations). Thus, it may be interesting to apply the cross-correlation tracking for the initial estimate on a much smaller scale than the LSM. Like this, we could improve matching in images, where cross-correlation does not provide correct tracking (because of too large distortions), but features in general can still be identified.
A possible approach could even be to track multiple very small image patches via cross-correlation, estimate an affine transformation from their respective translations (as is done for the image alignment) and provide this transformation as an initial guess for a larger-scale LSM matching.