Multiple views and projective geometry

[Lindley]

I am attempting to recover information about the camera motion from a series of video frames. I am referencing the book Multiple View Geometry by Hartley & Zisserman.

I have successfully computed the fundamental matrix between two frames, and extracted the second camera matrix assuming the first is the identity. However, I'm not certain whether or not this is enough to recover information about the camera motion.

I know that it won't be possible to compute the distance the camera has moved in any absolute sense, since there is a scale ambiguity. However, I would at least like to recover deltas in camera attitude and direction of motion.

The Fundamental matrix is sufficient to compute rotation and translation (up to scale) in a projective sense, but I don't understand precisely what that means or whether it's good enough for my purposes. If I consider the camera calibration to be known, I can get the Essential matrix, from which it is possible to compute 4 possible rotation/translation solutions (some of which can be ruled out by other constraints). It's probably possible to estimate the calibration matrix from the video sequence before trying to navigate off of it, but I haven't done that yet.

Is anyone here familiar with this area and able to offer insight?

 

[jvicens]

your understanding of multiple view geometry and camera motion can be very helpful in this situation. Based on the information provided, it seems that you have a good understanding of the fundamentals and are on the right track.

To answer your question about what it means to compute rotation and translation in a projective sense, it means that the solutions you obtain will be valid as long as the camera remains within the projective space. This is because the fundamental matrix is based on projective geometry, which does not have a concept of absolute scale.

To address your concern about whether this is good enough for your purposes, it depends on what you are trying to achieve with the recovered camera motion information. If your goal is to estimate the camera's movement relative to its starting position, then the fundamental matrix and Essential matrix solutions should be sufficient. However, if you need to know the exact distance the camera has moved, then you will need to incorporate additional information, such as camera calibration or known reference points in the scene.

Furthermore, it is worth noting that the Essential matrix solutions may still have some ambiguity, even after taking into account the camera calibration. This is because there are still four possible solutions, as you mentioned, and some of them may be invalid depending on the constraints of your specific scenario.

In summary, it seems that you have a solid understanding of the concepts and techniques involved in recovering camera motion from video frames. It may be helpful to further explore the calibration process and consider incorporating additional constraints to refine your solutions. Good luck with your research!