next up previous
Next: Bibliography

Thesis Proposal

Felix Tang

"Optical flow is the distribution of apparent velocities of movement of brightness patterns in an image"[1]
Optical flow has been modelled as:

Projective flow models the motion of planar objects precisely. This is what VideoOrbits does, it calculates the projective parameters that relate two images in a video sequence. The video sequence is taken with the Eyetap device. The EyeTap device is a Reality Mediator that is portable and allows the eye to function as both a camera and a display. The projective parameters that VideoOrbits calculates are a model of the motion of the eye/camera.

However, when there are multiple motions that comprise a large percentage of the image region the algorithm will estimate the motion as a linear combination of those present. It is proposed that applying techniques such as robust estimation, deterministic annealing and the expectation maximization (EM) algorithm would allow us to discern these multiple motions and make an intelligent decision.

These decisions would be dependent on how much of the image region these motions take up. In order to get more accurate estimates for the projective parameters of the EyeTap motion when there is a small percentage of extraneous motion, robust estimation could be applied. In cases when there are a couple of motions that comprise a significant amount of the image, deterministic annealing (hard clustering) could be useful in separating these motions. As well as providing an initial estimate to the EM-algorithm which would provide a more accurate result for these sets of projective parameters.

``Diminished reality is as important as Augmented reality, and both are possible with a new device called a Reality Mediator. The Reality Mediator allows the wearers visual perception of reality to be altered in such a way that the user can delete or diminish undesirable visual detritus from their perceived environment...By applying the VideoOrbits algorithm, the device can alter the light from a particular portion of the scene so a user perceives a computationally mediated version of the scene, giving rise to the possibility of computer controlled selectively diminished reality, allowing for additional information to be inserted without causing the user to experience information overload.''[2]
This separation of the motions as applied to VideoOrbits could possibly allow us to identify regions of a certain size that have common projective parameters in order to automate the technique described in [2]. So that simply clicking on a planar region would allow you to mediate that portion of the scene and diminish reality as wanted (e.g. removal of advertisements).

As a practical application this could all be used in an attempt to assist a growing number of people with vision problems (e.g. poor vision, poor night vision, blindness, etc.) by building a "Visual Memory Prosthetic" using the EyeTap and WearComp as in [3]. This has started by forming a community of networked individuals who would benefit from this type of device. Initially it has consisted of members of Professor Steve Mann's research group of which I am a member.




next up previous
Next: Bibliography
Debian User 2001-09-20