Mixed state models for automatic target recognition and behavior analysis in video sequences

Mixed state or hybrid state space systems are useful tools for various problems in computer vision. These systems model complicated system dynamics as a mixture of inherently simple sub-systems, with an additional mechanism to switch between the sub-systems. This approach of modeling using simpler systems allows for ease in learning the parameters of the system and in solving the inference problem. In this paper, we study the use of such mixed state space systems for problems in recognition and behavior analysis in video sequences. We begin with a dynamical system formulation for recognition of faces from a video. This system is used to introduce the simultaneous tracking and recognition paradigm that allows for improved performance in both tracking and recognition. We extend this framework to design a second system for verification of vehicles across non-overlapping views using structural and textural fingerprints for characterizing the identity of the target. Finally, we show the use of such modeling for tracking and behavior analysis of bees from video.