Hidden markov model for camera handoff

• US 8,432,449 B2
• Filed: 08/13/2007
• Issued: 04/30/2013
• Est. Priority Date: 08/13/2007
• Status: Expired due to Fees

• Current Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Inventors: Chen, Francine R., Yang, Tao, Kimber, Donald G.
• Primary Examiner(s): Meky, Moustafa M
• Assistant Examiner(s): SHIU, HO T
• Application Number:
  US11/838,074
• Publication Number:
  US 20090046153A1
• Time in Patent Office: 2,087 Days
• Field of Search: None
• US Class Current: 348/159
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

An integrated method for modeling the handoff between cameras for tracking a specific individual, including: creating a representation of overlaps, gaps, and allowable movement among the fields of view of the cameras, wherein the representation is modeled as states in a Hidden Markov Model (HMM); training the HMM using video of people walking through the fields of view of the cameras; selecting a person to be tracked; and identifying the best camera area using the HMM.

Claims

1. An integrated method for modeling object tracking handoff between cameras, comprising, at a computer system with one or more processors and memory:

creating a representation of allowable movement of an object within a floor layout that includes one or more rooms and one or more halls, wherein the movement is monitored by a plurality of cameras, wherein;

the representation is modeled as a set of states in a Hidden Markov Model (HMM) including a first state, a second state, and a third state;

the first state corresponds to a first camera area that represents at least a portion of a field of view of a first camera in the floor layout; and

the second state corresponds to a first blind subregion of the floor layout that is not in a field of view of any of the plurality of cameras;

a third state that corresponds to a second blind subregion of the floor layout that is not in the field of view of any of the plurality of cameras, wherein the second blind subregion is different from the first blind subregion; and

training the HMM using video of activity in the fields of view of the cameras, wherein training the HMM includes determining a respective transition probability corresponding to movement of an object between the first camera area the first blind subregion and the second blind sub-region.

2. The method of claim 1, further comprising:

selecting a respective object to be tracked; and

identifying a best camera area for tracking the respective object, wherein the best camera area is determined based at least in part on the respective transition probability.

3. The method of claim 2, further comprising using similarity of object appearance as a feature for tracking the respective object in addition to the HMM.

4. The method of claim 2, further comprising using overlap of a region outlining the object with respective camera areas as a feature for tracking the respective object in addition to the HMM.

5. The method of claim 1, wherein Viterbi decoding is used to determine the best path between the set of states in the HMM.

6. The method of claim 1, wherein the HMM provides a formal framework for specifying allowable movement between views from the plurality of cameras.

7. The method of claim 1, wherein the first camera area includes a portion of the field of view of the first camera where an object can appear in or disappear from the field of view of the first camera.

8. The method of claim 7, wherein:

the set of states in the HMM include a third state that corresponds to a second camera area that represents at least a portion of a field of view of a second camera in the floor layout;

training the HMM includes determining a respective transition probability corresponding to movement of an object between the first camera area and the second camera area; and

the second camera area includes a portion of the field of view of the second camera where an object can appear in or disappear from the field of view of the second camera.

9. The method of claim 8, wherein the field of view of the first camera is distinct from the field of view of the second camera.

10. The method of claim 8, wherein the HMM does not include pixel-based correspondence points between the first camera area and the second camera area.

11. The method of claim 1, wherein change in velocity of the respective person is incorporated into the HMM.

12. The method of claim 1, wherein the HMM includes a plurality of states corresponding the first camera area, including a state representing a first direction that an object can move in the first camera area and a state representing a second direction, distinct from the first direction, that an object can move in the first camera area.

13. The method of claim 1, wherein:

the set of states in the HMM include a third state that corresponds to a second camera area that represents at least a portion of a field of view of a second camera in the floor layout;

training the HMM includes determining a respective transition probability corresponding to movement of an object between the first camera area and the second camera area; and

the method further comprises, after training the HMM;

in accordance with detection of a respective object in the first camera area, determining that the second camera area is a best camera area for tracking the respective object; and

after determining that the second camera area is a best camera area for tracking the respective object handing off tracking of the respective object to the second camera area.

14. The method of claim 1, wherein the first blind subregion is a room in the floor layout.

15. The method of claim 1, wherein the first blind subregion is a mode of transportation between floors of the floor layout.

16. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computer system with one or more processors, cause the computer system to:

create a representation of allowable movement of an object within a floor layout that includes one or more rooms and one or more halls, wherein the movement is monitored by a plurality of cameras, wherein;

the representation is modeled as a set of states in a Hidden Markov Model (HMM) including a first state, a second state, and a third state;

the first state corresponds to a first camera area that represents at least a portion of a field of view of a first camera in the floor layout; and

the second state corresponds to a first blind subregion of the floor layout that is not in a field of view of any of the plurality of cameras;

a third state that corresponds to a second blind subregion of the floor layout that is not in the field of view of any of the plurality of cameras, wherein the second blind subregion is different from the first blind subregion; and

train the HMM using video of activity in the fields of view of the cameras, wherein training the HMM includes determining a respective transition probability corresponding to movement of an object between the first camera area, the first blind subregion, and the second blind sub-region.

17. The computer readable storage medium of claim 16, wherein Viterbi decoding is used to determine the best path between the set of states in the HMM.

18. The computer readable storage medium of claim 16, wherein the HMM provides a formal framework for specifying allowable movement between views from the plurality of cameras.

19. The computer readable storage medium of claim 16, further comprising instructions to:

select a respective object to be tracked; and

identify a best camera area for tracking the respective object, wherein the best camera area is determined based at least in part on the respective transition probability.

20. The computer readable storage medium of claim 16, wherein:

the set of states in the HMM include a third state that corresponds to a second camera area that represents at least a portion of a field of view of a second camera in the floor layout; and

training the HMM includes determining a respective transition probability corresponding to movement of an object between the first camera area and the second camera area.

21. The computer readable storage medium of claim 16, wherein the first blind subregion is a room in the floor layout.

22. The computer readable storage medium of claim 16, wherein the first blind subregion is a mode of transportation between floors of the floor layout.

23. A computer system, comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

creating a representation of allowable movement of an object within a floor layout that includes one or more rooms and one or more halls, wherein the movement is monitored by a plurality of cameras, wherein;

the representation is modeled as a set of states in a Hidden Markov Model (HMM) including a first state, a second state, and a third state;

the first state corresponds to a first camera area that represents at least a portion of a field of view of a first camera in the floor layout; and

the second state corresponds to a first blind subregion of the floor layout that is not in a field of view of any of the plurality of cameras;

a third state that corresponds to a second blind subregion of the floor layout that is not in the field of view of any of the plurality of cameras, wherein the second blind subregion is different from the first blind subregion; and

training the HMM using video of activity in the fields of view of the cameras, wherein training the HMM includes determining a respective transition probability corresponding to movement of an object between the first camera area the first blind subregion, and the second blind sub-region.

24. The system of claim 23, further comprising instructions for:

selecting a respective object to be tracked; and

identifying a best camera area for tracking the respective object, wherein the best camera area is determined based at least in part on the respective transition probability.

25. The system of claim 23, wherein:

the set of states in the HMM include a third state that corresponds to a second camera area that represents at least a portion of a field of view of a second camera in the floor layout; and

training the HMM includes determining a respective transition probability corresponding to movement of an object between the first camera area and the second camera area.

26. The system of claim 23, wherein the first blind subregion is a room in the floor layout.

27. The system of claim 23, wherein the first blind subregion is a mode of transportation between floors of the floor layout.