Contextual behavior state filter for sensor registration

• US 8,818,036 B1
• Filed: 07/25/2012
• Issued: 08/26/2014
• Est. Priority Date: 07/25/2012
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Owechko, Yuri, Cheng, Shinko Y., Medasani, Swarup, Kim, Kyungnam
• Primary Examiner(s): Alavi, Amir
• Assistant Examiner(s): CESE, KENNY A
• Application Number:
  US13/558,257
• Time in Patent Office: 762 Days
• Field of Search: 382/103
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

Described is a system for registering a viewpoint of an imaging sensor with respect to a geospatial model or map. An image of a scene of a geospatial region comprising an object is received as input. The image of the scene is captured by a sensor having a current sensor state. Observation data related to the object'"'"'s state is received, wherein the observation data comprises an object behavior of the object given the geospatial region. An estimate of the current sensor state is generated using a probability of an observation from the observation data given the current sensor state x. Finally, the image of the scene is registered with a geospatial model or map based on the estimate of the current sensor state.

Claims

1. A system for sensor viewpoint registration, the system comprising:

one or more processors and a memory having instructions such that when the instructions are executed, the one or more processors perform operations of;

receiving an image of a scene of a geospatial region, wherein the image of the scene comprises at least one object, and wherein the image of the scene is captured by a sensor having a current sensor state x and a previous sensor state;

receiving observation data z related to the at least one object'"'"'s state, wherein the observation data z comprises at least one object behavior of the at least one object given the geospatial region;

generating an estimate of the current sensor state x using a probability of an observation from the observation data z given the current sensor state x; and

registering the image of the scene with a geospatial model or map based on the estimate of the current sensor state x,

wherein the system further performs an operation of converging a set of particles on the estimate of the current sensor state x using the observation data z and the previous sensor state, wherein the set of particles represent a set of candidate registration solutions.

2. The system for sensor viewpoint registration as set forth in claim 1, wherein the system further performs an operation of updating the set of particles {st(n), n=1 . . . N} and a set of weights of the particles {π

t(n), n=1 . . . N} from which the estimate of the current sensor slate x given the observation data z is determined as follows;

3. The system for sensor viewpoint registration as set forth in claim 2, wherein the system further performs an operation of determining the weights of each particle in the set of particles using a data likelihood function P(z|x), wherein the data likelihood function P(z|x) describes the probability of an observation being made at z given the current sensor state x.

4. The system for sensor viewpoint registration as set forth in claim 3, wherein the system further performs an operation of factorizing the data likelihood function P(z|x) into two densities:

a function of a distance from the current sensor state x which captures noise of the sensor, and a function that determines whether the observation data z is located in a high likelihood geospatial region, according to the following;

P(z|x)=P_a(z|x)P_b(z|x), where

5. The system for sensor viewpoint registration as set forth in claim 4, wherein the system further performs an operation of using the estimate of the current sensor state x to improve sensor measurements of the observation data z.

6. A computer-implemented method for sensor viewpoint registration, comprising an act of causing a data processor to execute instructions stored on a memory such that upon execution, the data processor performs operations of:

receiving an image of a scene of a geospatial region, wherein the image of the scene comprises at least one object, and wherein the image of the scene is captured by a sensor having a current sensor state x and a previous sensor state;

receiving observation data z related to the at least one object'"'"'s state, wherein the observation data z comprises at least one object behavior of the at least one object given the geospatial region;

generating an estimate of the current sensor state x using a probability of an observation from the observation data z given the current sensor state x; and

registering the image of the scene with a geospatial model or map based on the estimate of the current sensor state x,

further comprising an act of causing a data processor to perform an operation of converging a set of particles on the estimate of the current sensor state x using the observation data z and the previous sensor state, wherein the set of particles represent a set of candidate registration solutions.

7. The method for sensor viewpoint registration as set forth in claim 6, further comprising an act of causing a data processor to perform an operation of updating the set of particles {st(n), n=1 . . . N} and their weights {π

t(n), n=1 . . . N} from which the estimate of the current sensor slate x given the observation data z is determined as follows;

8. The method for sensor viewpoint registration as set forth in claim 7, further comprising an act of causing a data processor to perform an operation of determining the weights of each particle in the set of particles using a data likelihood function P(z|x), wherein the data likelihood function P(z|x) describes the probability of an observation being made at z given the current sensor state x.

9. The method for sensor viewpoint registration as set forth in claim 8, further comprising an act of causing a data processor to perform an operation of factorizing the data likelihood function P(z|x) into two densities:

a function of a distance from the current sensor state x which captures noise of the sensor, and a function that determines whether the observation data z is located in a high likelihood geospatial region, according to the following;

P(z|x)=P_a(z|x)P_b(z|x), where

10. The method for sensor viewpoint registration as set forth in claim 9, further comprising an act of causing a data processor to perform an operation of using the estimate of the current sensor state x to improve sensor measurements of the observation data z.

11. A computer program product for sensor viewpoint registration, the computer program product comprising:

computer-readable instruction means stored on a non-transitory computer readable medium that are executable by a computer having a processor for causing the processor to perform operations of;

receiving an image of a scene of a geospatial region, wherein the image of the scene comprises at least one object, and wherein the image of the scene is captured by a sensor having a current sensor state x and a previous sensor state;

receiving observation data z related to the at least one object'"'"'s state, wherein the observation data z comprises at least one object behavior of the at least one object given the geospatial region;

generating an estimate of the current sensor state x using a probability of an observation from the observation data z given the current sensor state x; and

registering the image of the scene with a geospatial model or map based on the estimate of the current sensor state x,

further comprising instruction means for causing a processor to perform an operation of converging a set of particles on the estimate of the current sensor state x using the observation data z and the previous sensor state, wherein the set of particles represent a set of candidate registration solutions.

12. The computer program product for sensor viewpoint registration as set forth in claim 11, further comprising instruction means for causing a processor to perform an operation of updating the set of particles {st(n), n=1 . . . N} and their weights {π

t(n), n=1 . . . N} from which the estimate of the current sensor slate x given the observation data z is determined as follows;

13. The computer program product for sensor viewpoint registration as set forth in claim 12, further comprising instruction means for causing a processor to perform an operation of determining the weights of each particle in the set of particles using a data likelihood function P(z|x), wherein the data likelihood function P(z|x) describes the probability of an observation being made at z given the current sensor state x.

14. The computer program product for sensor viewpoint registration as set forth in claim 13, further comprising instruction means for causing a processor to perform an operation of factorizing the data likelihood function P(z|x) into two densities:

a function of a distance from the current sensor state x which captures noise of the sensor, and a function that determines whether the observation data z is located in a high likelihood geospatial region, according to the following;

P(z|x)=P_a(z|x)P_b(z|x)

15. The computer program product for sensor viewpoint registration as set forth in claim 14, further comprising instruction means for causing a processor to perform an operation of using the estimate of the current sensor state x to improve sensor measurements of the observation data z.