Identifying spatial locations of events within video image data

• US 9,189,688 B2
• Filed: 01/13/2015
• Issued: 11/17/2015
• Est. Priority Date: 03/19/2009
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Desimone, Michael J., Hampapur, Arun, Lu, Zuoxuan, Mercier, Carl P., Milite, Christopher S., Russo, Stephen R., Shu, Chiao-Fe, Tan, Chek K.
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US14/595,726
• Publication Number:
  US 20150161453A1
• Time in Patent Office: 308 Days
• Field of Search: 382/103, 382/107
• US Class Current: 1/1
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/7328 Query by example, e.g. a complete video frame or video sequence graphical querying G06F16/7335
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

An invention for identifying a spatial location of an event within video image data is provided. Disclosed are embodiments for detecting an object and obtaining trajectory data of a trajectory of the object within the video image data from a sensor device; converting the trajectory data into a contour-coded compressed image; generating, based on the trajectory data, a searchable code that contains a set of locations traversed by the trajectory of the object within the video image; associating the searchable code with the contour-coded compressed image in a database; and returning, in response to a query having a selected location that corresponds a location of the set of locations in the searchable code, an image of the trajectory data corresponding to the object based on the contour-coded compressed image in the database.

Claims

1. A method for identifying a spatial location of an event within video image data comprising:

detecting, by at least one computer device, an object and obtaining trajectory data of a trajectory of the object within the video image data from a sensor device;

converting, by the at least one computer device, the trajectory data into a contour-coded compressed image;

generating, by the at least one computer device based on the trajectory data, a searchable code that contains a set of locations traversed by the trajectory of the object within the video image;

associating, by the at least one computer device, the searchable code with the contour-coded compressed image in a database; and

returning, by the at least one computer device in response to a query having a selected location that corresponds a location of the set of locations in the searchable code, an image of the trajectory data corresponding to the object based on the contour-coded compressed image in the database.

2. The method according to claim 1, further comprising searching the database to identify a spatial location of the event within the video image data.

3. The method according to claim 2, the searching comprising:

converting the area of interest to a lossy query code; and

comparing the lossy query code to the searchable code of the trajectory data within the video image data.

4. The method according to claim 3, further comprising:

decompressing the contour-coded compressed image corresponding to the lossy query code based on the comparing; and

plotting the trajectory of the trajectory data within the video image data from the contour-coded compressed image.

5. The method according to claim 1, wherein the converting further comprises:

dividing the video image data into a plurality of pixel regions;

determining, for each pixel region in the plurality of pixel regions, whether the trajectory intersects any pixel in the pixel region; and

indicating, for each pixel region, a result of the determining in a memory location in the contour-coded compressed image that corresponds to the pixel region.

6. The method according to claim 5, the converting further comprising:

determining, for each pixel region that the trajectory intersects, a direction of the trajectory from a location in the video image associated with the pixel region; and

assigning a value that indicates the direction of the trajectory from the associated location to the memory location in the contour-coded compressed image that corresponds to each pixel region that the trajectory intersects.

7. The method according to claim 1, wherein the generating further comprises:

dividing the video image data into a plurality of pixel regions;

determining, for each pixel region in the plurality of pixel regions, whether the trajectory intersects any pixel in the pixel region; and

indicating, for each pixel region, a result of the determining in a memory location in the searchable code that corresponds to the pixel region.

8. A system for identifying a spatial location of an event within video image data comprising:

at least one processing device;

memory operably associated with the at least one processing device; and

a spatial representation tool storable in memory and executable by the at least one processing device, the spatial representation tool causing the at least one processing device to;

detect an object and obtaining trajectory data of a trajectory of the object within the video image data from a sensor device;

convert the trajectory data into a contour-coded compressed image;

generate, based on the trajectory data, a searchable code that contains a set of locations traversed by the trajectory of the object within the video image;

associate the searchable code with the contour-coded compressed image in a database; and

return, in response to a query having a selected location that corresponds a location of the set of locations in the searchable code, an image of the trajectory data corresponding to the object based on the contour-coded compressed image in the database.

9. The system of claim 8, the spatial representation tool further causing the at least one processing device to search the database to identify a spatial location of the event within the video image data.

10. The system of claim 9, the searching comprising:

converting the area of interest to a lossy query code; and

comparing the lossy query code to the searchable code of the trajectory data within the video image data.

11. The system according to claim 10, the spatial representation tool further causing the at least one processing device to:

decompress the contour-coded compressed image corresponding to the lossy query code based on the comparing; and

plot the trajectory of the trajectory data within the video image data from the contour-coded compressed image.

12. The system according to claim 8, wherein the converting further comprises:

dividing the video image data into a plurality of pixel regions;

determining, for each pixel region in the plurality of pixel regions, whether the trajectory intersects any pixel in the pixel region; and

indicating, for each pixel region, a result of the determining in a memory location in the contour-coded compressed image that corresponds to the pixel region.

13. The system according to claim 12, the converting further comprising:

determining, for each pixel region that the trajectory intersects, a direction of the trajectory from a location in the video image associated with the pixel region; and

assigning a value that indicates the direction of the trajectory from the associated location to the memory location in the contour-coded compressed image that corresponds to each pixel region that the trajectory intersects.

14. The system according to claim 8, wherein the generating further comprises:

dividing the video image data into a plurality of pixel regions;

determining, for each pixel region in the plurality of pixel regions, whether the trajectory intersects any pixel in the pixel region; and

indicating, for each pixel region, a result of the determining in a memory location in the searchable code that corresponds to the pixel region.

15. A computer-readable storage device storing computer instructions, which when executed, enables a computer system to identify a spatial location of an event within video image data, the computer instructions comprising:

detecting an object and obtaining trajectory data of a trajectory of the object within the video image data from a sensor device;

converting the trajectory data into a contour-coded compressed image;

generating, based on the trajectory data, a searchable code that contains a set of locations traversed by the trajectory of the object within the video image;

associating the searchable code with the contour-coded compressed image in a database; and

returning, in response to a query having a selected location that corresponds a location of the set of locations in the searchable code, an image of the trajectory data corresponding to the object based on the contour-coded compressed image in the database.

16. The computer-readable storage device according to claim 15, further comprising computer instructions for searching the database to identify a spatial location of the event within the video image data, the searching comprising:

converting the area of interest to a lossy query code; and

comparing the lossy query code to the searchable code of the trajectory data within the video image data.

17. The computer-readable storage device according to claim 16, further comprising computer instructions for:

decompressing the contour-coded compressed image corresponding to the lossy query code based on the comparing; and

plotting the trajectory of the trajectory data within the video image data from the contour-coded compressed image.

18. The computer-readable storage device according to claim 15, wherein the converting further comprises:

dividing the video image data into a plurality of pixel regions;

determining, for each pixel region in the plurality of pixel regions, whether the trajectory intersects any pixel in the pixel region; and

indicating, for each pixel region, a result of the determining in a memory location in the contour-coded compressed image that corresponds to the pixel region.

19. The computer-readable storage device according to claim 18, the converting further comprising:

determining, for each pixel region that the trajectory intersects, a direction of the trajectory from a location in the video image associated with the pixel region; and

assigning a value that indicates the direction of the trajectory from the associated location to the memory location in the contour-coded compressed image that corresponds to each pixel region that the trajectory intersects.

20. The computer-readable storage device according to claim 15, wherein the generating further comprises:

dividing the video image data into a plurality of pixel regions;

determining, for each pixel region in the plurality of pixel regions, whether the trajectory intersects any pixel in the pixel region; and

indicating, for each pixel region, a result of the determining in a memory location in the searchable code that corresponds to the pixel region.