Identifying spatial locations of events within video image data

• US 9,729,834 B2
• Filed: 08/22/2016
• Issued: 08/08/2017
• Est. Priority Date: 03/19/2009
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Desimone, Michael J., Hampapur, Arun, Lu, Zuoxuan, Mercier, Carl P., Milite, Christopher S., Russo, Stephen R., Shu, Chiao-Fe, Tan, Chek K.
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US15/242,699
• Publication Number:
  US 20160360156A1
• Time in Patent Office: 351 Days
• Field of Search: 382103, 382104, 382106, 382107
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/7328 Query by example, e.g. a complete video frame or video sequence graphical querying G06F16/7335
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

An invention for identifying a spatial location of an event within video image data is provided. Disclosed are embodiments for detecting an object and obtaining trajectory data of a trajectory of the object within the video image data from a sensor device; converting the trajectory data into a contour-coded compressed image; generating, based on the trajectory data, a searchable code that contains a set of locations traversed by the trajectory of the object within the video image; associating the searchable code with the contour-coded compressed image in a database; and returning, in response to a query having a selected location that corresponds a location of the set of locations in the searchable code, an image of the trajectory data corresponding to the object based on the contour-coded compressed image in the database.

Claims

1. A method for identifying a spatial location of an event within video image data comprising:

generating, by at least one computer device, based on trajectory data of a trajectory of an object within video image data, a searchable code that contains a set of locations traversed by the trajectory of the object within the video image, wherein the generating further comprises;

dividing the video image data into a plurality of pixel regions;

determining, for each pixel region in the plurality of pixel regions, whether the trajectory intersects any pixel in the pixel region; and

indicating, for each pixel region, a result of the determining in a memory location in the searchable code that corresponds to the pixel region;

converting, by the at least one computer device, the trajectory data into a contour-coded compressed image; and

returning from a database, in response to a query having a selected location that corresponds a location of the set of locations in the searchable code, an image of the trajectory data corresponding to the object based on the contour-coded compressed image in the database.

2. The method according to claim 1, further comprising searching the database to identify a spatial location of the event within the video image data.

3. The method according to claim 2, the searching comprising:

converting the area of interest to a lossy query code; and

comparing the lossy query code to the searchable code of the trajectory data within the video image data.

4. The method according to claim 3, further comprising:

decompressing the contour-coded compressed image corresponding to the lossy query code based on the comparing; and

plotting the trajectory of the trajectory data within the video image data from the contour-coded compressed image.

5. The method according to claim 1, wherein the converting further comprises:

dividing the video image data into a plurality of pixel regions;

determining, for each pixel region in the plurality of pixel regions, whether the trajectory intersects any pixel in the pixel region; and

indicating, for each pixel region, a result of the determining in a memory location in the contour-coded compressed image that corresponds to the pixel region.

6. The method according to claim 5, the converting further comprising:

determining, for each pixel region that the trajectory intersects, a direction of the trajectory from a location in the video image associated with the pixel region; and

assigning a value that indicates the direction of the trajectory from the associated location to the memory location in the contour-coded compressed image that corresponds to each pixel region that the trajectory intersects.

7. A system for identifying a spatial location of an event within video image data comprising:

at least one processing device;

memory operably associated with the at least one processing device; and

a spatial representation tool storable in memory and executable by the at least one processing device, the spatial representation tool causing the at least one processing device to;

generate, based on trajectory data of a trajectory of an object within video image data, a searchable code that contains a set of locations traversed by the trajectory of the object within the video image, wherein the generating further comprises;

dividing the video image data into a plurality of pixel regions;

determining, for each pixel region in the plurality of pixel regions, whether the trajectory intersects any pixel in the pixel region; and

indicating, for each pixel region, a result of the determining in a memory location in the searchable code that corresponds to the pixel region;

convert the trajectory data into a contour-coded compressed image; and

return from a database, in response to a query having a selected location that corresponds a location of the set of locations in the searchable code, an image of the trajectory data corresponding to the object based on the contour-coded compressed image in the database.

8. The system of claim 7, the spatial representation tool further causing the at least one processing device to search the database to identify a spatial location of the event within the video image data.

9. The system of claim 8, the searching comprising:

converting the area of interest to a lossy query code; and

comparing the lossy query code to the searchable code of the trajectory data within the video image data.

10. The spatial representation tool according to claim 9 further causing the at least one processing device to:

decompress the contour-coded compressed image corresponding to the lossy query code based on the comparing; and

plot the trajectory of the trajectory data within the video image data from the contour-coded compressed image.

11. The system according to claim 7, wherein the converting further comprises:

dividing the video image data into a plurality of pixel regions;

determining, for each pixel region in the plurality of pixel regions, whether the trajectory intersects any pixel in the pixel region; and

indicating, for each pixel region, a result of the determining in a memory location in the contour-coded compressed image that corresponds to the pixel region.

12. The system according to claim 11, the converting further comprising:

determining, for each pixel region that the trajectory intersects, a direction of the trajectory from a location in the video image associated with the pixel region; and

assigning a value that indicates the direction of the trajectory from the associated location to the memory location in the contour-coded compressed image that corresponds to each pixel region that the trajectory intersects.

13. A computer-readable storage device storing computer instructions, which when executed, enables a computer system to identify a spatial location of an event within video image data, the computer instructions comprising:

generating, based on trajectory data of a trajectory of an object within video image data, a searchable code that contains a set of locations traversed by the trajectory of the object within the video image, wherein the generating further comprises;

dividing the video image data into a plurality of pixel regions;

determining, for each pixel region in the plurality of pixel regions, whether the trajectory intersects any pixel in the pixel region; and

indicating, for each pixel region, a result of the determining in a memory location in the searchable code that corresponds to the pixel region;

converting the trajectory data into a contour-coded compressed image; and

returning from a database, in response to a query having a selected location that corresponds a location of the set of locations in the searchable code, an image of the trajectory data corresponding to the object based on the contour-coded compressed image in the database.

14. The computer-readable storage device according to claim 13, further comprising computer instructions for searching the database to identify a spatial location of the event within the video image data, the searching comprising:

converting the area of interest to a lossy query code; and

comparing the lossy query code to the searchable code of the trajectory data within the video image data.

15. The computer-readable storage device according to claim 14, further comprising computer instructions for:

decompressing the contour-coded compressed image corresponding to the lossy query code based on the comparing; and

plotting the trajectory of the trajectory data within the video image data from the contour-coded compressed image.

16. The computer readable storage device according to claim 13, wherein the converting further comprises:

dividing the video image data into a plurality of pixel regions;

determining, for each pixel region in the plurality of pixel regions, whether the trajectory intersects any pixel in the pixel region; and

indicating, for each pixel region, a result of the determining in a memory location in the contour-coded compressed image that corresponds to the pixel region.

17. The computer readable storage device according to claim 16, the converting further comprising:

determining, for each pixel region that the trajectory intersects, a direction of the trajectory from a location in the video image associated with the pixel region; and

assigning a value that indicates the direction of the trajectory from the associated location to the memory location in the contour-coded compressed image that corresponds to each pixel region that the trajectory intersects.