SYSTEMS, METHODS, AND APPARATUS FOR GENERATING METADATA RELATING TO SPATIAL REGIONS OF NON-UNIFORM SIZE

• US 20150178930A1
• Filed: 12/18/2014
• Published: 06/25/2015
• Est. Priority Date: 12/20/2013
• Status: Abandoned Application

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Gao, Dashan, Zhong, Xin, Kandhadai, Ananthapadmanabhan Arasanipalai, Beith, Scott, Laver, Steven Douglas
• Application Number:
  US14/576,035
• Publication Number:
  US 20150178930A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/56 having vectorial format
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44

Abstract

A method of generating metadata includes using at least one digital image to select at least one among a plurality of objects, wherein the at least one digital image depicts the plurality of objects in relation to a physical space. The method also includes, in response to the selecting at least one object, determining a position of the at least one object in a location space. The method also includes, based on said determined position, producing metadata that identifies one among a plurality of separate regions that divide the location space, wherein said plurality of separate regions includes regions of unequal size.

Claims

1. A method of generating metadata, said method comprising:

using at least one digital image to select at least one among a plurality of objects, wherein the at least one digital image depicts the plurality of objects in relation to a physical space;

by at least one processor, and in response to said selecting at least one object, determining a position of said at least one object in a location space; and

by at least one processor, and based on said determined position, producing metadata that identifies one among a plurality of separate regions that divide the location space, wherein said plurality of separate regions includes regions of unequal size.

2. The method of generating metadata according to claim 1, wherein said determined position is based on positions of each of more than one object among the plurality of objects in one among (A) a pixel coordinate space of said at least one digital image and (B) the physical space.

3. The method of generating metadata according to claim 1, wherein said produced metadata indicates a region, among the plurality of regions, that includes said determined position.

4. The method of generating metadata according to claim 1, wherein said location space is a ground plane, and wherein said determining the position comprises projecting a position in a pixel coordinate space of said at least one digital image onto the ground plane.

5. The method of generating metadata according to claim 4, wherein said projecting is based on information from at least one orientation sensor.

6. The method of generating metadata according to claim 1, wherein said producing metadata comprises, based on the determined position, selecting a codebook from among a set of codebooks, wherein each codebook among the set of codebooks corresponds to a different one of the plurality of separate regions.

7. The method of generating metadata according to claim 1, wherein said method comprises:

associating the produced metadata with a frame of a video stream;

determining a second position of said at least one object in the location space;

based on said determined second position, producing second metadata that identifies a different one among the plurality of separate regions; and

associating the second metadata with a second frame of the video stream.

8. The method of generating metadata according to claim 1, wherein said at least one digital image is at least one frame of a video stream, and

wherein said method comprises tracking, within the video stream, at least one of said selected at least one object.

9. The method of generating metadata according to claim 1, wherein said location space is (A) a pixel coordinate space of the at least one digital image or (B) the physical space.

10. The method of generating metadata according to claim 1, wherein producing metadata comprises selecting an index into a quantization codebook that indicates a division of the location space into said plurality of separate regions.

11. The method of generating metadata according to claim 1, wherein said method comprises retrieving said at least one digital image from storage, based on a result of comparing the produced metadata to a search request that specifies one among the plurality of separate regions.

12. A non-transitory computer-readable medium storing instructions that when executed by a processor cause the processor to:

use at least one digital image to select at least one among a plurality of objects, wherein the at least one digital image depicts the plurality of objects in relation to a physical space;

determine, in response to said selecting at least one object, a position of said at least one object in a location space; and

produce, based on said determined position, metadata that identifies one among a plurality of separate regions that divide the location space, wherein said plurality of separate regions includes regions of unequal size.

13. The non-transitory computer-readable medium according to claim 12, wherein said determined position is based on positions of each of more than one object among the plurality of objects in one among (A) a pixel coordinate space of said at least one digital image and (B) the physical space.

14. The non-transitory computer-readable medium according to claim 12, wherein said location space is a ground plane, and wherein said determining the position comprises projecting a position in a pixel coordinate space of said at least one digital image onto the ground plane.

15. The non-transitory computer-readable medium according to claim 12, wherein said producing metadata comprises, based on the determined position, selecting a codebook from among a set of codebooks, wherein each codebook among the set of codebooks corresponds to a different one of the plurality of separate regions.

16. The non-transitory computer-readable medium according to claim 12, wherein said instructions include instructions that when executed by the processor cause the processor to:

associate the produced metadata with a frame of a video stream;

determine a second position of said at least one object in the location space;

produce, based on said determined second position, second metadata that identifies a different one among the plurality of separate regions; and

associate the second metadata with a second frame of the video stream.

17. The non-transitory computer-readable medium according to claim 12, wherein said at least one digital image is at least one frame of a video stream, and

wherein said instructions include instructions that when executed by the processor cause the processor to track, within the video stream, at least one of said selected at least one object.

18. An apparatus for generating metadata, said apparatus comprising:

means for using at least one digital image to select at least one among a plurality of objects, wherein the at least one digital image depicts the plurality of objects in relation to a physical space;

means for determining, in response to said selecting at least one object, a position of said at least one object in a location space; and

means for producing, based on said determined position, metadata that identifies one among a plurality of separate regions that divide the location space, wherein said plurality of separate regions includes regions of unequal size.

19. The apparatus according to claim 18, wherein said determined position is based on positions of each of more than one object among the plurality of objects in one among (A) a pixel coordinate space of said at least one digital image and (B) the physical space.

20. The apparatus according to claim 18, wherein said location space is a ground plane, and wherein said determining the position comprises projecting a position in a pixel coordinate space of said at least one digital image onto the ground plane.

21. The apparatus according to claim 18, wherein said producing metadata comprises, based on the determined position, selecting a codebook from among a set of codebooks, wherein each codebook among the set of codebooks corresponds to a different one of the plurality of separate regions.

22. The apparatus according to claim 18, wherein said apparatus comprises:

means for associating the produced metadata with a frame of a video stream;

means for determining a second position of said at least one object in the location space;

means for producing, based on said determined second position, second metadata that identifies a different one among the plurality of separate regions; and

means for associating the second metadata with a second frame of the video stream.

23. The apparatus according to claim 18, wherein said at least one digital image is at least one frame of a video stream, and

wherein said apparatus includes means for tracking, within the video stream, at least one of said selected at least one object.

24. An apparatus for generating metadata, said apparatus comprising:

a discriminator configured to use at least one digital image to select at least one among a plurality of objects, wherein the at least one digital image depicts the plurality of objects in relation to a physical space;

an object tracker configured to determine, in response to said selecting at least one object, a position of said at least one object in a location space; and

a selector configured to produce, based on said determined position, metadata that identifies one among a plurality of separate regions that divide the location space, wherein said plurality of separate regions includes regions of unequal size.

25. The apparatus according to claim 24, wherein said determined position is based on positions of each of more than one object among the plurality of objects in one among (A) a pixel coordinate space of said at least one digital image and (B) the physical space.

26. The apparatus according to claim 24, wherein said location space is a ground plane, and wherein said object tracker is configured to determine the position by projecting a position in a pixel coordinate space of said at least one digital image onto the ground plane.

27. The apparatus according to claim 24, wherein said selector is configured to produce said metadata by selecting, based on the determined position, a codebook from among a set of codebooks, wherein each codebook among the set of codebooks corresponds to a different one of the plurality of separate regions.

28. The apparatus according to claim 24, wherein said apparatus comprises a video stream editor configured to associate the produced metadata with a frame of a video stream, and

wherein said object tracker is configured to determine a second position of said at least one object in the location space, and

wherein said selector is configured to produce, based on said determined second position, second metadata that identifies a different one among the plurality of separate regions, and

wherein said video stream editor is configured to associate the second metadata with a second frame of the video stream.

29. The apparatus according to claim 24, wherein said at least one digital image is at least one frame of a video stream, and

wherein said object tracker is configured to track, within the video stream, at least one of said selected at least one object.

30. The apparatus according to claim 24, wherein said selector is configured to produce said metadata by selecting an index into a quantization codebook that indicates a division of the location space into said plurality of separate regions.