Systems, methods, and apparatus for digital composition and/or retrieval

• US 10,346,465 B2
• Filed: 12/18/2014
• Issued: 07/09/2019
• Est. Priority Date: 12/20/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Gao, Dashan, Zhong, Xin
• Primary Examiner(s): Wilson, Nicholas R
• Application Number:
  US14/576,014
• Publication Number:
  US 20150178953A1
• Time in Patent Office: 1,664 Days
• Field of Search: None
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/56 having vectorial format
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44

Abstract

A method includes receiving information that identifies a reference position in a location space. The method also includes receiving data that identifies one among a plurality of candidate geometrical arrangements. The method also includes producing a representation that depicts a plurality of objects which are arranged, relative to the reference position in the location space, according to the identified candidate geometrical arrangement.

Claims

1. A method of producing a representation of an image, the method comprising:

receiving, at a processor, information that identifies a reference position in a location space, the location space corresponding to a pixel coordinate space, wherein the reference position indicates a position of a formation of multiple selected particular objects or a center of mass of the multiple selected particular objects;

receiving, at the processor, data that identifies a first candidate shape from a plurality of candidate shapes, the first candidate shape corresponding to a metric for a particular distribution of the multiple selected particular objects relative to the reference position in the location space, wherein each shape of the plurality of candidate shapes defines a corresponding area in which the multiple selected particular objects are renderable; and

based on the information and the data, producing the representation of the image, at the processor, by rendering at least one object of the multiple selected particular objects in the image, the at least one object rendered in one or more positions relative to the reference position and corresponding to the first candidate shape.

2. The method of claim 1, wherein the location space corresponds to a site of a public demonstration, and further comprising producing a representation of a second image, at the processor, by rendering a second set of objects in the second image, wherein the second set of objects comprise people at the public demonstration, and wherein the representation of the image corresponds to a graphical reproduction of the people at the public demonstration.

3. The method of claim 1, wherein rendering the at least one object in the image corresponds to performing operations to decode one or more images associated with the multiple selected particular objects based on one or more index values in a codebook, the codebook used to:

quantize the particular distribution of the multiple selected particular objects in each candidate shape of the plurality of candidate shapes; and

map determined positions of the multiple selected particular objects in the location space to a corresponding position in the pixel coordinate space.

4. The method of claim 1, wherein each of the multiple selected particular objects is selected from an image frame based on a user input, wherein producing the representation of the image corresponds to reproducing a particular image of an event associated with the at least one object, and wherein the first candidate shape comprises a polygon having a vertex that corresponds to the reference position.

5. The method of claim 1, further comprising:

receiving the image;

receiving, at the processor, second data that identifies a second candidate shape of the plurality of candidate shapes; and

rendering, by the processor, a second image, the second image rendered based on the information and based on the second data, the image and the second image associated with a movement of multiple instances of the at least one object within the location space.

6. The method of claim 1, wherein the first candidate shape corresponds to a region of the image in which multiple instances of the at least one object are rendered, wherein the location space comprises a three-dimensional space, and further comprising:

mapping, by the processor, the information to a corresponding position in the pixel coordinate space; and

receiving, at the processor, a data value indicating a number of instances of the at least one object.

7. The method of claim 1, wherein the data that identifies the first candidate shape indicates a total number of objects and indicates a distribution of multiple instances of the total number of objects among one or more regions of the location space, wherein each rendered instance of the multiple instances indicates a location of the at least one object relative to the reference position, and wherein each indicated location is associated with the first candidate shape.

8. The method of claim 1, wherein each shape of the plurality of candidate shapes is defined by the positions of the multiple selected particular objects.

9. The method of claim 1, wherein the metric indicates a value of a ratio based on a first number of the multiple selected particular objects in a first region of the location space and a second number of the multiple selected particular objects in a second region of the location space.

10. The method of claim 1, wherein the first candidate shape comprises a polygon having vertices, each vertex including a position of a different instance of the at least one object, wherein the information comprises an indication of Cartesian coordinates, polar coordinates, spherical coordinates, or cylindrical coordinates corresponding to a location in the pixel coordinate space, and further comprising mapping the information to the location in the pixel coordinate space.

11. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:

receiving information that identifies a reference position in a location space, the location space corresponding to a pixel coordinate space, wherein the reference position indicates a position of a formation of multiple selected particular objects or a center of mass of the multiple selected particular objects;

receiving data that identifies a first candidate shape from a plurality of candidate shapes, the first candidate shape corresponding to a metric for a particular distribution of the multiple selected particular objects relative to the reference position in the location space, wherein each shape of the plurality of candidate shapes defines a corresponding area in which the multiple selected particular objects are renderable; and

based on the information and the data, producing a representation of an image by rendering at least one object of the multiple selected particular objects in the image, the at least one object rendered in one or more positions relative to the reference position and corresponding to the first candidate shape.

12. The non-transitory computer-readable medium of claim 11, wherein the data that identifies the first candidate shape includes a description of the at least one object.

13. The non-transitory computer-readable medium of claim 11, wherein the at least one object comprises an avatar, wherein the data includes avatar position information associated with the location space, wherein the image comprises a digital image, and wherein the operations further comprise displaying the digital image.

14. The non-transitory computer-readable medium of claim 11, wherein the operations further comprise rendering a second object, and wherein the first candidate shape corresponds to a distribution of the second object and multiple instances of the at least one object in the location space.

15. The non-transitory computer-readable medium of claim 14, wherein the operations further comprise receiving a name label of a person associated with the second object.

16. The non-transitory computer-readable medium of claim 11, wherein the operations further comprise producing a second representation by rendering a plurality of objects based on the information and based on the data, and wherein the data that identifies the first candidate shape includes an indication of a degree of movement of multiple instances of the at least one object over a time interval.

17. An apparatus comprising:

means for receiving information that identifies a reference position in a location space, the location space corresponding to a pixel coordinate space, wherein the reference position indicates a position of a formation of multiple selected particular objects or a center of mass of the multiple selected particular objects;

means for receiving data that identifies a first candidate shape from a plurality of candidate shapes, the first candidate shape corresponding to a metric for a particular distribution of the multiple selected particular objects relative to the reference position in the location space, wherein each shape of the plurality of candidate shapes defines a corresponding area in which the multiple selected particular objects are renderable; and

means for rendering, based on the information and the data, at least one object of the multiple selected particular objects in an image to produce a representation of the image, the at least one object rendered in one or more positions relative to the reference position and corresponding to the first candidate shape.

18. The apparatus of claim 17, wherein the image comprises a digital image, wherein the location space corresponds to the image and the reference position comprises a particular point of the image, and wherein the means for receiving the information, the means for receiving the data, and the means for rendering are incorporated in a mobile communication device.

19. The apparatus of claim 17, wherein the means for rendering the at least one object in the image is further configured to render a plurality of objects in the image, the plurality of objects including the at least one object, and wherein the plurality of objects includes people, animals, or vehicles.

20. The apparatus of claim 17, wherein the data that identifies the first candidate shape includes a scale factor associated with the first candidate shape, an aspect factor associated with the first candidate shape, and an orientation direction of the first candidate shape.

21. The apparatus of claim 17, wherein the location space corresponds to a site of a rock concert.

22. The apparatus of claim 17, further comprising:

means for capturing the image;

means for storing the image;

means for receiving a description of the at least one object; and

means for retrieving, based on the description, a representation of the at least one object.

23. The apparatus according to claim 17, further comprising means for producing a second representation that depicts multiple instances of the at least one object arranged relative to a second reference position distinct from the reference position in the location space.

24. An apparatus comprising:

an interface configured to receive information that identifies a reference position in a location space and to receive data that identifies a first candidate shape from a plurality of candidate shapes, the location space corresponding to a pixel coordinate space, wherein the reference position indicates a position of a formation of multiple selected particular objects or a center of mass of the multiple selected particular objects, wherein the first candidate shape corresponds to a metric for a particular distribution of multiple selected particular objects relative to the reference position in the location space, and wherein each shape of the plurality of candidate shapes includes a corresponding area in which the multiple selected particular objects are renderable; and

a renderer configured to produce, based on the information and the data, a representation of an image by rendering at least one object of the multiple selected particular objects in the image, the at least one object rendered in one or more positions relative to the reference position and corresponding to the first candidate shape.

25. The apparatus of claim 24, wherein the image comprises a digital image, and further comprising:

a camera configured to capture the image;

a memory coupled the renderer and configured to store the image; and

a display coupled to the renderer and configured to display the representation of the image.

26. The apparatus of claim 24, wherein the renderer is further configured to render a plurality of objects, the plurality of objects including the at least one object, and wherein the at least one object corresponds to an avatar.

27. The apparatus of claim 24, wherein the first candidate shape indicates a position of the at least one object relative to the reference position, and wherein the renderer is further configured to render multiple instances of the at least one object based on terrain information corresponding to the position.

28. The apparatus of claim 24, wherein the interface is further configured to receive a description of the at least one object, and wherein the description is associated with a structure or an appearance of the at least one object.

29. The apparatus of claim 24, wherein the image corresponds to a graphical reproduction of an event, wherein the location space comprises a three-dimensional space or a ground plane, and wherein the interface and the renderer are incorporated into a mobile communication device.

30. The apparatus of claim 24, wherein the location space corresponds to a site of a holiday celebration, wherein the at least one object comprises a person, and wherein the image corresponds to a graphical reproduction of the holiday celebration.