Methods and systems for content processing

• US 8,805,110 B2
• Filed: 01/21/2011
• Issued: 08/12/2014
• Est. Priority Date: 08/19/2008
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rhoads, Geoffrey B., Rodriguez, Tony F., Lord, John D., MacIntosh, Brian T., Rhoads, Nicole, Conwell, William Y.
• Primary Examiner(s): Akhavannik, Hadi
• Assistant Examiner(s): RASHIDIAN, MOHAMMAD M
• Application Number:
  US13/011,618
• Publication Number:
  US 20110212717A1
• Time in Patent Office: 1,299 Days
• Field of Search: 382/100, 382/103, 382/106, 382/107, 382/115, 382/117, 382/118, 382/190, 382/199, 382/209, 382/255, 382/276, 348/61, 348143-160, 348/273, 348/376, 348/208.15, 34833301-33304, 348/333.11, 348/240.99, 348/240.03, 348/208.4, 455/403, 455/418, 455/420, 356/3, 726/17, 707/687, 707/690, 707/769, 707770-771
• US Class Current: 382/255
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/9554 by using bar codes
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: G08C 2201/93 Remote control using other portable devices, e.g. mobile phone, PDA, laptop
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 2013/0074 : Stereoscopic image analysis
• CPC Class Codes: H04N 2101/00 : Still video cameras
• CPC Class Codes: H04N 2201/3278 : Transmission
• CPC Class Codes: H04N 23/611 : where the recognised objects include parts of the human body
• CPC Class Codes: H04W 88/02 : Terminal devices

Abstract

Mobile phones and other portable devices are equipped with a variety of technologies by which existing functionality can be improved, and new functionality can be provided. Some aspects relate to visual search capabilities, and determining appropriate actions responsive to different image inputs. Others relate to processing of image data. Still others concern metadata generation, processing, and representation. Yet others concern user interface improvements. Other aspects relate to imaging architectures, in which a mobile phone'"'"'s image sensor is one in a chain of stages that successively act on packetized instructions/data, to capture and later process imagery. Still other aspects relate to distribution of processing tasks between the mobile device and remote resources (“the cloud”). Elemental image processing (e.g., simple filtering and edge detection) can be performed on the mobile phone, while other operations can be referred out to remote service providers. The remote service providers can be selected using techniques such as reverse auctions, through which they compete for processing tasks. A great number of other features and arrangements are also detailed.

Claims

1. A method comprising:

configuring a GPU of a mobile device to implement plural stream processors, each examining a sub-part of imagery captured by the mobile device to determine presence of an oval shape; and

thereafter, re-configuring said GPU to examine sub-parts of the imagery determined to have an oval shape therein, to seek a facial feature associated with an oval shape;

wherein a first of the plural stream processors examines a first sub-part of the imagery captured by the mobile device to determine presence of an oval shape, and a second of the plural stream processors examines a second sub-part of said imagery to determine presence of an oval shape, and wherein the first and second sub-parts are different.

2. The method of claim 1 that includes configuring the GPU to implement N stream processors to determine presence of oval shapes in different sub-parts of the imagery, and thereafter configuring the GPU to implement M stream processors to seek facial features, where M<

N.

3. The method of claim 2 wherein the M stream processors are each configured to determine at least one of:

(a) positions of eye pupils, (b) nose location, or (c) distance across the mouth.

4. A mobile device including a camera, a graphics processing unit (GPU), and a memory, the memory containing software instructions to perform acts including:

configuring the GPU to implement plural stream processors, each examining a sub-part of imagery captured by the mobile device camera to determine presence of an oval shape; and

thereafter, re-configuring said GPU to examine sub-parts of the imagery determined to have an oval shape therein, to seek a facial feature associated with an oval shape;

wherein a first of the plural stream processors examines a first sub-part of the imagery captured by the mobile device to determine presence of an oval shape, and a second of the plural stream processors examines a second sub-part of said imagery to determine presence of an oval shape, and wherein the first and second sub-parts are different.

5. The mobile device of claim 4 in which said instructions perform acts including configuring the GPU to implement N stream processors to determine presence of oval shapes in different sub-parts of the imagery, and thereafter configuring the GPU to implement M stream processors to seek facial features, where M<

N.

6. The mobile device of claim 5 wherein the M stream processors are each configured to determine at least one of:

(a) positions of eye pupils, (b) nose location, or (c) distance across the mouth.

7. A non-transitory computer readable storage medium comprising software instructions for:

configuring a graphics processing unit (GPU) of a camera-equipped mobile device to implement plural stream processors, each examining a sub-part of imagery captured by the mobile device camera to determine presence of an oval shape; and

thereafter, re-configuring said GPU to examine sub-parts of the imagery determined to have an oval shape therein, to seek a facial feature associated with an oval shape;

wherein the software instructions configure a first of the plural stream processors to examine a first sub-part of the imagery captured by the mobile device to determine presence of an oval shape, and the software instructions configure a second of the plural stream processors to examine a second sub-part of said imagery to determine presence of an oval shape, and wherein the first and second sub-parts are different.

8. The computer readable storage medium of claim 7 in which said software instructions configure the GPU to implement N stream processors to determine presence of oval shapes in different sub-parts of the imagery, and thereafter configure the GPU to implement M stream processors to seek facial features, where M<

N.

9. The computer readable storage medium of claim 8 wherein the M stream processors are each configured to determine at least one of:

(a) positions of eye pupils, (b) nose location, or (c) distance across the mouth.