IMAGE PROCESSING ARCHITECTURES AND METHODS

• US 20130295894A1
• Filed: 07/03/2013
• Published: 11/07/2013
• Est. Priority Date: 08/19/2008
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rhoads, Geoffrey B., Rodriguez, Tony F.
• Granted Patent: US 8,849,259 B2
• Time in Patent Office: Days
• US Class Current: 455/414.1
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes
• CPC Class Codes: G06V 20/80 Recognising image objects characterised by unique random patterns
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04M 1/0202 Portable telephone sets, e.g. cordless phones, mobile phones or bar type handsets constructional features of telephone transmitters or receivers, e.g. of speakers or microphones H04M1/03
• CPC Class Codes: H04M 1/0264 for a camera module assembly
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/2133 Recording or reproducing at a specific moment, e.g. time interval or time-lapse
• CPC Class Codes: H04N 1/32101 Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/3225 of data relating to an image, a page or a document
• CPC Class Codes: H04N 2201/3274 Storage or retrieval of prestored additional information
• CPC Class Codes: H04N 2201/3278 Transmission
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/64 : Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04W 4/50 : Service provisioning or reconfiguring

Abstract

Cell phones and other portable devices are equipped with a variety of technologies by which existing functionality is improved, and new functionality is provided. Some aspects relate to imaging architectures, in which a cell phone'"'"'s image sensor is one in a chain of stages that successively act on instructions/data, to capture and later process imagery. Other aspects relate to distribution of processing tasks between the device and remote resources (“the cloud”). Elemental image processing, such as filtering and edge detection—and even some simpler template matching operations—may be performed on the cell phone. Other operations are referred out to remote service providers. The remote service providers can be identified using techniques such as a reverse auction, though which they compete for processing tasks. Other aspects of the disclosed technologies relate to visual search capabilities, and determining appropriate actions responsive to different image inputs. Still others concern metadata generation, processing, and representation. A great number of other features and arrangements are also detailed.

Claims

1-2. -2. (canceled)

3. A system comprising a cell phone and one or more remote processors for performing image-related processing, the cell phone including a microphone, a cellular radio transceiver, a memory, and an image sensor operative to capture frames of image information, the cell phone including a keyvector processing module operative to package image data from the sensor into keyvector packets—

at least some of which comprise less than a single frame of image data, the cell phone further comprising a routing module for routing certain keyvector packets for processing of image-related data within the cell phone, and for routing certain keyvector packets for processing of image-related data by a remote processor, the system further comprising a module remote from the cell phone, cooperating with the routing module, operative to distribute data from keyvectors to remote processors, and to collect processed data from the remote processors for return to the cell phone.

4. The system of claim 3 in which the image sensor comprises elements fabricated on a semiconductor substrate, the substrate also having fabricated thereon programmable circuitry configured in accordance with instructions in the memory to serve as the keyvector processor module.

5. A method of processing image data captured using an image sensor in a cell phone, comprising:

performing a first image processing operation on the captured image data using a processing module in the cell phone, yielding internally-processed image data;

evaluating a competition between two or more external service providers, and sending the internally-processed image data to a winner of said competition for further processing, the winning service provider delivering externally-processed image data; and

taking an action based on the externally-processed image data.

6. The method of claim 5 wherein a module in the cell phone generates plural sets of visual keyvector data, and wherein the method further includes:

routing one of said sets of keyvector data to the winning service provider; and

routing another of said sets of keyvector data to a different external service provider, identified without reference to a competition.

7. The method of claim 5 wherein the competition comprises a reverse auction.

8-17. -17. (canceled)

18. An image processing method including the acts:

obtaining first and second criteria;

selecting a software service to perform a given task connected with image processing—

from among multiple different software services that are each capable of performing the given processing task, said selecting being dependent on the first and second criteria; and

sending image-based data to the selected software service so that it can perform said given processing task;

wherein at least one of said acts is performed by a hardware processor in a user'"'"'s cell phone.

19. The method of claim 18 in which said task comprises a task selected from the list consisting of:

performing an FFT operation, performing a SIFT-related operation, performing a log-polar transform, computing a histogram, computing eigenvectors, and performing an edge detection operation.

20. The method of claim 18 that includes sending said image-based data to plural different software services that are each capable of performing the given processing task, before said selecting.

21. The method of claim 18 in which said multiple different software services include software services provided by plural different commercial vendors, who compete to perform said given task.

22. The method of claim 21 in which the software services provided by said plural different commercial vendors are performed at processors remote from said cell phone.

23. The method of claim 18 that includes weighting the first and second criteria in accordance with user preference data.

24. The method of claim 18 that includes initiating said selecting act in advance of the image-related data being available for performance of said task.

25. The method of claim 18 in which said selecting includes checking at least one of said received criteria against published data concerning different software services.

26. The method of claim 18 in which at least one of said criteria comprises a user preference criterion.

27. The method of claim 26 in which the user preference criterion comprises compliance with a green standard of corporate practice.

28. The method of claim 18 in which at least one of said criteria comprises a task performance criterion.

29. The method of claim 28 in which the task performance criterion comprises a required turnaround time.

30. The method of claim 18 that further includes presenting information on a display of the cell phone, the presented information being determined—

in part—

by performance of the given processing task by the selected software service.

31. The method of claim 18 that includes:

performing the method of claim 18 a first time, resulting in provision of image-based data to a first software service for performance of the given processing task; and

performing the method of claim 18 a second time, resulting in provision of image-based data to a second software service, which is different than the first software service, for performance of the given processing task;

wherein selection of said different first and second software services is due to different user contexts at the first and second times.