Sensor-based mobile search, related methods and systems
First Claim
1. A visual search method comprising the acts:
- receiving image data corresponding to imagery of a scene, captured by a camera;
processing the received image data with at least first, second and third different baseline image processing operations, to thereby produce at least first, second and third different sets of processed data derived from the image data, said processing being performed using a particular apparatus, namely a portable system including said camera, a microphone, and a display;
opportunistically identifying, from a set of plural additional image processing operations, a smaller subset of one or more further image processing operations that should be automatically invoked, based at least in part on said processed data, at least one of the identified further processing operations having a computational cost greater than each of the first, second and third baseline image processing operations;
invoking said identified smaller subset of one or more further image processing operations; and
as a consequence of said further image processing operations, providing information concerning an object depicted in said imagery;
wherein different further image processing operations are identified and invoked, based on results of the baseline image processing operations, thereby providing an experience in which the portable system seems to respond intuitively and adapt image processing based on the scene from which the image data was captured.
0 Assignments
0 Petitions
Accused Products
Abstract
A smart phone senses audio, imagery, and/or other stimulus from a user'"'"'s environment, and acts autonomously to fulfill inferred or anticipated user desires. In one aspect, the detailed technology concerns phone-based cognition of a scene viewed by the phone'"'"'s camera. The image processing tasks applied to the scene can be selected from among various alternatives by reference to resource costs, resource constraints, other stimulus information (e.g., audio), task substitutability, etc. The phone can apply more or less resources to an image processing task depending on how successfully the task is proceeding, or based on the user'"'"'s apparent interest in the task. In some arrangements, data may be referred to the cloud for analysis, or for gleaning. Cognition, and identification of appropriate device response(s), can be aided by collateral information, such as context. A great number of other features and arrangements are also detailed.
-
Citations
54 Claims
-
1. A visual search method comprising the acts:
-
receiving image data corresponding to imagery of a scene, captured by a camera; processing the received image data with at least first, second and third different baseline image processing operations, to thereby produce at least first, second and third different sets of processed data derived from the image data, said processing being performed using a particular apparatus, namely a portable system including said camera, a microphone, and a display; opportunistically identifying, from a set of plural additional image processing operations, a smaller subset of one or more further image processing operations that should be automatically invoked, based at least in part on said processed data, at least one of the identified further processing operations having a computational cost greater than each of the first, second and third baseline image processing operations; invoking said identified smaller subset of one or more further image processing operations; and as a consequence of said further image processing operations, providing information concerning an object depicted in said imagery; wherein different further image processing operations are identified and invoked, based on results of the baseline image processing operations, thereby providing an experience in which the portable system seems to respond intuitively and adapt image processing based on the scene from which the image data was captured. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51)
-
-
52. An article of manufacture including a non-transitory computer-readable medium having instructions stored thereon that, responsive to execution by a computing device, cause the computing device to perform operations comprising:
-
receiving image data corresponding to imagery of a scene; processing the received image data with at least first, second and third different baseline image processing operations, to thereby produce at least first, second and third different sets of processed data derived from the image data; opportunistically identifying, from a set of plural additional image processing operations, a smaller subset of one or more further image processing operations that should be automatically invoked, based at least in part on said processed data, at least one of the identified further processing operations having a computational cost greater than each of the first, second and third baseline image processing operations; invoking said identified smaller subset of one or more further image processing operations; and as a consequence of said further image processing operations, providing a user with information concerning an object depicted in said imagery; wherein different further image processing operations are identified and invoked, based on results of the baseline image processing operations, thereby providing a user experience in which the computing device seems to respond intuitively and adapt image processing based on the scene to which the received image data corresponds.
-
-
53. A visual method comprising the acts:
-
receiving a set of image data corresponding to imagery captured by a camera; processing the received image data with at least first, second and third different initial image processing operations, to thereby produce at least first, second and third different sets of processed data derived from the image data, said processing being performed using a particular apparatus, namely a portable system including said camera, a microphone, and a display; based at least in part on said processed data, launching a further data processing operation; making an assessment concerning an output of the further data processing operation; and allocating resources to said further data processing operation in an amount based, at least in part, on said assessment; wherein the method further includes; cyclically performing the first, second and third initial processing operations multiple times, each time performing said initial processing operations on a different frame of captured imagery; and launching the further data processing operation on a particular frame of captured imagery, only when output data from one of said cycles indicates that said particular frame of captured imagery merits such further data processing operation.
-
-
54. An article of manufacture including a non-transitory computer-readable medium having instructions stored thereon that, responsive to execution by a computing device, cause the computing device to perform operations comprising:
-
receiving image data corresponding to imagery; processing the received image data with at least first, second and third different initial image processing operations, to thereby produce at least first, second and third different sets of processed data derived from the image data; based at least in part on said processed data, launching a further data processing operation; making an assessment concerning an output of the further data processing operation; and allocating resources to said further data processing operation in an amount based, at least in part, on said assessment; said operations further including; cyclically performing the first, second and third initial processing operations multiple times, each time performing said initial processing operations on a different frame of captured imagery; and
pg,115launching the further data processing operation on a particular frame of captured imagery, only when output data from one of said cycles indicates that said particular frame of captured imagery merits such further data processing operation.
-
Specification