Sensor-based mobile search, related methods and systems
First Claim
1. A method comprising the acts:
- receiving image data corresponding to imagery of a scene towards which a human user pointed a camera of a portable system, the imagery including regions of different luminance;
processing the received image data with at least first, second and third different baseline image processing operations, to thereby produce at least first, second and third different sets of processed data derived from the image data, said processing being performed by one or more processors in said portable device configured to perform such acts;
opportunistically identifying, from a set of plural additional image processing operations, a smaller subset of one or more further image processing operations that should be automatically invoked, based at least in part on said processed data, at least one of the identified further processing operations having a computational cost greater than each of the first, second and third baseline image processing operations;
invoking said identified smaller subset of one or more further image processing operations; and
as a consequence of said further image processing operations, providing the user with information concerning an object depicted in said imagery;
wherein different further image processing operations are identified and invoked, based on results of the baseline image processing operations, thereby providing a user experience in which the portable system seems to respond intuitively and adapt image processing based on the scene to which the user pointed the camera.
0 Assignments
0 Petitions
Accused Products
Abstract
A smart phone senses audio, imagery, and/or other stimulus from a user'"'"'s environment, and acts autonomously to fulfill inferred or anticipated user desires. In one aspect, the detailed technology concerns phone-based cognition of a scene viewed by the phone'"'"'s camera. The image processing tasks applied to the scene can be selected from among various alternatives by reference to resource costs, resource constraints, other stimulus information (e.g., audio), task substitutability, etc. The phone can apply more or less resources to an image processing task depending on how successfully the task is proceeding, or based on the user'"'"'s apparent interest in the task. In some arrangements, data may be referred to the cloud for analysis, or for gleaning. Cognition, and identification of appropriate device response(s), can be aided by collateral information, such as context. A great number of other features and arrangements are also detailed.
-
Citations
54 Claims
-
1. A method comprising the acts:
-
receiving image data corresponding to imagery of a scene towards which a human user pointed a camera of a portable system, the imagery including regions of different luminance; processing the received image data with at least first, second and third different baseline image processing operations, to thereby produce at least first, second and third different sets of processed data derived from the image data, said processing being performed by one or more processors in said portable device configured to perform such acts; opportunistically identifying, from a set of plural additional image processing operations, a smaller subset of one or more further image processing operations that should be automatically invoked, based at least in part on said processed data, at least one of the identified further processing operations having a computational cost greater than each of the first, second and third baseline image processing operations; invoking said identified smaller subset of one or more further image processing operations; and as a consequence of said further image processing operations, providing the user with information concerning an object depicted in said imagery; wherein different further image processing operations are identified and invoked, based on results of the baseline image processing operations, thereby providing a user experience in which the portable system seems to respond intuitively and adapt image processing based on the scene to which the user pointed the camera. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51)
-
-
52. An article of manufacture including a computer-readable medium having instructions stored thereon that, responsive to execution by a computing device, cause the computing device to perform operations comprising:
-
receiving image data corresponding to imagery of a scene, the imagery including regions of different luminance; processing the received image data with at least first, second and third different baseline image processing operations, to thereby produce at least first, second and third different sets of processed data derived from the image data; opportunistically identifying, from a set of plural additional image processing operations, a smaller subset of one or more further image processing operations that should be automatically invoked, based at least in part on said processed data, at least one of the identified further processing operations having a computational cost greater than each of the first, second and third baseline image processing operations; invoking said identified smaller subset of one or more further image processing operations; and as a consequence of said further image processing operations, providing a user with information concerning an object depicted in said imagery; wherein different further image processing operations are identified and invoked, based on results of the baseline image processing operations, thereby providing a user experience in which the computing device seems to respond intuitively and adapt image processing based on the scene to which the received image data corresponds.
-
-
53. A method comprising the acts:
-
receiving a set of image data corresponding to imagery captured by a camera in a portable device, the imagery including regions of different luminance; processing the received image data with at least first, second and third different initial image processing operations, to thereby produce at least first, second and third different sets of processed data derived from the image data, said processing being performed by one or more processors in said portable device configured to perform such acts; based at least in part on said processed data, launching a further data processing operation; making an assessment concerning an output of the further data processing operation; and allocating resources to said further data processing operation in an amount based, at least in part, on said assessment;
wherein the method further includes;cyclically performing the first, second and third initial processing operations multiple times, each time performing said initial processing operations on a different frame of captured imagery; and launching the further data processing operation on a particular frame of captured imagery, only when output data from one of said cycles indicates that said particular frame of captured imagery merits such further data processing operation.
-
-
54. An article of manufacture including a computer-readable medium having instructions stored thereon that, responsive to execution by a computing device, cause the computing device to perform operations comprising:
-
receiving image data corresponding to imagery, the imagery including regions of different luminance; processing the received image data with at least first, second and third different initial image processing operations, to thereby produce at least first, second and third different sets of processed data derived from the image data; based at least in part on said processed data, launching a further data processing operation; making an assessment concerning an output of the further data processing operation; and allocating resources to said further data processing operation in an amount based, at least in part, on said assessment;
said operations further including;cyclically performing the first, second and third initial processing operations multiple times, each time performing said initial processing operations on a different frame of captured imagery; and launching the further data processing operation on a particular frame of captured imagery, only when output data from one of said cycles indicates that said particular frame of captured imagery merits such further data processing operation.
-
Specification