SMARTPHONE-BASED METHODS AND SYSTEMS
First Claim
1. A method comprising the acts:
- receiving audio data using a microphone of a user'"'"'s portable device;
receiving image data using a camera of said device;
recognition-processing both the received audio and image data—
without a user requirement to switch modes therebetween;
recognizing an item of audio content, and displaying a visual indicia related thereto, the indicia being selectable by the user to launch an online payoff related to the recognized audio content, said recognizing being performed by a hardware processor configured to perform such act; and
recognizing an item of visual content, and launching an online payoff related to the recognized visual content without waiting for user action.
1 Assignment
0 Petitions
Accused Products
Abstract
Arrangements involving portable devices (e.g., smartphones and tablet computers) are disclosed. One arrangement enables a content creator to select software with which that creator'"'"'s content should be rendered—assuring continuity between artistic intention and delivery. Another utilizes a device camera to identify nearby subjects, and take actions based thereon. Others rely on near field chip (RFID) identification of objects, or on identification of audio streams (e.g., music, voice). Some technologies concern improvements to the user interfaces associated with such devices. For example, some arrangements enable discovery of both audio and visual content, without any user requirement to switch modes. Other technologies involve use of these devices in connection with shopping, text entry, and vision-based discovery. Still other improvements are architectural in nature, e.g., relating to evidence-based state machines, and blackboard systems. Yet other technologies concern computational photography. A great variety of other features and arrangements are also detailed.
-
Citations
22 Claims
-
1. A method comprising the acts:
-
receiving audio data using a microphone of a user'"'"'s portable device; receiving image data using a camera of said device; recognition-processing both the received audio and image data—
without a user requirement to switch modes therebetween;recognizing an item of audio content, and displaying a visual indicia related thereto, the indicia being selectable by the user to launch an online payoff related to the recognized audio content, said recognizing being performed by a hardware processor configured to perform such act; and recognizing an item of visual content, and launching an online payoff related to the recognized visual content without waiting for user action. - View Dependent Claims (2)
-
-
3. A method comprising the acts:
-
receiving sensor data, the sensor data including audio data sensed using a microphone of a user'"'"'s portable device, and image data sensed using a camera of said device; applying three or more different recognition-processing techniques to the received sensor data in cyclical serial fashion or in parallel fashion, including applying at least one recognition-processing technique to audio data, and applying at least one recognition-processing technique to image data; and disabling one or more of said recognition-processing techniques based on context information indicating that the received sensor data is unsuitable for said one or more techniques, said context information being sensor-derived based on information from one or more sensors other than the camera. - View Dependent Claims (4, 5)
-
-
6. A method comprising the acts:
-
receiving ambient content information based on data from a sensor in a user'"'"'s portable system, at first and second times; invoking a recognition-processing technique on the content information at the first time; and not invoking said recognition-processing technique on the content information at the second time; wherein a processor in the portable system determines not to invoke the recognition-processing technique at the second time at least in part because of system context, the system context comprising battery state, or on-going processes from which resources should not be diverted.
-
-
7. A method comprising the acts:
-
receiving audio data using a microphone of a user'"'"'s portable device; receiving image data using a camera of said device; recognition-processing both the received audio and image data—
without a user requirement to switch modes therebetween;inferring, based on input data, that the user has a preference for discovering visual content over audio content, said inferring being performed by a processor configured to apply rules to said input data to reach an inference; notifying the user about recognition relating to the image data; and not notifying the user about recognition relating to the audio data. - View Dependent Claims (8, 9)
-
-
10. An extended discovery method comprising the acts:
-
capturing imagery depicting a subject of interest to a user, yielding image data, the capturing employing a camera-equipped device conveyed by the user; performing recognition processing on the image data, said processing yielding data identifying the subject; based on the identifying data, conducting an online investigation—
employing a variety of online resources—
into the subject, to develop a collection of diverse results corresponding to said subject; andpresenting said results using a display of a second device, and displaying said captured imagery in association with said presented results, the second device not being carried by the user. - View Dependent Claims (11, 12, 13)
-
-
14. A seeing and hearing discovery method comprising the acts:
-
receiving sensor data corresponding to both audio and imagery, from sensors in a portable apparatus; based on the sensor data, obtaining audio content identification data identifying a source of the audio, and presenting a first graphic that visually alerts a user to the obtained audio content identification data; based on the sensor data, obtaining visual content identification data identifying an object depicted in the imagery, and presenting a second graphic that visually alerts the user to the obtained visual content identification data, the second graphic being different than the first graphic; storing the obtained audio content identification data and the visual content identification data; receiving user input requesting recall of information about historical discovery activity; and retrieving both the audio content identification data the visual content identification data as a consequence of said user input, and presenting third and fourth graphics respectively related thereto for user selection; wherein a single user input results in retrieval of both audio and visual content identification data; and wherein the fourth graphic is different than the second graphic.
-
-
15. A method comprising the acts:
-
determining, by context, whether a user is interested in audio or visual content; and controlling notifications about content recognition, based on a result of said determining; wherein at least one of said acts is performed by a hardware processor configured to perform said act(s).
-
-
16. A non-transitory computer readable medium having instructions stored thereon that, if executed by a computing device, cause the computing device to perform operations comprising:
-
presenting on a touch screen of the device a graphical user interface including first, second and third buttons; in connection with a user tap of the first button, launching an audio discovery, wherein said audio discovery comprises processing audio captured by a microphone of the device, said audio processing yielding data that is sent to a remote computer, which returns first identification data, said first identification data identifying a source of the audio; in connection with a user tap of the second button, launching an image discovery, wherein said image discovery comprises processing imagery captured by a camera of the device, said image processing yielding data that is sent to a remote computer, which returns second identification data, said second identification data identifying an object depicted in the imagery; and responsive to a user tap of the third button, recalling the first and second data about previous audio and image discoveries, and presenting associated audio and image discovery information in a scrollable list on said display. - View Dependent Claims (17, 18, 19, 20, 21)
-
-
22. A method comprising:
-
obtaining data indicating a user'"'"'s location; by reference to said data, identifying a collection of images that depict retail products near the user; sending certain of said images to a user device for presentation to the user; and associating at least one of said images or products together with the user, by data posted to a social network service.
-
Specification