Smartphone-based methods and systems
First Claim
1. A method comprising the acts:
- receiving audio data using a microphone of a user'"'"'s portable device;
receiving image data using a camera of said device;
recognition-processing both the received audio and image data—
without a user being required to operate a user interface control to switch between an audio recognition mode and an image recognition mode, said recognition-processing being performed by a hardware processor configured to perform such act;
presenting a graphical user interface on the screen of the portable device, and displaying the image data received from the camera in a viewfinder region of said graphical user interface;
also presenting, in said graphical user interface, a stack of tiles, each of which corresponds to an item of recognized content, said stack including a first tile corresponding to a first item of recognized audio content, and a second tile corresponding to a second item of recognized visual content, wherein said user interface similarly represents items of recognized audio and visual content by tiles corresponding thereto;
said stack of tiles growing in size on said screen as successive items of content are recognized, until a first dimension is reached, after which older tiles disappear off the screen, wherein said first dimension assures that the viewfinder region is preserved for presentation of the image data from the camera;
each of the tiles in the stack having a payoff associated therewith, the user interface requiring user interaction with the first tile to initiate a first payoff corresponding to said item of recognized audio content, but not requiring user interaction with the second tile to initiate a second payoff corresponding to said item of recognized visual content, said second payoff instead being initiated automatically;
wherein said user interface, which similarly represents items of recognized audio and visual content by tiles corresponding thereto, differently treats initiations of payoffs for said items.
1 Assignment
0 Petitions
Accused Products
Abstract
Arrangements involving portable devices (e.g., smartphones and tablet computers) are disclosed. One arrangement enables a content creator to select software with which that creator'"'"'s content should be rendered—assuring continuity between artistic intention and delivery. Another utilizes a device camera to identify nearby subjects, and take actions based thereon. Others rely on near field chip (RFID) identification of objects, or on identification of audio streams (e.g., music, voice). Some technologies concern improvements to the user interfaces associated with such devices. For example, some arrangements enable discovery of both audio and visual content, without any user requirement to switch modes. Other technologies involve use of these devices in connection with shopping, text entry, and vision-based discovery. Still other improvements are architectural in nature, e.g., relating to evidence-based state machines, and blackboard systems. Yet other technologies concern computational photography. A great variety of other features and arrangements are also detailed.
57 Citations
8 Claims
-
1. A method comprising the acts:
-
receiving audio data using a microphone of a user'"'"'s portable device; receiving image data using a camera of said device; recognition-processing both the received audio and image data—
without a user being required to operate a user interface control to switch between an audio recognition mode and an image recognition mode, said recognition-processing being performed by a hardware processor configured to perform such act;presenting a graphical user interface on the screen of the portable device, and displaying the image data received from the camera in a viewfinder region of said graphical user interface; also presenting, in said graphical user interface, a stack of tiles, each of which corresponds to an item of recognized content, said stack including a first tile corresponding to a first item of recognized audio content, and a second tile corresponding to a second item of recognized visual content, wherein said user interface similarly represents items of recognized audio and visual content by tiles corresponding thereto; said stack of tiles growing in size on said screen as successive items of content are recognized, until a first dimension is reached, after which older tiles disappear off the screen, wherein said first dimension assures that the viewfinder region is preserved for presentation of the image data from the camera; each of the tiles in the stack having a payoff associated therewith, the user interface requiring user interaction with the first tile to initiate a first payoff corresponding to said item of recognized audio content, but not requiring user interaction with the second tile to initiate a second payoff corresponding to said item of recognized visual content, said second payoff instead being initiated automatically; wherein said user interface, which similarly represents items of recognized audio and visual content by tiles corresponding thereto, differently treats initiations of payoffs for said items. - View Dependent Claims (2, 3, 4, 5)
-
-
6. A smartphone comprising a processor, a memory, a screen, a microphone and a camera, the memory containing instructions configuring the smartphone to perform acts including:
-
producing audio data using the microphone; producing image data using the camera; recognition-processing both the audio and image data—
without a user being required to operate a user interface control to switch between an audio recognition mode and an image recognition mode;presenting a graphical user interface on the screen, and displaying image data from the camera in a viewfinder region of said graphical user interface; also presenting, in said graphical user interface, a stack of tiles, each of which corresponds to an item of recognized content, said stack including a first tile corresponding to a first item of recognized audio content, and a second tile corresponding to a second item of recognized visual content, said stack of tiles thereby similarly serving to represent instances of both audio and visual content recognition; said stack of tiles growing in size on said screen as successive items of content are recognized, until a first dimension is reached, after which older tiles disappear off the screen, wherein said first dimension assures that the viewfinder region is preserved for presentation of the image data from the camera; each of the tiles in the stack having a payoff associated therewith, the user interface requiring user interaction with the first tile to initiate a first payoff corresponding to said first item of recognized audio content, but not requiring user interaction with the second tile to initiate a second payoff corresponding to said second item of recognized visual content, said second payoff instead being initiated automatically; wherein said user interface, which similarly represents items of recognized audio and visual content by tiles corresponding thereto, differently treats initiations of payoffs for said items. - View Dependent Claims (7)
-
-
8. A non-transitory computer readable medium containing instructions for configuring a camera-equipped smartphone to perform acts including:
-
recognition-processing both received audio and image data—
without a user being required to operate a user interface control to switch between an audio recognition mode and an image recognition mode;presenting a graphical user interface on a screen of said smartphone, and displaying image data from the camera in a viewfinder region of said graphical user interface; also presenting, in said graphical user interface, a stack of tiles, each of which corresponds to an item of recognized content, said stack including a first tile corresponding to a first item of recognized audio content, and a second tile corresponding to a second item of recognized visual content, wherein said user interface similarly represents items of recognized audio and visual content by tiles corresponding thereto; said stack of tiles growing in size on said screen as successive items of content are recognized, until a first dimension is reached, after which older tiles disappear off the screen, wherein said first dimension assures that the viewfinder region is preserved for presentation of the image data from the camera; each of the tiles in the stack having a payoff associated therewith, the user interface requiring user interaction with the first tile to initiate a first payoff corresponding to said first item of recognized audio content, but not requiring user interaction with the second tile to initiate a second payoff corresponding to said second item of recognized visual content, said second payoff instead being initiated automatically; wherein said user interface, which similarly represents items of recognized audio and visual content by tiles corresponding thereto, differently treats initiations of payoffs for said items.
-
Specification