Intuitive computing methods and systems

• US 10,666,784 B2
• Filed: 04/22/2019
• Issued: 05/26/2020
• Est. Priority Date: 10/28/2009
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rhoads, Geoffrey B., Rodriguez, Tony F.
• Primary Examiner(s): Marc, McDieunel
• Application Number:
  US16/390,798
• Publication Number:
  US 20190320056A1
• Time in Patent Office: 400 Days
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/023 Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04N 23/00 : Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/667 : Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/027 : using movement velocity, acceleration information
• CPC Class Codes: H04W 4/029 : Location-based management or tracking services

Abstract

A smart phone receives imagery depicting a scene, and overlays indicia (augmentations) corresponding to supplemental data that relate to one or more features in the scene. The overlaid indicia are narrowed from a larger universe of options based on two criteria. A first criterion comprises factors such as actions previously taken by the user or third parties expressing interest or disinterest in presented indicia, time of day, and topical preference data for the user. A second criterion comprises a setting of a user-settable visual verbosity control. A user can thereby vary how much information, identified using the first criteria, is overlaid on the displayed scene, by varying the setting of the visual verbosity control. A great number of other features and arrangements are also detailed.

Claims

1. A method comprising the acts:

receiving imagery depicting a scene; and

presenting said imagery depicting the scene to a user on a display, overlaid with indicia corresponding to supplemental data that relate to one or more features in the scene;

wherein the method further includes narrowing a universe of possible indicia for display, down to a smaller number of indicia, said narrowing being a function of first and second criteria, where;

the first criterion comprises one or more factors selected from the list;

actions previously taken by the user expressing interest or disinterest in presented indicia;

actions previously taken by one or more third parties expressing interest or disinterest in presented indicia;

time of day;

topical preference data for the user;

areas of the display in which indicia would respectively be overlaid; and

payments offered by one or more commercial parties for presentation of their indicia; and

the second criterion comprises a setting of a user-settable verbosity control;

wherein the user can vary how much information, identified using the first criterion, is overlaid on the displayed scene, by varying the setting of said verbosity control.

2. The method of claim 1 wherein said indicia comprise baubles.

3. The method of claim 1 in which some of said indicia are commercial in nature, and some are non-commercial.

4. The method of claim 3 that comprises limiting a number of said commercial indicia, in accordance with the user-settable verbosity control.

5. The method of claim 3 in which said limiting comprises establishing a maximum ratio of commercial indicia to non-commercial indicia, in accordance with the user-settable verbosity control.

6. The method of claim 1 wherein at least one of said indicia presented on the display is a result of an auction process, said auction process having been based at least in part on information about a context of the user.

7. In an augmented reality method practiced using a user'"'"'s portable system, a method of preventing a display of the portable system from being overrun by augmentations vying for the user'"'"'s attention, the method comprising the acts:

capturing imagery depicting a scene from the user'"'"'s physical environment, with a camera apparatus of the portable system;

recognizing multiple subjects depicted in the captured imagery;

identifying supplemental data that relate to plural of the subjects recognized in the depicted scene, including first supplemental data that relates to a first of said subjects;

combining the captured imagery with indicia corresponding to said identified supplemental data; and

presenting the combined imagery to the user on the display of said portable system;

wherein the method further includes a software-configured hardware processor narrowing a universe of possible indicia for display down to a smaller number of indicia, said narrowing being a function of first and second different criteria, where;

the first criterion comprises one or more factors selected from the list;

actions previously taken by the user expressing interest or disinterest in presented indicia;

actions previously taken by one or more third parties expressing interest or disinterest in presented indicia;

time of day;

topical preference data for the user;

areas of the display in which indicia would respectively be overlaid; and

payments offered by one or more commercial parties for presentation of their indicia; and

the second criterion comprises data created for the purpose of display control, namely data from a user-settable verbosity control;

wherein the user can vary how much information identified using the first criterion is overlaid on the displayed scene by varying the setting of said verbosity control.

8. The method of claim 7 wherein said indicia comprise baubles.

9. The method of claim 7 in which some of said indicia are commercial in nature, and some are non-commercial.

10. The method of claim 9 that comprises limiting a number of said commercial indicia, in accordance with a user-settable control.

11. The method of claim 9 in which said limiting comprises establishing a maximum ratio of commercial indicia to non-commercial indicia, in accordance with a user-settable control.

12. The method of claim 7 wherein at least one of said indicia presented on the display is a result of an auction process, said auction process having been based at least in part on information about a context of the user.

13. The method of claim 7 wherein at least one of said indicia is selected for presentation on the display based on data from a sensor in a portable device conveyed by the user, and also based on stored user history data.

14. The method of claim 7 in which said stored user history data comprises data about previous interactions by the user with indicia, wherein indicia that are evident favorites of the user are more likely presented.

15. The method of claim 7 wherein the appearance of one of said indicia evolves over time, as additional information is discerned about the subject to which it corresponds.

16. The method of claim 7 wherein the first criterion comprises actions previously taken by the user expressing interest in certain of the indicia.

17. The method of claim 7 wherein the first criterion comprises actions previously taken by the user expressing disinterest in certain of the indicia.

18. The method of claim 7 wherein the first criterion comprises actions previously taken by a third party expressing interest in certain of the indicia.

19. The method of claim 7 wherein the first criterion comprises actions previously taken by a third party expressing disinterest in certain of the indicia.

20. A portable system comprising:

a battery;

a camera portion;

a display;

a memory;

a processor powered by the battery and responsive to instructions stored in the memory;

the memory containing instructions that configure the system to perform acts including;

capturing imagery depicting a scene from the user'"'"'s physical environment, with said camera portion of the system;

recognizing multiple subjects depicted in the captured imagery;

identifying supplemental data that relate to plural of the subjects recognized in the depicted scene, including first supplemental data that relates to a first of said subjects;

combining the captured imagery with indicia corresponding to said identified supplemental data;

said instructions further configuring the system to perform acts including;

narrowing a universe of possible indicia for display down to a smaller number of indicia, said narrowing being a function of first and second different criteria, where;

the first criterion comprises one or more factors selected from the list;

actions previously taken by the user expressing interest or disinterest in presented indicia;

actions previously taken by one or more third parties expressing interest or disinterest in presented indicia;

time of day;

topical preference data for the user;

areas of the display in which indicia would respectively be overlaid; and

payments offered by one or more commercial parties for presentation of their indicia; and

the second criterion comprises data created for the purpose of display control, namely data from a user-settable verbosity control; and

presenting the combined imagery to the user on the display of said portable system;

wherein the user can vary how much information identified using the first criterion is overlaid on the displayed scene by varying the setting of said verbosity control.