Wristwatch based interface for augmented reality eyewear

• US 10,591,730 B2
• Filed: 08/25/2017
• Issued: 03/17/2020
• Est. Priority Date: 08/25/2017
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Jonathan M. Rodriguez, II
• Inventors: Rodriguez, II, Jonathan M.
• Primary Examiner(s): Pringle-Parker, Jason A
• Application Number:
  US15/686,512
• Publication Number:
  US 20190146219A1
• Time in Patent Office: 935 Days
• Field of Search: None
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0362 with detection of 1D translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Augmented reality eyewear devices allow users to experience a version of our “real” physical world augmented with virtual objects. Augmented reality eyewear may present a user with a graphical user interface that appears to be in the airspace directly in front of the user thereby encouraging the user to interact with virtual objects in socially undesirable ways, such as by making sweeping hand gestures in the airspace in front of the user. Anchoring various input mechanisms or the graphical user interface of an augmented reality eyewear application to a wristwatch may allow a user to interact with an augmented reality eyewear device in a more socially acceptable manner. Combining the displays of a smartwatch and an augmented reality eyewear device into a single graphical user interface may provide enhanced display function and more responsive gestural input.

Claims

1. A method for combining a display of an augmented reality eyewear device with a display of a smartwatch comprising:

establishing a wireless connection between an augmented reality eyewear device and a smartwatch;

running a first application on the augmented reality eyewear device and a second application on the smartwatch;

determining on the augmented reality eyewear device the position and orientation of the smartwatch;

rendering a graphical user interface of the second application on the smartwatch;

sending rendering data from the smartwatch to the augmented reality eyewear device;

processing the rendering data on the augmented reality eyewear device;

wherein the rendering data includes information regarding the rendering of the graphical user interface of the second application;

rendering a graphical user interface of the first application on the augmented reality eyewear device;

wherein the rendering of the graphical user interface of the second application is subtracted from the rendering of the graphical user interface of the first application;

displaying the graphical user interface of the second application on the smartwatch;

displaying the graphical user interface of the first application on the augmented reality eyewear device;

wherein at least a portion of the graphical user interface of the second application is displayed within the perimeter of the graphical user interface of the first application;

wherein the graphical user interface of the first application includes an empty area; and

wherein the empty area is of a position, a shape, and a size such that the graphical user interface of the second application appears to be substantially continuous with the graphical user interface of the first application.

2. The method of claim 1 wherein the second application is the same application as the first application.

3. The method of claim 2 further comprising determining on the augmented reality eyewear device that the second application is the same application as the first application.

4. The method of claim 1 wherein the second application is a smartwatch specific version of the first application.

5. The method of claim 4 further comprising determining on the augmented reality eyewear device that the second application is a smartwatch specific version of the first application.

6. The method of claim 1 wherein the following steps are performed in order:

determining on the augmented reality eyewear device the position and orientation of the smartwatch;

rendering a graphical user interface of the second application on the smartwatch;

sending rendering data from the smartwatch to the augmented reality eyewear device; and

processing the rendering data on the augmented reality eyewear device.

7. The method of claim 1 further comprising:

detecting one or more user gestures on the augmented reality eyewear device;

rendering a second graphical user interface of the first application in accordance with the one or more gestures;

rendering a second graphical user interface of the second application in accordance with the one or more gestures.

8. The method of claim 1 wherein the first and second application are ridesharing applications.

9. The method of claim 1 wherein the edges of the smartwatch display are faded.

10. A method for combining a display of an augmented reality eyewear device with a display of a smartwatch comprising:

running an application on a smartwatch;

determining on the smartwatch the position and orientation of the augmented reality eyewear device;

generating on the smartwatch a first rendering and a second rendering of a graphical user interface of the application;

wherein the first rendering comprises a perimeter, an interior, and an empty space within the interior;

wherein the second rendering comprises a perimeter and an interior;

wherein the perimeter of the first rendering is larger than the perimeter of the second rendering, at least some portion of the second rendering appears within the perimeter of the first rendering, and the size and shape of the empty space within the interior of the first rendering is substantially the same as the size and shape of the portion of the second rendering that appears within the perimeter of the first rendering;

sending the second rendering to the augmented reality eyewear device;

displaying the second rendering on the smartwatch and the first rendering on the augmented reality eyewear device;

wherein the displays of the augmented reality eyewear device and the smartwatch appear to be a single, substantially continuous, graphical user interface.

11. The method of claim 10 wherein the position and orientation of the augmented reality eyewear device is determined on the smartwatch by receiving on the smartwatch position and orientation data from the augmented reality eyewear device.

12. The method of claim 10 wherein the following steps are performed in order:

generating on the smartwatch a first rendering and a second rendering of a graphical user interface of the application;

sending the second rendering to the augmented reality eyewear device;

displaying the first rendering on the smartwatch and the second rendering on the augmented reality eyewear device.

13. The method of claim 10 wherein the edges of the smartwatch display are faded.

14. The method of claim 10 wherein the application is a ridesharing application.