Wearable Computer with Superimposed Controls and Instructions for External Device

• US 20130069985A1
• Filed: 09/21/2011
• Published: 03/21/2013
• Est. Priority Date: 09/21/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Wong, Adrian, Miao, Xiaoyu
• Granted Patent: US 8,941,560 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0189 Sight systems
• CPC Class Codes: G05B 19/124 using tapes, cards or discs with optically sensed marks or codes G05B19/128, G05B19/14 take precedence
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 2200/1613 Supporting arrangements, e.g. for filters or documents associated to a laptop display
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units

Abstract

A wearable computing device includes a head-mounted display (HMD) that provides a field of view in which at least a portion of the environment of the wearable computing device is viewable. The HMD is operable to display images superimposed over the field of view. When the wearable computing device determines that a target device is within its environment, the wearable computing device obtains target device information related to the target device. The target device information may include information that defines a virtual control interface for controlling the target device and an identification of a defined area of the target device on which the virtual control image is to be provided. The wearable computing device controls the HMD to display the virtual control image as an image superimposed over the defined area of the target device in the field of view.

Claims

1. A method comprising:

determining that a target device is within an environment of a wearable computing device comprising a head-mounted display (HMD), wherein the HMD provides a field of view in which at least a portion of the environment is viewable, and wherein the HMD is operable to display images superimposed over the field of view;

obtaining target device information related to the target device, wherein the target device information defines a virtual control interface for controlling the target device and identifies a defined area of the target device on which the virtual control interface is to be provided; and

controlling the HMD to display the virtual control interface as an image superimposed over the defined area of the target device in the field of view.

2. The method of claim 1, wherein the wearable computing device further comprises a camera configured to obtain images from a point-of-view.

3. The method of claim 2, wherein the point-of-view of the camera substantially corresponds to the field of view provided by the HMD.

4. The method of claim 2, wherein determining that a target device is within an environment of a wearable computing device comprises:

receiving a point-of-view image obtained by the camera; and

identifying the target device in the point-of-view image.

5. The method of claim 4, wherein the point-of-view image is a video image.

6. The method of claim 1, wherein determining that a target device is within an environment of a wearable computing device comprises:

receiving a beacon from the target device.

7. The method of claim 1, wherein determining that a target device is within an environment of a wearable computing device comprises:

recognizing a radio frequency identification (RFID) tag on the target device.

8. The method of claim 1, wherein determining that a target device is within an environment of a wearable computing device comprises:

recognizing an optical identification on the target device.

9. The method of claim 8, wherein the optical identification is a barcode or a quick response (QR) code.

10. The method of claim 1, wherein determining that a target device is within an environment of a wearable computing device comprises:

the wearable computing device obtaining environment information regarding the environment of the wearable computing device; and

the wearable computing device identifying the target device based on the environment information.

11. The method of claim 1, wherein determining that a target device is within an environment of a wearable computing device comprises:

the wearable computing device obtaining environment information regarding the environment of the wearable computing device;

the wearable computing device transmitting the environment information to a server network; and

the wearable computing device receiving an identification of the target device from the server network.

12. The method of claim 11, wherein the environment information comprises a physical location of the wearable computing device.

13. The method of claim 1, wherein obtaining target device information related to the target device comprises:

the wearable computing device retrieving the target device information from a target device database.

14. The method of claim 1, wherein obtaining target device information related to the target device comprises:

the wearable computing device receiving the target device information from a server network.

15. The method of claim 1, wherein the defined area of the target device is an external surface of the target device.

16. The method of claim 15, further comprising:

determining a perspective appearance of the defined area in the field of view; and

generating the image of the virtual control interface based on the perspective appearance of the defined area in the field of view.

17. The method of claim 15, further comprising:

determining a distance from the HMD to the defined area of the target device; and

generating the image of the virtual control interface based on the distance.

18. The method of claim 1, further comprising:

recognizing a control instruction for the target device; and

transmitting the control instruction to the target device.

19. The method of claim 18, wherein transmitting the control instruction to the target device comprises the wireless computing device transmitting the control instruction to the target device via a WiFi, Bluetooth, or infrared link.

20. The method of claim 18, wherein transmitting the control instruction to the target device comprises the wireless computing device transmitting the control instruction to the server network for transmission to the target device.

21. The method of claim 18, wherein the wearable computing device comprises a user interface, and wherein recognizing a control instruction for the target device comprises:

receiving input through the user interface.

22. The method of claim 21, wherein the user interface comprises a touchpad on the HMD, and wherein the input comprises a touch interaction with the touchpad.

23. The method of claim 21, wherein the user interface comprises a microphone, and wherein the input comprises a voice command.

24. The method of claim 18, wherein recognizing a control instruction for the target device comprises:

recognizing a gesture corresponding to interaction with the virtual control interface.

25. The method of claim 1, further comprising:

the HMD displaying instructions related to the target device.

26. The method of claim 1, further comprising:

the HMD displaying status information related to the target device.

27. A non-transitory computer readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

determining that a target device is within a field of view provided by a see-through display controlled by the computing device;

obtaining target device information related to the target device, wherein the target device information defines a virtual control interface for controlling the target device and identifies a defined area of the target device on which the virtual control interface is to be provided; and

controlling the see-through display to display the virtual control interface as an image superimposed over the defined area of the target device in the field of view

28. A wearable computing device, comprising:

a head-mounted display (HMD), wherein the HMD is configured to provide a field of view in which at least a portion of an environment of the wearable computing device is viewable, and wherein the HMD is operable to display images superimposed over the field of view; and

a controller, wherein the controller is configured to;

(a) determine that a target device is within the environment of the wearable computing device;

(b) obtain target device information related to the target device, wherein the target device information defines a virtual control interface for controlling the target device and identifies a defined area of the target device on which the virtual control interface is to be provided; and

(c) control the HMD to display the virtual control interface as an image superimposed over the defined area of the target device in the field of view.

29. The wearable computing device of claim 28, further comprising:

a camera configured to image a portion of the environment located in front of the HMD, wherein the controller is configured to identify the target device from one or more images obtained by the camera.

31. The wearable computing device of claim 28, further comprising:

a communication interface, wherein the controller is configured to obtain the target device information by communicating with a server network via the communication interface.

32. The wearable computing device of claim 28, wherein the controller is further configured to:

(d) recognize a control instruction for the target device; and

(e) transmit the control instruction to the target device.

33. The wearable computing device of claim 32, further comprising:

a communication interface, wherein the controller is configured to transmit the control instruction to the target device via the communication interface.

34. The wearable computing device of claim 33, wherein the controller is configured to recognize a gesture corresponding to interaction with the virtual control interface as the control instruction.

35. The wearable computing device of claim 33, further comprising:

a user interface, wherein the controller is configured to recognize input through the user interface as the control instruction.

30. The wearable computing device of 28, further comprising:

a target device database, wherein the controller is configured to obtain the target device information from the database.