Methods and devices for augmenting a field of view

• US 8,462,010 B2
• Filed: 05/01/2012
• Issued: 06/11/2013
• Est. Priority Date: 03/14/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Chi, Liang-Yu (Tom), Ryskamp, Robert Allen, Gomez, Luis Ricardo Prada
• Primary Examiner(s): Wu, Daniel
• Assistant Examiner(s): Terrell, Emily C
• Application Number:
  US13/461,492
• Publication Number:
  US 20120235827A1
• Time in Patent Office: 406 Days
• Field of Search: 340/815.45, 340/815.4, 340/815, 340/944, 345/7, 382/103, 359/196.1, 359/237, 359/362, 359/443, 359/462, 362/800
• US Class Current: 340/815.45
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02C 11/10 Electronic devices other than hearing aids
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09F 9/33 being semiconductor devices, e.g. diodes
• CPC Class Codes: Y10S 362/80 Light emitting diode

Abstract

The present application discloses methods and systems for augmenting a field of view of a user. In one embodiment, a device may include a detector, a processor, and an array of visual indicators. According to an embodiment of the method, the detector may image an environment. The processor may identify within the environment imaged by the detector an object of interest. The processor may also determine a relative spatial location of the identified object of interest. Further, the array of visual indicators may be used to indicate the determined relative spatial location of the identified object of interest. In particular, a set of one or more visual indicators may be selected that has a relative spatial location in the array of visual indicators that is indicative of the relative spatial location of the identified object of interest.

Claims

1. A head-mountable device, comprising:

a frame comprising an external side and an internal side, wherein the internal side substantially faces a front of at least one eye when the head-mountable device is worn;

an array of visual indicators mounted along at least a portion of the internal side of the frame, wherein the array of visual indicators is arranged on the frame such that when the head-mountable device is worn, the visual indicators emit light towards the at least one eye; and

a computing system configured to;

receive image data of a first field of view of an environment, wherein the first field of view corresponds to a field of view of an image-capture device mounted on the head-mountable device;

detect an object within the image data of the first field of view of the environment, wherein the object is outside of a second field of view, wherein the second field of view corresponds to a field of view of a wearer of the head-mountable device when the head-mountable device is worn;

determine a relative spatial location of the object with respect to the second field of view; and

cause the array of visual indicators to indicate the relative spatial location of the object.

2. The head-mountable device of claim 1, wherein causing the array of visual indicators to indicate the relative spatial location of the object comprises at least one of illuminating at least one visual indicator in the array of visual indicators and changing the color of at least one visual indicator in the array of visual indicators.

3. The head-mountable device of claim 2, wherein the at least one visual indicator in the array of visual indicators comprises at least one visual indicator in the array of visual indicators having a relative spatial location in the array that is indicative of the relative spatial location of the object.

4. The head-mountable device of claim 1, wherein the array of visual indicators comprises at least one of a horizontal linear array of light emitting diodes (LEDs), a vertical linear array of LEDs, and a two-dimensional array of LEDs.

5. The head-mountable device of claim 1, wherein the device is configured to be worn over at least one eye of the wearer.

6. The head-mountable device of claim 1, wherein the relative spatial location of the object is relative to the wearer.

7. The head-mountable device of claim 1, wherein the head-mountable device is configured to receive an indication of the object and an indication of the relative spatial location of the object.

8. The head-mountable device of claim 1, wherein the first field of view is larger than the second field of view.

9. The head-mountable device of claim 1, further comprising an image-capture device configured to:

capture the image data of the first field of view of the environment; and

transmit the image data to the computing system.

10. The head-mountable device of claim 9, wherein the relative spatial location of the object is relative to the image-capture device.

11. The head-mountable device of claim 1, further comprising an orientation sensor configured to detect movement of the device.

12. The head-mountable device of claim 11, wherein the array of visual indicators is configured to indicate an adjusted relative spatial location of the object based on movement of the device detected by the orientation sensor.

13. A method, comprising:

receiving image data of a first field of view of an environment, wherein the first field of view corresponds to a field of view of an image-capture device mounted on a head-mountable device, wherein the head mountable device comprises a frame;

detecting an object within the image data of the first field of view of the environment, wherein the object is outside of a second field of view, wherein the second field of view corresponds to a field of view of a wearer of the head-mountable device when the head-mountable device is worn;

determining a relative spatial location of the object with respect to the second field of view; and

causing an array of visual indicators to indicate the relative spatial location of the object, wherein the array of visual indicators is mounted along at least a portion of an internal side of the frame, wherein the internal side substantially faces a front of at least one eye when the head-mountable device is worn, and wherein the array of visual indicators is arranged on the frame such that when the head-mountable device is worn, the visual indictors emit light towards the at least one eye.

14. The method of claim 13, wherein the relative spatial location of the object is relative to the wearer.

15. The method of claim 13, wherein causing the array of visual indicators to indicate the relative spatial location of the object comprises at least one of illuminating at least one visual indicator in the array of visual indicators and changing the color of at least one visual indicator in the array of visual indicators.

16. The method of claim 15, wherein the at least one visual indicator in the array of visual indicators comprises at least one visual indicator in the array of visual indicators having a relative spatial location in the array that is indicative of the relative spatial location of the object.

17. The method of claim 13, further comprising receiving an indication of the object and an indication of the relative spatial location of the object.

18. The method of claim 13, wherein the relative spatial location of the object comprises a first relative spatial location of the object, the method further comprising using the array of visual indicators to indicate a second relative spatial location of the object, wherein the object is outside the second field of view.

19. The method of claim 18, wherein using the array of visual indicators to indicate the second relative spatial location of the object comprises using the array of visual indicators to indicate the second relative spatial location of the object in response to detecting that the wearer has moved.

20. The method of claim 18, wherein using the array of visual indicators to indicate the second relative spatial location of the object comprises using the array of visual indicators to indicate the second relative spatial location of the object in response to detecting that the object has moved.

21. The method of claim 13, wherein detecting the object within the image data of the first field of view of the environment comprises:

identifying at least one object in the image data of the first field of view of the environment;

comparing the at least one object with a plurality of objects of interest stored in a database; and

based on the comparison, selecting the object from the plurality of objects of interest.

22. The method of claim 13, wherein causing the array of visual indicators to indicate the relative spatial location of the detected object comprises transmitting to the head-mountable device an indication of the detected object and the determined relative spatial location of the detected object.