Input method designed for augmented reality goggles

• US 9,152,226 B2
• Filed: 06/15/2012
• Issued: 10/06/2015
• Est. Priority Date: 06/15/2012
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Forutanpour, Babak, Balasubramanyam, Shivakumar, Mann, Liberty Edward IV
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): NAWAZ, TALHA M
• Application Number:
  US13/525,102
• Publication Number:
  US 20130335573A1
• Time in Patent Office: 1,208 Days
• Field of Search: 348 41-161
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

Apparatuses, methods, systems and computer-readable media for using proximity inputs on or near a touch screen lens to select objects within a field of view are presented. In some embodiments, a viewing apparatus (e.g. head mounted display, augmented reality goggles) may include at least one lens, wherein the lens can sense touches or near-touches and output data indicative of a location of the proximity input by the user. A processor may receive the data and may select an object within the field of view of the user corresponding to the data, wherein the object and the location of the proximity input on or near the lens by the user are on a common line of sight. In some embodiments, the viewing apparatus may include at least one camera that is configured to record at least one image representative of the user'"'"'s field of view.

Claims

1. A viewing apparatus for a user comprising:

at least one lens through which the user can visually perceive a field of view; and

a processor configured to;

receive first data indicative of a first proximity input within a first region with respect to the at least one lens, wherein the first proximity input is indicative of a location of a first touch by the user on the at least one lens;

perform a first operation in response to the received first data;

receive second data indicative of a second proximity input within a second region with respect to the at least one lens, wherein the second proximity input is indicative of a location of a second touch by the user on the at least one lens;

select an object within the field of view corresponding to the first data, wherein the object and the location of the first touch on the at least one lens are on a common line of sight of the user; and

perform a second operation pertaining to the object in response to the received second data, wherein the second operation is an operation from one or more operations presented in response to the received first data.

2. The viewing apparatus of claim 1, further comprising:

at least one camera configured to record at least one image representative of the field of view; and

the processor, further configured to;

receive the at least one image,

identify a location on the at least one image that is mapped to the location of the first touch on the at least one lens, and

wherein the selection of the object within the field of view further corresponds to the location on the at least one image.

3. The viewing apparatus of claim 2, wherein:

the at least one camera is each further configured to perform a zooming function that magnifies a viewing location within the field of view based at least in part on a gesture performed by the user.

4. The viewing apparatus of claim 2, wherein the at least one image is recorded in response to the reception of the first data by the processor.

5. The viewing apparatus of claim 1, further comprising:

an image projection module configured to project visual information on the at least one lens, wherein the visual information is based on information about the selected object; and

the processor, further configured to;

instruct the image projection module to display the visual information on the at least one lens.

6. The viewing apparatus in claim 5, wherein:

the at least one lens is one of two lenses; and

the processor is further configured to;

detect which lens is touched, and

instruct the image projection module to display the visual information, wherein the visual information is based further on which lens is touched.

7. The viewing apparatus in claim 5, wherein:

the at least one lens is further configured to output the first or second data to the processor, the first or second data being further indicative of an amount of pressure of the first or second touch by the user on the at least one lens; and

the processor is further configured to;

instruct the image projection module to display the visual information, wherein the visual information is based further on the amount of pressure of the first or second touch.

8. The viewing apparatus of claim 1, wherein the one or more operations comprise transmitting information to another device.

9. The viewing apparatus of claim 1, wherein the one or more operations comprise displaying information related to an object or individual proximate to the user.

10. The viewing apparatus of claim 9, further comprising a Satellite Positioning System (SPS) unit, wherein the object or individual proximate to the user is determined based at least in part on input received from the SPS unit.

11. The viewing apparatus of claim 9, further comprising a camera configured to capture an image of at least a portion of the field of view, wherein the object or individual proximate to the user is determined by identifying the object or individual in the captured image.

12. The viewing apparatus of claim 1, wherein the at least one lens is touch sensitive and is configured to output the first or second data when a user touches the at least one lens.

13. The viewing apparatus of claim 1, wherein the first or second proximity input comprises an object hovering near the at least one lens.

14. The viewing apparatus of claim 13, further comprising at least one camera configured to capture an image of the object, wherein the first data received by the processor comprises information based on the image, and wherein the processor is further configured to identify when an input has occurred based on the received first data.

15. The viewing apparatus of claim 14, wherein the at least one camera is further configured to use infrared (IR) data to detect position or depth of the object.

16. The viewing apparatus of claim 14, wherein the at least one camera is two cameras, configured to provide a stereo image of the object.

17. The viewing apparatus of claim 14, wherein the at least one camera is further configured to capture an image of at least a portion of the field of view.

18. The viewing apparatus of claim 1, wherein the first touch and second touch by the user occur in a sequence in time.

19. A method for use with a viewing apparatus, comprising:

receiving first data indicative of a first proximity input performed within a first region with respect to at least one lens of the viewing apparatus, a field of view being observable through the at least one lens by a user of the viewing apparatus, wherein the first proximity input includes a location of a first touch by the user on the at least one lens;

performing a first operation in response to the received first data;

receiving second data indicative of a second proximity input performed within a second region with respect to the at least one lens of the viewing apparatus, wherein the second proximity input includes a location of a second touch by the user on the at least one lens;

selecting an object within the field of view corresponding to the first data, wherein the object and the location of the first touch on the at least one lens are on a common line of sight of the user; and

performing a second operation pertaining to the object in response to the received second data, wherein the second operation is an operation from one or more operations presented in response to the received first data.

20. The method of claim 19, wherein the first touch further causes:

at least one camera to record at least one image representative of the field of view; and

the processor to;

receive the at least one image,

identify a location on the at least one image that is mapped to the location of the first touch on the at least one lens, and

wherein the selection of the object within the field of view further corresponds to the location on the at least one image.

21. The method of claim 20, wherein:

the at least one camera is each further configured to perform a zooming function that magnifies a viewing location within the field of view based at least in part on a gesture performed by the user.

22. The method of claim 20, wherein the at least one image is recorded in response to the reception of the first data.

23. The method of claim 19, wherein the first touch further causes:

an image projection module to project visual information on the at least one lens, wherein the visual information is based on information about the selected object; and

the processor to instruct the image projection module to display the visual information on the at least one lens.

24. The method of claim 23, wherein:

the at least one lens comprises two lenses; and

the first or second touch by the user on the at least one lens comprises a first or second touch by the user on one of the two lenses, wherein the method further comprises;

detecting which lens is touched, and

instructing the image projection module to display the visual information, wherein the visual information is based further on which lens is touched.

25. The method of claim 23, wherein the first or second touch further causes:

the at least one lens to output the first or second data, the first or second data being indicative of an amount of pressure of the touch by the user on the at least one lens; and

the processor to instruct the image projection module to display the visual information, wherein the visual information is based further on the amount of pressure of the first or second touch.

26. The method of claim 19, wherein the one or more operations comprise transmitting information to another device.

27. The method of claim 19, wherein the one or more operations comprise displaying information related to an object or individual proximate to the user.

28. The method of claim 27, further comprising determining the object or individual proximate to the user based at least in part on input received from a Satellite Positioning System (SPS) unit.

29. The method of claim 27, further comprising capturing an image of at least a portion of the field of view, wherein the object or individual proximate to the user is determined by identifying the object or individual in the captured image.

30. The method of claim 19, wherein the first or second proximity input comprises an object hovering near the at least one lens.

31. The method of claim 30, further comprising at least one camera detecting the object hovering near the at least one lens.

32. The method of claim 30, further comprising determining that an object moving within a threshold distance to the at least one lens comprises the first or second proximity input.

33. The method of claim 19, wherein the first or second proximity input comprises both at least one touch on the at least one lens and an object hovering near the at least one lens.

34. The method of claim 19, wherein the first touch and second touch by the user occur in a sequence in time.

35. A viewing apparatus for a user comprising:

at least one viewing means through which the user can visually perceive a field of view; and

means for receiving first data indicative of a first proximity input within a first region with respect to the at least one lens, wherein the first proximity input is indicative of a location of a touch by the user on the at least one lens;

means for performing a first operation in response to the received first data;

means for receiving second data indicative of a second proximity input within a second region with respect to the at least one lens, wherein the second proximity input is indicative of a second location of a touch by the user on the at least one lens;

means for selecting an object within the field of view corresponding to the first data, wherein the object and the location of the touch on the at least one lens are on a common line of sight of the user; and

means for performing a second operation pertaining to the object in response to the received second data, wherein the second operation is an operation from one or more operations presented in response to the received first data.

36. The viewing apparatus of claim 35, further comprising:

recording means to record at least one image representative of the field of view;

means for receiving the at least one image;

means for identifying a location on the at least one image that is mapped to the location of the first touch on the at least one viewing means, and

wherein the means for selecting the object within the field of view further corresponds to the location of the first touch on the at least one viewing means.

37. A computer program product residing on a non-transitory processor-readable medium and comprising processor-readable instructions configured to cause a processor to:

receive first data indicative of a first proximity input within a first region with respect to at least one lens through which a user can visually perceive a field of view, wherein the first proximity input is indicative of a first location of a touch by the user on the at least one lens;

perform a first operation in response to the received first data;

receive second data indicative of a second proximity input within a second region with respect to the at least one lens, wherein the second proximity input is indicative of a second location of a touch by the user on the at least one lens

select an object within the field of view corresponding to the first data, wherein the object and the location of the touch on the at least one lens are on a common line of sight of the user; and

perform a second operation pertaining to the object in response to the received second data, wherein the second operation is an operation from one or more operations presented in response to the received first data.

38. The computer program product of claim 37, wherein the processor-readable instructions further cause the processor to:

receive at least one image from at least one camera configured to record the at least one image representative of the field of view; and

,

identify a location on the at least one image that is mapped to the location of the first touch on the at least one lens, and

wherein the selection of the object within the field of view further corresponds to the location on the at least one image.

39. The computer program product of claim 37, wherein the processor-readable instructions further cause the processor to:

instruct an image projection module to display visual information on the at least one lens;

wherein the image projection module is configured to project the visual information on the at least one lens; and

wherein the visual information is based on information about the selected object.