Enhanced field of view to augment three-dimensional (3D) sensory space for free-space gesture interpretation

• US 9,632,572 B2
• Filed: 10/03/2014
• Issued: 04/25/2017
• Est. Priority Date: 10/03/2013
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: Holz, David S, Durdik, Paul
• Primary Examiner(s): Mehmood, Jennifer
• Assistant Examiner(s): NEUPANE, KRISHNA P.
• Application Number:
  US14/506,596
• Publication Number:
  US 20150097768A1
• Time in Patent Office: 935 Days
• Field of Search: None
• CPC Class Codes: G02B 5/04 Prisms
• CPC Class Codes: G02B 5/045 Prism arrays
• CPC Class Codes: G02B 7/1805 for prisms G02B7/181 takes precedence
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 2200/1633 Protecting arrangement for the entire housing of the computer
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

The technology disclosed relates to enhancing the fields of view of one or more cameras of a gesture recognition system for augmenting the three-dimensional (3D) sensory space of the gesture recognition system. The augmented 3D sensory space allows for inclusion of previously uncaptured of regions and points for which gestures can be interpreted i.e. blind spots of the cameras of the gesture recognition system. Some examples of such blind spots include areas underneath the cameras and/or within 20-85 degrees of a tangential axis of the cameras. In particular, the technology disclosed uses a Fresnel prismatic element and/or a triangular prism element to redirect the optical axis of the cameras, giving the cameras fields of view that cover at least 45 to 80 degrees from tangential to the vertical axis of a display screen on which the cameras are mounted.

Claims

1. A rim mounted space imaging apparatus, mounted in a rim of a display that has a vertical axis, comprising:

two cameras mounted in the rim of a display with optical axes facing within 20 degrees of tangential to a vertical axis of the display;

at least one Fresnel prismatic element that redirects the optical axes of the cameras, giving each camera a field of view that covers at least 45 to 80 degrees from tangential to the vertical axis of the display; and

a camera controller coupled to the two cameras that compensates for redirection by the Fresnel prismatic element and determines a position of at least one control object within the camera fields of view.

2. The apparatus of claim 1, further comprising:

a bender that bends or reorients the Fresnel prismatic element of at least one of the cameras to modify the relative optical axes of the two cameras mounted in the rim of the display.

3. The apparatus of claim 1, further comprising:

at least one illumination source; and

the Fresnel prismatic element that redirects the illumination source to cover an effective area of the camera fields of view.

4. The apparatus of claim 1, further comprising:

the Fresnel prismatic element that redirects the optical axes of the cameras, giving each camera a field of view that covers at least 20 to 85 degrees from tangential to the vertical axis of the display.

5. The apparatus of claim 1, further comprising:

at least one camera mounted in the rim of the display with an optical axis facing within 20 degrees of tangential to a vertical axis of the display, wherein the camera is different from the two cameras of claim 1; and

the Fresnel prismatic element does not redirect the optical axis of the camera.

6. The apparatus of claim 1, wherein the display is at least one of planar or cylindrical.

7. The apparatus of claim 1, wherein the Fresnel prismatic element is an optical film.

8. A rim mounted space imaging apparatus, mounted in a rim of a display that has a vertical axis, comprising:

two cameras mounted in the rim of a display with optical axes facing within 20 degrees of tangential to a vertical axis of the display;

at least one triangular solid prism that redirects the optical axes of the cameras, giving each camera a field of view that covers at least 45 to 80 degrees from tangential to the vertical axis of the display; and

a camera controller coupled to the two cameras that compensates for redirection by the triangular solid prism and determines a position of at least one control object within the camera fields of view.

9. The apparatus of claim 8, further comprising:

a bender that bends or reorients the triangular solid prism of at least one of the cameras to modify the relative optical axes of the two cameras mounted in the rim of the display.

10. The apparatus of claim 8, further comprising:

at least one illumination source; and

the triangular solid prism that redirects the illumination source to cover an effective area of the camera fields of view.

11. The apparatus of claim 8, further comprising:

the triangular solid prism that redirects the optical axes of the cameras, giving each camera a field of view that covers at least 20 to 85 degrees from tangential to the vertical axis of the display.

12. The apparatus of claim 8, further comprising:

at least one camera mounted in the rim of the display with an optical axis facing within 20 degrees of tangential to a vertical axis of the display, wherein the camera is different from the two cameras of claim 8; and

the triangular solid prism does not redirect the optical axis of the camera.

13. The apparatus of claim 8, wherein the display is at least one of planar or cylindrical.

14. The apparatus of claim 8, wherein the triangular solid prism is an optical film.

20. The portable electronic device of claim 15, further comprising:

at least one other camera mounted in the bezel with an optical axis facing within 20 degrees of tangential to a vertical axis of the display screen, wherein the other camera is different from the camera of claim 14; and

the Fresnel prismatic element does not redirect the optical axis of the other camera.

15. A portable electronic device with a bezel for a display screen, comprising:

a bezel for a display screen that has a vertical axis;

at least one camera mounted in the bezel with optical axis facing within 20 degrees of tangential to the vertical axis of the display screen;

a Fresnel prismatic element that redirects the optical axis of the camera, giving the camera a field of view that covers at least 45 to 80 degrees from tangential to the vertical axis of the display screen; and

a camera controller coupled to the camera that compensates for redirection by the Fresnel prismatic element and determines a position of at least one control object within the camera field of view.

16. The apparatus of claim 15, wherein the Fresnel prismatic element is an optical film.

17. The portable electronic device of claim 15, further comprising:

a bender that bends or reorients the Fresnel prismatic element of the camera to modify the optical axis of the camera.

18. The portable electronic device of claim 15, further comprising:

at least one illumination source; and

the Fresnel prismatic element that redirects the illumination source to cover an effective area of the camera field of view.

19. The portable electronic device of claim 15, further comprising:

the Fresnel prismatic element that redirects the optical axis of the camera, giving the camera a field of view that covers at least 20 to 85 degrees from tangential to the vertical axis of the display screen.