USER GESTURE RECOGNITION

• US 20170090586A1
• Filed: 03/05/2015
• Published: 03/30/2017
• Est. Priority Date: 03/21/2014
• Status: Active Grant

• Current Assignee: iNUI Studio SA
• Original Assignee: Artnolens SA
• Inventors: Raulot, Olivier
• Granted Patent: US 10,310,619 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0485 Scrolling or panning

Abstract

A method and device for gesture recognition, wherein the gesture is executed by a user in a gesture region which may be defined relative to a display surface. In an embodiment, the gesture comprises a select gesture and the device comprises at least three cameras operating in the visual range where a first camera is used to determine a horizontal location of the select gesture and the other cameras are used to determine a vertical location thereof. A device for providing input to a computing device comprises a rectangular display having a viewing surface and at least three cameras having respective fields of view. A first camera and a second camera are located at respective adjacent corners of the display and a third camera is located at an edge of the display between the first and second cameras.

Claims

1. A method of providing input to a computing device, the computing device comprising a processor, memory, a display having a viewing surface and at least three cameras having respective fields of view, the method comprising the steps of:

capturing corresponding images with the cameras;

identifying input gestures using the captured images; and

interpreting the identified gestures as input to the computing device;

wherein

at least one of the cameras is orientated such that the respective field of view lies adjacent to the viewing surface of the display; and

wherein

identifying input gestures in the captured images comprises first choosing between images captured by either a first camera or a second camera and then using the chosen image to identify input gestures,

wherein identifying input gestures in the captured images is further based on images captured by a third camera,

wherein choosing between images captured by either the first camera or the second camera comprises comparing information between images captured by the first camera and images captured by the second camera, and

wherein comparing information comprises comparing a standard deviation of information contained in an image captured by the first camera to a standard deviation of information contained in an image captured by the second camera.

2. The method according to claim 1 wherein a determination of a first dimension for use in identifying input gestures is based on images captured by the third camera.

3. The method according to claim 2 wherein an input gesture is recognized by determining a change in images captured by the first camera at different times and calculating a threshold based on the change.

4. The method according to claim 2 where a determination of a second dimension for use in identifying input gestures is based on images captured by the first camera or the second camera.

5. The method according to claim 4 wherein an input gesture is performed by a user'"'"'s appendage and wherein the step of determining the second dimension of the input gesture comprises recognizing the user'"'"'s appendage.

6. The method according to claim 5 wherein the user'"'"'s appendage is recognized according to a predetermined range of hue, saturation and value of pixels in images captured by the second camera.

7. The method according to claim 4, further comprising establishing a mapping between a co-ordinate system of the first camera and/or the second camera and a co-ordinate system of the viewing surface.

8. The method according to claim 7 wherein the mapping comprises a linear mapping.

9. The method according to claim 8 wherein the mapping comprises a first linear mapping applied to a first section of the co-ordinates of the viewing surface and a second linear mapping applied to a second section of the viewing surface.

10. The method according to claim 1, further comprising processing each image by designating a region of the image as a gesture region and determining changes in the gesture region.

11. The method according to claim 10, further comprising defining respective gesture regions adjacent to the viewing surface of the display.

12. The method according to claim 10, further comprising defining gesture regions spaced from the viewing surface of the display.

13. The method according to claim 1, further comprising tracking an input gesture and determining a slide action in dependence on detected motion of the input gesture.

14. The method according to claim 1 wherein all of the cameras are situated near an edge of the display and orientated with intersecting fields of view.

15. The method according to claim 1 wherein the computing device further comprises a fourth camera which is positioned adjacent the third camera and wherein a determination of a first dimension comprises choosing between images captured by the third camera and images captured by the fourth camera.

16. A device for providing input to a computing device, the computing device comprising a processor and memory, the device comprising a rectangular display having a viewing surface and at least three cameras having respective fields of view,

wherein a first camera and a second camera are located at respective adjacent corners of the display and a third camera is located at an edge of the display between the first and second cameras.

17. The device according to claim 16, further comprising a fourth camera located at an edge of the display between the first and second cameras.

18. The device according to claim 17, further comprising a mounting frame located at a periphery of the display, wherein at least said first and second camera are mounted to the mounting frame.

19. The device according to claim 18 wherein each camera is mounted to the mounting frame.

20. The device according to claim 17 wherein at least one of the cameras is orientated such that the respective field of view lies adjacent to the viewing surface of the display.

21. The device according to claim 17 wherein the input comprises an input gesture and wherein the memory stores instructions which, when processed by the processor, identify input gestures using the captured images, wherein

identifying input gestures in the captured images comprises choosing between images captured by either the first camera or the second camera and further using the chosen image to identify input gestures.

22. The device according to claim 21 wherein identifying input gestures in the captured images is further based on images captured by the third camera.

23. The device according to claim 21 wherein choosing between images captured by either the first camera or the second camera comprises comparing information between images captured by the first camera and images captured by the second camera.

24. The device according to claim 23 wherein the input gesture is performed by a user'"'"'s appendage and wherein the instructions further comprise instructions for recognizing the user'"'"'s appendage.

25. The device according to claim 24 wherein the user'"'"'s appendage is recognized according to a predetermined range of hue, saturation and value of pixels in images captured by one or more of the cameras.

26. The device according to claim 21 wherein the instructions, when processed by the processor, determine a first dimension for use in identifying input gestures based on images captured by the third camera and determine a second dimension for use in identifying input gestures based on images captured by the first camera or the second camera, wherein the input gesture is performed by a user'"'"'s appendage, wherein the step of determining the second dimension of the input gesture comprises recognizing the user'"'"'s appendage, and wherein the user'"'"'s appendage is recognized according to a predetermined range of hue, saturation and value of pixels in images captured by the second camera.

27. The device according to claim 17 wherein at least one of the cameras is orientated so that the respective fields of view lie adjacent to the viewing surface of the display.

28. The device according to claim 17 wherein all of the cameras are situated near an edge of the display and orientated with intersecting fields of view.

29. The device according to claim 21 wherein the instructions, when processed by the processor, process each image by designating a region of the image as a gesture region and determining changes in the gesture region.

30. The device according to claim 29 wherein the instructions, when processed by the processor, further define respective gesture regions adjacent to the viewing surface of the display.

31. The device according to claim 29 wherein the instructions, when processed by the processor, further define respective gesture regions spaced from the viewing surface of the display.