Method and Apparatus for Gross-Level User and Input Detection Using Similar or Dissimilar Camera Pair

• US 20170150118A1
• Filed: 11/24/2015
• Published: 05/25/2017
• Est. Priority Date: 11/24/2015
• Status: Active Grant

• Current Assignee: Dell Products LP (Dell Technologies Inc.)
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Aurongzeb, Deeder M., Pacheco, Roman Joel, Lanzoni, Thomas
• Granted Patent: US 10,021,371 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

An information handling system includes a RGB digital camera and a secondary digital camera that can be any type of two-dimensional or three-dimensional digital camera known in the art and a processor. The processor executes code instructions of a gross-level input detection system to detect objects in images taken contemporaneously by the RGB digital camera and the secondary digital camera using object detection techniques, and to calculate the positions of regions of interest within those objects. Further, the processor executes code instructions to detect the orientation of regions of interest within identified objects, and to associate those orientations, changes in orientations, or movement of regions of interest with user commands.

Claims

1. An information handling system comprising:

a first digital camera that is a RGB digital camera to capture a first image;

a second digital camera to contemporaneously capture a second image;

a processor operatively coupled to the first digital camera and the second digital camera;

the processor executing code instructions of a gross-level input detection system to detect at least one object in each of the first image and the second image using object detection techniques and to normalize the first image and the second image in comparison to one another; and

the processor executing code instructions of a gross-level input detection system to further identify a region of interest within the at least one object in each of the first image and the second image and calculate a gross-level three-dimensional position of the region of interest within the at least one object in each of the first image and the second image by comparing differences between normalized pixel coordinates of the region of interest within the at least one object in each of the first image and the second image.

2. The information handling system of claim 1, further comprising:

the first digital camera to capture a third image at a later time;

the second digital camera to capture a fourth image at a later time, contemporaneously with the capture of the third image;

the processor executing code instructions of the gross-level input detection system to detect at least one object in each of the third image and the fourth image using object detection techniques and to normalize the third image and the fourth image in comparison to one another;

the processor executing code instructions of a gross-level input detection system to further identify a region of interest within the at least one object in each of the third image and the fourth image and calculate a gross-level three-dimensional position of the region of interest within the at least one object in each of the third image and the fourth image by comparing differences between normalized pixel coordinates of the region of interest within the object in each of the third image and the fourth image;

the processor executing code instructions of a gross-level input detection system to detect a difference between the gross-level three-dimensional position of the region of interest within the at least one object of each of the third image and the fourth image and the gross-level three-dimensional position of the region of interest within the at least one object in each of the first image and the second image; and

the processor executing code instructions of a gross-level input detection system to associate the difference between the gross-level three-dimensional position of the region of interest within the at least one object in each of the third image and the fourth image and the gross-level three-dimensional position of the region of interest within the at least one object in each of the first image and the second image with at least one user command.

3. The information handling system of claim 1, wherein the second digital camera is also a RGB digital camera.

4. The information handling system of claim 1, wherein the second digital camera is a non-standard digital camera selected from an infrared digital camera, a telephoto lens digital camera, a fish-eye digital camera, a wide-angle digital camera, a close-focus digital camera, a three-dimensional camera including a stereo triangulation camera, a sheet of light triangulation camera, a structured light camera, a time-of-flight camera, an interferometry camera, a coded aperture camera, an array camera, or any other type of non-standard digital camera known in the art.

5. The information handling system of claim 4, wherein the processor to additionally rectify the first image and the second image before calculating the gross-level three-dimensional position of the region of interest within the at least one object in each of the first image and the second image.

6. The information handling system of claim 1, wherein the second digital camera is an infrared camera.

7. The information handling system of claim 4, wherein the processor directs the non-standard digital camera to capture a preliminary image, the processor executing code instructions of a gross-level input detection system to detect at least one object in the preliminary image using sparse coding object detection techniques, and, in response to the processor successfully detecting at least one object in the preliminary image, to direct the RGB digital camera to capture the first image.

8. The information handling system of claim 1, further comprising:

the processor executing code instructions of a gross-level input detection system to identify a plurality of regions of interest within the at least one object in each of the first image and the second image and calculate a gross-level three-dimensional position of each of the plurality of regions of interest within the at least one object in each of the first image and the second image by comparing differences between normalized pixel coordinates of each of the plurality of regions of interest within the at least one object in each of the first image and the second image;

the processor to additionally detect the orientation of each of the plurality of regions of interest within the at least one object in each of the first image and the second image with respect to one another; and

the gross-level input detection system associating the orientation with respect to one another of each of the plurality of regions of interest within the at least one object in each of the first image and the second image with at least one user command.

9. A computer implemented method comprising:

directing at least one RGB digital camera to capture a first image;

directing at least one second digital camera calibrated for use with the at least one RGB digital camera to contemporaneously capture a second image;

detecting at least one object in each of the first image and the second image using object detection techniques;

normalizing the first image and the second image in comparison to one another;

identifying a region of interest within the at least one object in each of the first image and the second image; and

calculating a gross-level three-dimensional position of the region of interest within the at least one object in each of the first image and the second image by comparing differences between normalized pixel coordinates of the region of interest within the at least one object in each of the first image and the second image.

10. The computer implemented method of claim 9, further comprising:

identifying a plurality of regions of interest within the at least one object in each of the first image and the second image;

detecting the orientation of each of the plurality of regions of interest within the at least one object in each of the first image and the second image with respect to one another; and

associating the orientation of each of the plurality of regions of interest within the at least one object in each of the first image and the second image with at least one user command.

11. The computer implemented method of claim 9, further comprising:

directing the at least one RGB camera to capture a third image at a later time than the capture of the first image;

directing the at least one second digital camera calibrated for use with the at least one RGB camera to capture a fourth image contemporaneously with the capture of the third image;

detecting at least one object in each of the third image and the fourth image using object detection techniques;

normalizing the third image and the fourth image in comparison to one another;

identifying a region of interest within the at least one object in each of the third image and the fourth image and calculating a gross-level three-dimensional position of the region of interest within the at least one object in each of the third image and the fourth image by comparing differences between normalized pixel coordinates of the region of interest within the object in each of the third image and the fourth image;

detecting a difference between the gross-level three-dimensional position of the region of interest within the at least one object of each of the third image and the fourth image and the gross-level three-dimensional position of the region of interest within the at least one object in each of the first image and the second image; and

associating the difference between the gross-level three-dimensional position of the region of interest within the at least one object in each of the third image and the fourth image and the gross-level three-dimensional position of the region of interest within the at least one object in each of the first image and the second image with at least one user command.

12. The computer implemented method of claim 9, wherein the second digital camera is also a RGB digital camera.

13. The computer implemented method of claim 9, wherein the second digital camera is a non-standard digital camera selected from an infrared digital camera, a telephoto lens digital camera, a fish-eye digital camera, a wide-angle digital camera, a close-focus digital camera, a three-dimensional camera including a stereo triangulation camera, a sheet of light triangulation camera, a structured light camera, a time-of-flight camera, an interferometry camera, a coded aperture camera, an array camera, or any other type of non-standard digital camera known in the art.

14. The computer implemented method of claim 9, further comprising rectifying the first image and the second image before calculating the gross-level three-dimensional position of the region of interest within the at least one object in each of the first image and the second image.

15. The computer implemented method of claim 9, wherein the second digital camera is an infrared camera.

16. The computer implemented method of claim 13, further comprising:

directing the non-standard camera to capture a preliminary non-standard image before the capture of the first image;

detecting at least one object in the preliminary non-standard image using sparse coding techniques before the capture of the first image; and

directing the RGB digital camera to capture the first image only if the processor successfully detects at least one object in the preliminary non-standard image.

17. An information handling system comprising:

a first digital camera that is a RGB digital camera to capture a first image;

a second digital camera to contemporaneously capture a second image;

a processor operatively coupled to the first digital camera and the second digital camera;

the processor executing code instructions of a gross-level input detection system to detect at least one object in each of the first image and the second image using object detection techniques and to normalize the first image and the second image in comparison to one another;

the processor executing code instructions of a gross-level input detection system to identify a plurality of regions of interest within the at least one object in each of the first image and the second image and calculate a gross-level three-dimensional position of each of the plurality of regions of interest within the at least one object in each of the first image and the second image by comparing differences between normalized pixel coordinates of each of the plurality of regions of interest within the at least one object in each of the first image and the second image;

the processor to additionally detect the orientation of each of the plurality of regions of interest within the at least one object in each of the first image and the second image with respect to one another; and

the gross-level input detection system associating the orientation with respect to one another of each of the plurality of regions of interest within the at least one object in each of the first image and the second image with at least one user command.

18. The information handling system of claim 17, further comprising:

the first digital camera to capture a third image at a later time;

the second digital camera to capture a fourth image at a later time, contemporaneously with the capture of the third image;

the processor executing code instructions of a gross-level input detection system to detect at least one object in each of the third image and the fourth image using object detection techniques and to normalize the third image and the fourth image in comparison to one another;

the processor executing code instructions of a gross-level input detection system to identify a plurality of regions of interest within the at least one object in each of the third image and the fourth image and calculate a gross-level three-dimensional position of each of the plurality of regions of interest within the at least one object in each of the third image and the fourth image by comparing differences between normalized pixel coordinates of each of the plurality of regions of interest within the at least one object in each of the third image and the fourth image;

the processor to additionally detect the orientation of each of the plurality of regions of interest within the at least one object in each of the third image and the fourth image with respect to one another;

the processor executing code instructions of a gross-level input detection system to detect a difference between the orientation of each of the plurality of regions of interest within the at least one object in each of the third image and the fourth image with respect to one another and the orientation of each of the plurality of regions of interest within the at least one object in each of the first image and the second image with respect to one another; and

the processor executing code instructions of a gross-level input detection system to associate the difference between the orientation of each of the plurality of regions of interest within the at least one object in each of the third image and the fourth image with respect to one another and the orientation of each of the plurality of regions of interest within the at least one object in each of the first image and the second image with respect to one another with at least one user command.

19. The information handling system of claim 17, wherein the second digital camera is also a RGB digital camera.

20. The information handling system of claim 17, wherein the second digital camera is a non-standard digital camera selected from an infrared digital camera, a telephoto lens digital camera, a fish-eye digital camera, a wide-angle digital camera, a close-focus digital camera, a three-dimensional camera including a stereo triangulation camera, a sheet of light triangulation camera, a structured light camera, a time-of-flight camera, an interferometry camera, a coded aperture camera, an array camera or any other type of non-standard digital camera known in the art.