Determining three dimensional information using a single camera

• US 9,912,934 B2
• Filed: 05/29/2015
• Issued: 03/06/2018
• Est. Priority Date: 07/21/2014
• Status: Active Grant

• Current Assignee: Moaath Alrajab
• Original Assignee: Moaath Alrajab
• Inventors: Alrajab, Moaath
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): Fereja, Samuel D
• Application Number:
  US14/726,169
• Publication Number:
  US 20160021357A1
• Time in Patent Office: 1,012 Days
• Field of Search: None
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

Two dimensional images captured by a camera or other device may be used to generate three dimensional information for target objects included in the two dimensional images. Sensor information and other information associated with the device capturing the two dimensional images may be obtained and used to determine a displacement or movement of the camera during capture of the two dimensional images. The displacement or movement may be used to calculate a distance of the target object in the two dimensional images. The distance information may be used to generate virtual planes corresponding to the target objects.

Claims

1. A computer-implemented method for comprising:

capturing, by a computer system of the one or more computer systems, a first image of a scene including at least one target object at a first interval of time;

capturing, by the computer system, a second image of the scene including the at least one target object at a second interval of time after the first interval of time and distinct from the first interval of time;

determining a displacement of the computer system between the first interval of time and the second interval of time based at least in part on a sensor included in the computer system;

calculating three dimensional information corresponding to the target object based at least in part on the first image, the second image, and the determined displacement of the computer system between the first interval of time and the second interval of time by at least generating a set of virtual planes associated with the target object, at least one virtual plane of the set of virtual planes associated with the three dimensional information; and

generating a third image including the target object from the scene based at least in part on the calculated three dimensional information, where the third image includes the target object without background information from the scene included in the first image and the second image.

2. The computer-implemented method of claim 1, wherein generating the third image further comprises:

determining a virtual layer associated with the target object; and

determining a set of pixels of the first image and the second image corresponding to the virtual layer, the set of pixels to be included in the third image.

3. The computer-implemented method of claim 1, wherein determining the displacement on the computer system further comprises obtaining information from at least one additional sensor of the computer system.

4. The computer-implemented method of claim 1, wherein the calculated three dimensional information further comprises a determined distance of the target object from the computer system.

5. The computer-implemented method of claim 1, wherein the calculated three dimensional information further comprises disparity mapping between the first image and the second image.

6. A system, comprising:

at least one computing device configured to implement one or more services, wherein the one or more services are configured to;

capture two or more images of a scene including at least one target object, where at least two images of the two or more images are captured at different intervals of time;

determine an amount of movement of a device responsible for capturing the two or more images between the different intervals of time based at least in part on information obtained from one or more sensors of the device;

generate a set of virtual planes associated with the at least one target object, at least one virtual plane of the set of virtual planes including distance information of a corresponding target object included in the scene, the distance information determined based at least in part on the determined amount of movement and the two or more images; and

generate a set of images based at least in part on the set of virtual planes, at least one image of the set of images contains a single target object of the at least one target object.

7. The system of claim 6, wherein the one or more sensors of the device includes at least one of an accelerometer, a GPS sensor, a gyroscope, a camera sensor, a magnetometer, and a proximity sensor.

8. The system of claim 6, wherein the at least one computing device is further configured to determine at least one virtual plane of the set of virtual planes based at least in part on a disparity mapping corresponding to the two or more images.

9. The system of claim 6, wherein the device responsible for capturing the two or more images is a component of the system.

10. The system of claim 6, wherein the at least one computing device is further configured to receive an input from a user indicating a particular target object of the at least one target object to generate an image for, where the generated image contains a set of pixels corresponding to the particular target object from the two or more images.

11. The system of claim 6, wherein the at least one computing device is further configured to detect the at least one target object in the two or more images prior to generating the set of virtual planes.

12. The system of claim 6, wherein generating the set of virtual planes further comprises determining distance information of the corresponding target object included in the scene, such that the distance information indicates that the determined amount of movement of the device is sufficient to calculate a distance between the corresponding target object and the device.

13. The system of claim 6, wherein one or more services are further configured to:

determine a virtual plane of the set of virtual planes associated with the target object; and

determine a set of pixels of the two or more images corresponding to the virtual plane, the set of pixels to be included in the at least one image of the set of images.

14. A non-transitory computer-readable storage medium having code stored thereon that, when executed by a computer system, causes the computer system to perform operations including:

capture, by the computer system, a set of images including a target object, where at least a first image of the set of images is captured at a first interval of time and at least a second image of the set of images is captured at a second interval of time;

determine a displacement of the computer system between the first interval of time and the second interval of time;

generate a set of virtual planes corresponding to the target object based at least in part on the determined displacement of the computer system and the first image and the second image; and

generate a set of images based at least in part on the set of virtual planes and at least one image of the set of images contains the target object.

15. The non-transitory computer-readable storage medium of claim 14, wherein the instructions further comprise instructions that, when executed by the one or more processors, cause the computer system to:

generate a disparity mapping based at least in part on a first set of pixels from the first image and a second set of pixels from the second image; and

wherein generating the virtual plane is based at least in part on the generated disparity mapping.

16. The non-transitory computer-readable storage medium of claim 14, wherein the instructions that cause the computer system to capture the set of images further include instructions that cause the computer system to capture the set of images as a result of a single user input.

17. The non-transitory computer-readable storage medium of claim 14, wherein the instructions that cause the computer system to determine the displacement of the computer system further include instructions that cause the computer system to obtain information from one or more sensors of the computer system suitable for determining displacement.

18. The non-transitory computer-readable storage medium of claim 17, wherein the instructions that cause the computer system to obtain information from the one or more sensors of the computer system further include instructions that cause the computer system to obtain the information from at least one of an accelerometer, a GPS sensor, a gyroscope, a camera sensor, a magnetometer, and a proximity sensor.

19. The non-transitory computer-readable storage medium of claim 14, wherein the instructions further comprise instructions that, when executed by the one or more processors, cause the system to determine a set of pixels included in the first image and the second image and corresponding to the target object.

20. The non-transitory computer-readable storage medium of claim 19, wherein the instructions further comprise instructions that, when executed by the one or more processors, cause the system to generate an image of the target object based at least in part on the set of pixels.