Method and system for automatic 3-D image creation

• US 8,259,161 B1
• Filed: 02/06/2012
• Issued: 09/04/2012
• Est. Priority Date: 02/06/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Huang, Jonathan, Kvaalen, Samuel, Bradshaw, Peter
• Primary Examiner(s): SENFI, BEHROOZ M
• Application Number:
  US13/366,709
• Time in Patent Office: 211 Days
• Field of Search: 348/222.1, 348/51, 348/42, 348/333.12, 348/580, 348/571, 348/262, 348/207.99, 348/50, 382/103, 382/106, 382/154, 382/100, 382276-308
• US Class Current: 348/50
• CPC Class Codes: G03B 35/02 by sequential recording
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 2213/003 Aspects relating to the "2D+depth" image format

Abstract

Disclosed herein are methods, devices, and non-transitory computer readable media that relate to stereoscopic image creation. A camera captures an initial image at an initial position. A target displacement from the initial position is determined for a desired stereoscopic effect, and an instruction is provided that specifies a direction in which to move the camera from the initial position. While the camera is in motion, an estimated displacement from the initial position is calculated. When the estimated displacement corresponds to the target displacement, the camera automatically captures a candidate image. An acceptability analysis is performed to determine whether the candidate image has acceptable image quality and acceptable similarity to the initial image. If the candidate image passes the acceptability analysis, a stereoscopic image is created based on the initial and candidate images.

Claims

1. A method for stereoscopic image creation, the method comprising:

capturing an initial image using a camera at an initial position;

determining a target displacement from the initial position for a desired stereoscopic effect;

providing an instruction that specifies a direction in which to move the camera from the initial position;

while the camera is in motion, estimating an estimated displacement from the initial position;

in response to determining that the estimated displacement corresponds to the target displacement, the camera automatically capturing a candidate image;

in response to capturing the candidate image, automatically performing an acceptability analysis, wherein the acceptability analysis comprises analyzing the candidate image to determine whether the candidate image has acceptable image quality and comparing the candidate image to the initial image to determine whether the candidate image has acceptable similarity to the initial image; and

in response to determining that the candidate image has acceptable image quality and has acceptability similarity to the initial image, automatically creating a stereoscopic image based on the initial image and the candidate image.

2. The method of claim 1, further comprising:

storing the stereoscopic image as an image file.

3. The method of claim 1, further comprising:

displaying the stereoscopic image on a mobile phone.

4. The method of claim 1, further comprising:

in response to determining that the candidate image lacks acceptable similarity to the initial image, providing a further instruction.

5. The method of claim 4, wherein the further instruction comprises an instruction to move the camera back toward the initial position.

6. The method of claim 1, wherein comparing the candidate image to the initial image to determine whether the candidate image has acceptable similarity to the initial image comprises:

creating a predicted image based on the initial image and the target displacement; and

comparing the candidate image to the predicted image.

7. The method of claim 1, wherein estimating the estimated displacement from the initial position comprises:

obtaining at least one sensor reading from at least one motion sensor, wherein the at least one motion sensor comprises a gyroscope.

8. The method of claim 1, wherein estimating the estimated displacement from the initial position comprises:

obtaining at least one sensor reading from at least one motion sensor, wherein the at least one motion sensor comprises an accelerometer.

9. The method of claim 1, wherein estimating the estimated displacement from the initial position comprises:

the camera capturing multiple images while the camera is in motion; and

calculating motion vectors between individual images in the multiple images.

10. The method of claim 1, wherein estimating the estimated displacement from the initial position comprises:

obtaining a GPS position.

11. The method of claim 1, wherein estimating the estimated displacement from the initial position comprises:

obtaining a position using wireless triangulation.

12. The method of claim 1, wherein the instruction specifies a distance from the initial position in which to move the camera.

13. The method of claim 1, further comprising:

analyzing the initial image to determine whether the initial image has acceptable image quality.

14. A device, comprising:

an image sensor;

a user interface; and

a controller, wherein the controller is configured to;

capture an initial image using the image sensor when the device is at an initial position;

determine a target displacement from the initial position for a desired stereoscopic effect;

provide an instruction through the user interface, wherein the instruction specifies a direction in which to move the device from the initial position;

while the device is in motion, estimate a displacement of the device from the initial position;

in response to determining that the estimated displacement corresponds to the target displacement, automatically capture a candidate image using the image sensor;

in response to capturing the candidate image, automatically perform an acceptability analysis, wherein the acceptability analysis comprises analyzing the candidate image to determine whether the candidate image has acceptable image quality and comparing the candidate image to the initial image to determine whether the candidate image has acceptable similarity to the initial image; and

in response to determining that the candidate image has acceptable image quality and has acceptability similarity to the initial image, automatically create a stereoscopic image based on the initial image and the candidate image.

15. The device of claim 14, further comprising a global positioning system (GPS) transceiver and wherein the controller estimates the displacement of the device based at least on GPS data.

16. The device of claim 14, further comprising a motion sensor and wherein the controller estimates the displacement of the device based at least on motion sensor data.

17. The device of claim 14, further comprising a range finder and wherein the controller estimates the displacement of the device based at least on range finder data.

18. The device of claim 14, further comprising a memory, wherein the controller is further configured to store the stereoscopic image in the memory as an image file.

19. The device of claim 14, further comprising a display, wherein the controller is further configured to display the stereoscopic image on the display.

20. The device of claim 14, wherein the controller is further configured to, in response to determining that the candidate image lacks acceptable image quality, provide a further instruction through the user interface.

21. The device of claim 14, wherein the controller is further configured to, in response to determining that the candidate image lacks acceptable similarity to the initial image, provide a further instruction through the user interface.

22. The device of claim 21, wherein the further instruction comprises an instruction to move the device back toward the initial position.

23. The device of claim 14, wherein comparing the candidate image to the initial image to determine whether the candidate image has acceptable similarity to the initial image comprises:

creating a predicted image based on the initial image and the target displacement; and

comparing the candidate image to the predicted image.

24. The device of claim 14, wherein the instruction specifies a distance from the initial position in which to move the device.

25. A non-transitory computer readable medium having stored therein instructions executable by a computing device, the computing device including an image sensor and a user interface, to cause the computing device to perform functions comprising:

capturing an initial image using the image sensor when the computing device is at an initial position;

determining a target displacement from the initial position for a desired stereoscopic effect;

providing an instruction through the user interface, wherein the instruction specifies a direction in which to move the computing device from the initial position;

while the computing device is in motion, estimating a displacement of the computing device from the initial position;

in response to determining that the estimated displacement corresponds to the target displacement, automatically capturing a candidate image using the image sensor;

in response to capturing the candidate image, automatically performing an acceptability analysis, wherein the acceptability analysis comprises analyzing the candidate image to determine whether the candidate image has acceptable image quality and comparing the candidate image to the initial image to determine whether the candidate image has acceptable similarity to the initial image; and

in response to determining that the candidate image has acceptable image quality and has acceptability similarity to the initial image, automatically creating a stereoscopic image based on the initial image and the candidate image.

26. The non-transitory computer readable medium of claim 25, wherein the computing device performs functions further comprising:

in response to determining that the candidate image lacks acceptable image quality, provide a further instruction.

27. The non-transitory computer readable medium of claim 25, wherein the computing device performs functions further comprising:

in response to determining that the candidate image lacks acceptable similarity to the initial image, provide a further instruction.

28. The non-transitory computer readable medium of claim 27, wherein the further instruction comprises an instruction to move the computing device back toward the initial position.

29. The non-transitory computer readable medium of claim 25, wherein comparing the candidate image to the initial image to determine whether the candidate image has acceptable similarity to the initial image comprises:

creating a predicted image based on the initial image and the target displacement; and

comparing the candidate image to the predicted image.

30. The non-transitory computer readable medium of claim 25, wherein the instruction specifies a distance from the initial position in which to move the computing device.