Method and System for Automatic 3-D Image Creation

• US 20130201301A1
• Filed: 08/31/2012
• Published: 08/08/2013
• Est. Priority Date: 02/06/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Huang, Jonathan, Kvaalen, Samuel, Bradshaw, Peter
• Granted Patent: US 8,711,209 B2
• Time in Patent Office: Days
• US Class Current: 348/50
• CPC Class Codes: G03B 35/02 by sequential recording
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 2213/003 Aspects relating to the "2D+depth" image format

Abstract

Disclosed herein are methods, devices, and non-transitory computer readable media that relate to stereoscopic image creation. A camera captures an initial image at an initial position. A target displacement from the initial position is determined for a desired stereoscopic effect, and an instruction is provided that specifies a direction in which to move the camera from the initial position. While the camera is in motion, an estimated displacement from the initial position is calculated. When the estimated displacement corresponds to the target displacement, the camera automatically captures a candidate image. An acceptability analysis is performed to determine whether the candidate image has acceptable image quality and acceptable similarity to the initial image. If the candidate image passes the acceptability analysis, a stereoscopic image is created based on the initial and candidate images.

Claims

1-30. -30. (canceled)

31. A method for stereoscopic image creation, the method comprising:

capturing an initial image using a device at an initial position;

determining a target displacement from the initial position;

estimating a displacement of the device from the initial position;

in response to determining that the estimated displacement corresponds to the target displacement, the device automatically capturing a candidate image;

in response to capturing the candidate image, automatically performing an acceptability analysis, wherein the acceptability analysis comprises analyzing the candidate image to determine whether the candidate image has acceptable image quality and comparing the candidate image to the initial image to determine whether the candidate image has acceptable similarity to the initial image; and

in response to determining that the candidate image has acceptable image quality and has acceptability similarity to the initial image, automatically creating a stereoscopic image based on the initial image and the candidate image.

32. The method of claim 31, further comprising:

storing the stereoscopic image as an image file.

33. The method of claim 31, further comprising:

in response to determining that the candidate image lacks acceptable similarity to the initial image, providing an instruction to move the device back toward the initial position.

34. The method of claim 31, wherein comparing the candidate image to the initial image to determine whether the candidate image has acceptable similarity to the initial image comprises:

creating a predicted image based on the initial image and the target displacement; and

comparing the candidate image to the predicted image.

35. The method of claim 31, wherein estimating the displacement of the device from the initial position comprises:

obtaining at least one sensor reading from at least one motion sensor, wherein the at least one motion sensor comprises a gyroscope.

36. The method of claim 31, wherein estimating the displacement of the device from the initial position comprises:

obtaining at least one sensor reading from at least one motion sensor, wherein the at least one motion sensor comprises an accelerometer.

37. The method of claim 31, wherein estimating the displacement of the device from the initial position comprises:

the device capturing multiple images while the device is in motion; and

calculating motion vectors between individual images in the multiple images.

38. The method of claim 31, wherein estimating the displacement of the device from the initial position comprises:

obtaining a GPS position of the device.

39. The method of claim 31, wherein estimating the displacement of the device from the initial position comprises:

obtaining a position of the device using wireless triangulation.

40. The method of claim 31, further comprising:

providing an instruction that specifies a distance from the initial position in which to move the device.

41. A device, comprising:

an image sensor;

a user interface; and

a controller, wherein the controller is configured to;

capture an initial image using the image sensor when the device is at an initial position;

determine a target displacement from the initial position;

estimate a displacement of the device from the initial position;

in response to determining that the estimated displacement corresponds to the target displacement, automatically capture a candidate image using the image sensor;

in response to capturing the candidate image, automatically perform an acceptability analysis, wherein the acceptability analysis comprises analyzing the candidate image to determine whether the candidate image has acceptable image quality and comparing the candidate image to the initial image to determine whether the candidate image has acceptable similarity to the initial image; and

in response to determining that the candidate image has acceptable image quality and has acceptability similarity to the initial image, automatically create a stereoscopic image based on the initial image and the candidate image.

42. The device of claim 41, further comprising a global positioning system (GPS) transceiver and wherein the controller estimates the displacement of the device based at least on GPS data.

43. The device of claim 41, further comprising a motion sensor and wherein the controller estimates the displacement of the device based at least on motion sensor data.

44. The device of claim 41, further comprising a range finder and wherein the controller estimates the displacement of the device based at least on range finder data.

45. The device of claim 41, further comprising a memory, wherein the controller is further configured to store the stereoscopic image in the memory as an image file.

46. The device of claim 41, further comprising a display, wherein the controller is further configured to display the stereoscopic image on the display.

47. The device of claim 41, wherein the controller is further configured to, in response to determining that the candidate image lacks acceptable image quality, provide an instruction through the user interface.

48. The device of claim 41, wherein the controller is further configured to, in response to determining that the candidate image lacks acceptable similarity to the initial image, provide an instruction through the user interface.

49. A non-transitory computer readable medium having stored therein instructions executable by a computing device, the computing device including an image sensor and a user interface, to cause the computing device to perform functions comprising:

capturing an initial image using the image sensor when the computing device is at an initial position;

determining a target displacement from the initial position;

estimating a displacement of the computing device from the initial position;

in response to determining that the estimated displacement corresponds to the target displacement, automatically capturing a candidate image using the image sensor;

in response to capturing the candidate image, automatically performing an acceptability analysis, wherein the acceptability analysis comprises analyzing the candidate image to determine whether the candidate image has acceptable image quality and comparing the candidate image to the initial image to determine whether the candidate image has acceptable similarity to the initial image; and

in response to determining that the candidate image has acceptable image quality and has acceptability similarity to the initial image, automatically creating a stereoscopic image based on the initial image and the candidate image.

50. The non-transitory computer readable medium of claim 49, wherein comparing the candidate image to the initial image to determine whether the candidate image has acceptable similarity to the initial image comprises:

creating a predicted image based on the initial image and the target displacement; and

comparing the candidate image to the predicted image.