GENERATING A 3D INTERACTIVE IMMERSIVE EXPERIENCE FROM A 2D STATIC IMAGE

• US 20150062125A1
• Filed: 08/27/2014
• Published: 03/05/2015
• Est. Priority Date: 09/03/2013
• Status: Active Grant

• Current Assignee: 3DITIZE SL
• Original Assignee: 3DITIZE SL
• Inventors: Aguilera Perez, Jaime, Alonso Blazquez, Fernando, Gmez Fernandez, Juan Bautista
• Granted Patent: US 9,990,760 B2
• Time in Patent Office: Days
• US Class Current: 345/427
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20101 Interactive definition of point of interest, landmark or seed
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/543 from line drawings
• CPC Class Codes: G06T 7/75 involving models

Abstract

A two-dimensional (2D) static image may be used to generate a three-dimensional (3D) interactive immersive experience. An image type of the 2D image may first be identified. The image type may be selected from a set of types such as interior, exterior, people, corridor, landscape, and other. Each image type may have an associated main feature type. The main feature of the 2D image may be identified using the corresponding main feature type. Then, unless the 2D image is of the “other” image type, a 3D object with two or more planes may be generated. The planes may intersect on the identified main feature. A virtual camera may be positioned proximate the 3D object, and a 3D view of the 3D object may be generated and displayed for the user. The user may optionally move the virtual camera, within limits, to view the 3D object from other locations.

Claims

1. A computer-implemented method for generating a three-dimensional view from a first two-dimensional image, the method comprising:

receiving a first two-dimensional image;

identifying, from a plurality of image types, a first image type of the two-dimensional image;

identifying, within the first two-dimensional image, a first main feature having a first main feature type associated with the first image type;

at a processor, generating a first plane;

at the processor, generating a second plane that intersects the first plane at a line on the first main feature;

at the processor, applying the first two-dimensional image to the first plane and the second plane to define a first three-dimensional object;

at the processor, positioning a virtual camera at a first camera position proximate the first three-dimensional object;

at the processor, generating a first three-dimensional view of the first three-dimensional object from the virtual camera; and

at a display screen, displaying the first three-dimensional view.

2. The computer-implemented method of claim 1, wherein identifying the first image type comprises, at an input device, receiving first user input indicating the first image type.

3. The computer-implemented method of claim 2, further comprising, prior to identifying the first main feature, selecting the first main feature type from a plurality of main feature types based on the first image type.

4. The computer-implemented method of claim 3, wherein identifying the first main feature comprises, at the input device, receiving second user input identifying the first main feature.

5. The computer-implemented method of claim 1, wherein the plurality of image types comprises an interior image type, an exterior image type, a people image type, a corridor image type, and a landscape image type.

6. The computer-implemented method of claim 1, wherein the first image type comprises an interior image type indicative of representation by the first two-dimensional image of an interior of a structure;

wherein the first main feature comprises a concave vertical corner of the structure;

and wherein identifying the first main feature comprises positioning a vertical line proximate the concave vertical corner.

7. The computer-implemented method of claim 1, wherein the first image type comprises an exterior image type indicative of representation by the first two-dimensional image of an exterior of a structure;

wherein the first main feature comprises a convex vertical corner of the structure;

and wherein identifying the first main feature comprises positioning a vertical line proximate the convex vertical corner.

8. The computer-implemented method of claim 1, wherein the first image type comprises a people image type indicative of representation by the first two-dimensional image of a person;

wherein the first main feature comprises a face of the person;

and wherein identifying the first main feature comprises positioning a rectangle to frame the face.

9. The computer-implemented method of claim 1, wherein the first image type comprises a corridor image type indicative of representation by the first two-dimensional image of a corridor within a structure;

wherein the first main feature comprises the corridor;

and wherein identifying the first main feature comprises positioning a rectangle to define boundaries of the corridor.

10. The computer-implemented method of claim 1, wherein the first image type comprises a landscape image type indicative of representation by the first two-dimensional image of a landscape comprising a background and a floor;

wherein the first main feature comprises a concave horizontal corner defining an intersection between the background and the floor;

and wherein identifying the first main feature comprises positioning a horizontal line proximate the concave horizontal corner.

11. The computer-implemented method of claim 1, wherein the first two-dimensional image comprises an image plane that bounds a field of view;

wherein the first image type comprises a selection from the group consisting of an interior image type and an exterior image type;

wherein the first main feature comprises a vertical corner of a structure;

wherein identifying the first main feature comprises positioning a vertical line proximate the vertical corner;

wherein generating the first plane comprises;

defining an ellipse comprising a major axis within the image plane, wherein, at a first major end of the ellipse, the major axis intersects a first edge of the field of view to define a first intersection and, at a second major end of the ellipse, the major axis intersects a second edge of the field of view to define a second intersection; and

defining the first plane such that the first plane passes through the first intersection and the vertical line;

and wherein defining the second plane comprises defining the second plane such that the second plane passes through the second intersection and the vertical line.

12. The computer-implemented method of claim 11, wherein defining the ellipse further comprises setting a length of a minor axis of the ellipse such that the minor axis has a minor length ranging from one quarter to one half of a major length of the major axis.

13. The computer-implemented method of claim 1, wherein the first two-dimensional image comprises an image plane that bounds a field of view;

wherein the first image type comprises a selection from the group consisting of a people image type and a corridor image type;

wherein the first main feature is selected from the group consisting of a face of a person and a corridor;

wherein identifying the first main feature comprises positioning a rectangle proximate the first main feature, the rectangle comprising a first vertical edge and a second vertical edge;

wherein generating the first plane comprises;

defining an ellipse comprising a major axis within the image plane, wherein, at a first major end of the ellipse, the major axis intersects a first edge of the field of view to define a first intersection and, at a second major end of the ellipse, the major axis intersects a second edge of the field of view to define a second intersection; and

defining the first plane such that the first plane passes through the first intersection and the first vertical edge;

wherein defining the second plane comprises defining the second plane such that the second plane is tangent to the ellipse and passes through the first intersection and the second intersection;

wherein the method further comprises defining a third plane such that the third plane passes through the second vertical edge and the second intersection;

and wherein applying the two-dimensional image to the first plane and the second plane further comprises applying the first two-dimensional image to the third plane to define the first three-dimensional object.

14. The computer-implemented method of claim 13, wherein defining the ellipse further comprises setting a length of a minor axis of the ellipse such that the minor axis has a minor length of about five sixteenths of a major length of the major axis.

15. The computer-implemented method of claim 1, wherein the first two-dimensional image comprises an image plane that bounds a field of view;

wherein the first image type is a landscape image type;

wherein the first main feature comprises a concave horizontal corner defining an intersection between a background and a floor;

wherein identifying the first main feature comprises positioning a horizontal line proximate the concave horizontal corner;

wherein generating the first plane comprises defining the first plane such that the first plane extends horizontally and passes through the horizontal line;

and wherein defining the second plane comprises defining the second plane such that the second plane extends vertically and passes through the horizontal line.

16. The computer-implemented method of claim 1, further comprising:

receiving a second two-dimensional image;

identifying, from the plurality of image types, a second image type of the second two-dimensional image, wherein the second image type is an “

other”

image type;

at a processor, generating a third plane;

at the processor, applying the second two-dimensional image to the third plane to define a second three-dimensional object;

at the processor, positioning the virtual camera proximate the second three-dimensional object;

at the processor, generating a second three-dimensional view of the second three-dimensional object; and

at the display screen, displaying the second three-dimensional view.

17. The computer-implemented method of claim 1, further comprising:

at an input device, receiving a first user input;

in response to receipt of the first user input, positioning the virtual camera at a second camera position offset from the first camera position;

at the processor, generating a second three-dimensional view of the first three-dimensional object; and

at the display screen, displaying the second three-dimensional view.

18. The computer-implemented method of claim 17, further comprising:

at the input device, receiving a second user input indicating a desire to move the virtual camera to a third virtual camera position;

in response to receipt of the second user input, determining that, at the third virtual camera position, a field of view of the virtual camera would not be entirely occluded by the first three-dimensional object; and

in response to determining that, at the third virtual camera position, the field of view would not be entirely occluded by the first three-dimensional object, determining that the virtual camera will not be moved to the third virtual camera position.

19. The computer-implemented method of claim 1, wherein identifying the first image type comprises, at the processor, automatically determining the first image type based on identification of an image feature of the first two-dimensional image.

20. A computer program product for generating a three-dimensional view from a first two-dimensional image, the computer program product comprising:

a non-transitory storage medium; and

computer program code, encoded on the medium, configured to cause at least one processor to perform the steps of;

receiving a first two-dimensional image;

identifying, from a plurality of image types, a first image type of the two-dimensional image;

identifying, within the first two-dimensional image, a first main feature having a first main feature type associated with the first image type;

generating a first plane;

generating a second plane that intersects the first plane at a line on the first main feature;

applying the first two-dimensional image to the first plane and the second plane to define a first three-dimensional object;

positioning a virtual camera at a first camera position proximate the first three-dimensional object;

generating a first three-dimensional view of the first three-dimensional object from the virtual camera; and

causing a display screen to display the first three-dimensional view.

21. The computer program product of claim 20, wherein the computer program code is further configured to cause the at least one processor to identify the first image type by causing an input device to receive first user input indicating the first image type.

22. The computer program product of claim 21, wherein the computer program code is further configured to cause the at least one processor, prior to identifying the first main feature, to select the first main feature type from a plurality of main feature types based on the first image type;

and wherein the computer program code is further configured to cause the at least one processor to identify the first main feature by causing the input device to receive second user input identifying the first main feature.

23. The computer program product of claim 20, wherein the first image type comprises an interior image type indicative of representation by the first two-dimensional image of an interior of a structure;

wherein the first main feature comprises a concave vertical corner of the structure;

and wherein the computer program code is further configured to cause the at least one processor to identify the first main feature by positioning a vertical line proximate the concave vertical corner.

24. The computer program product of claim 20, wherein the first image type comprises an exterior image type indicative of representation by the first two-dimensional image of an exterior of a structure;

wherein the first main feature comprises a convex vertical corner of the structure;

and wherein the computer program code is further configured to cause the at least one processor to identify the first main feature by positioning a vertical line proximate the convex vertical corner.

25. The computer program product of claim 20, wherein the first image type comprises a people image type indicative of representation by the first two-dimensional image of a person;

wherein the first main feature comprises a face of the person;

and wherein the computer program code is further configured to cause the at least one processor to identify the first main feature by positioning a rectangle to frame the face.

26. The computer program product of claim 20, wherein the first image type comprises a corridor image type indicative of representation by the first two-dimensional image of a corridor within a structure;

wherein the first main feature comprises the corridor;

and wherein the computer program code is further configured to cause the at least one processor to identify the first main feature by positioning a rectangle to define boundaries of the corridor.

27. The computer program product of claim 20, wherein the first image type comprises a landscape image type indicative of representation by the first two-dimensional image of a landscape comprising a background and a floor;

wherein the first main feature comprises a concave horizontal corner defining an intersection between the background and the floor;

and wherein the computer program code is further configured to cause the at least one processor to identify the first main feature by positioning a horizontal line proximate the concave horizontal corner.

28. The computer program product of claim 20, wherein the first two-dimensional image comprises an image plane that bounds a field of view;

wherein the first image type comprises a selection from the group consisting of an interior image type and an exterior image type;

wherein the first main feature comprises a vertical corner of a structure;

wherein the computer program code is further configured to cause the at least one processor to identify the first main feature by positioning a vertical line proximate the vertical corner;

wherein the computer program code is further configured to cause the at least one processor to generate the first plane by;

defining an ellipse comprising a major axis within the image plane, wherein, at a first major end of the ellipse, the major axis intersects a first edge of the field of view to define a first intersection and, at a second major end of the ellipse, the major axis intersects a second edge of the field of view to define a second intersection; and

defining the first plane such that the first plane passes through the first intersection and the vertical line;

and wherein the computer program code is further configured to cause the at least one processor to define the second plane by defining the second plane such that the second plane passes through the second intersection and the vertical line.

29. The computer program product of claim 20, wherein the first two-dimensional image comprises an image plane that bounds a field of view;

wherein the first image type comprises a selection from the group consisting of a people image type and a corridor image type;

wherein the first main feature is selected from the group consisting of a face of a person and a corridor;

wherein the computer program code is further configured to cause the at least one processor to identify the first main feature by positioning a rectangle proximate the first main feature, the rectangle comprising a first vertical edge and a second vertical edge;

wherein the computer program code is further configured to cause the at least one processor to generate the first plane by;

defining an ellipse comprising a major axis within the image plane, wherein, at a first major end of the ellipse, the major axis intersects a first edge of the field of view to define a first intersection and, at a second major end of the ellipse, the major axis intersects a second edge of the field of view to define a second intersection; and

defining the first plane such that the first plane passes through the first intersection and the first vertical edge;

wherein the computer program code is further configured to cause the at least one processor to define the second plane by defining the second plane such that the second plane is tangent to the ellipse and passes through the first intersection and the second intersection;

wherein the computer program code is further configured to cause the at least one processor to define a third plane such that the third plane passes through the second vertical edge and the second intersection;

and wherein the computer program code is further configured to cause the at least one processor to apply the two-dimensional image to the first plane and the second plane by applying the first two-dimensional image to the third plane to define the first three-dimensional object.

30. The computer program product of claim 20, wherein the first two-dimensional image comprises an image plane that bounds a field of view;

wherein the first image type is a landscape image type;

wherein the first main feature comprises a concave horizontal corner defining an intersection between a background and a floor;

wherein the computer program code is further configured to cause the at least one processor to identify the first main feature by positioning a horizontal line proximate the concave horizontal corner;

wherein the computer program code is further configured to cause the at least one processor to generate the first plane by defining the first plane such that the first plane extends horizontally and passes through the horizontal line;

and wherein the computer program code is further configured to cause the at least one processor to define the second plane by defining the second plane such that the second plane extends vertically and passes through the horizontal line.

31. The computer program product of claim 20, wherein the computer program code is further configured to cause the at least one processor to perform the steps of:

causing an input device to receive a first user input;

in response to receipt of the first user input, positioning the virtual camera at a second camera position offset from the first camera position;

generating a second three-dimensional view of the first three-dimensional object; and

causing the display screen to display the second three-dimensional view.

32. The computer program product of claim 31, wherein the computer program code is further configured to cause the at least one processor to perform the steps of:

causing the input device to receive a second user input indicating a desire to move the virtual camera to a third virtual camera position;

in response to receipt of the second user input, determining that, at the third virtual camera position, a field of view of the virtual camera would not be entirely occluded by the first three-dimensional object; and

in response to determining that, at the third virtual camera position, the field of view would not be entirely occluded by the first three-dimensional object, determining that the virtual camera will not be moved to the third virtual camera position.

33. A system for generating a three-dimensional view from a first two-dimensional image, the system comprising:

a processor, configured to;

receive a first two-dimensional image;

identify, from a plurality of image types, a first image type of the two-dimensional image;

identify, within the first two-dimensional image, a first main feature having a first main feature type associated with the first image type;

generate a first plane;

generate a second plane that intersects the first plane at a line on the first main feature;

apply the first two-dimensional image to the first plane and the second plane to define a first three-dimensional object;

position a virtual camera at a first camera position proximate the first three-dimensional object; and

generate a first three-dimensional view of the first three-dimensional object from the virtual camera; and

a display screen, communicatively coupled to the processor, configured to display the first three-dimensional view.

34. The system of claim 33, further comprising,

an input device, communicatively coupled to the processor, configured to receive first user input indicating the first image type;

wherein the processor is further configured to identify the first image type based on the first user input.

35. The system of claim 34, wherein the processor is further configured, prior to identification of the first main feature, to select the first main feature type from a plurality of main feature types based on the first image type;

wherein the input device is further configured to receive second user input identifying the first main feature;

and wherein the processor is further configured to identify the first main feature based on the second user input.

36. The system of claim 33, wherein the first image type comprises an interior image type indicative of representation by the first two-dimensional image of an interior of a structure;

wherein the first main feature comprises a concave vertical corner of the structure;

and wherein the processor is further configured to identify the first main feature by positioning a vertical line proximate the concave vertical corner.

37. The system of claim 33, wherein the first image type comprises an exterior image type indicative of representation by the first two-dimensional image of an exterior of a structure;

wherein the first main feature comprises a convex vertical corner of the structure;

and wherein the processor is further configured to identify the first main feature by positioning a vertical line proximate the convex vertical corner.

38. The system of claim 33, wherein the first image type comprises a people image type indicative of representation by the first two-dimensional image of a person;

wherein the first main feature comprises a face of the person;

and wherein the processor is further configured to identify the first main feature by positioning a rectangle to frame the face.

39. The system of claim 33, wherein the first image type comprises a corridor image type indicative of representation by the first two-dimensional image of a corridor within a structure;

wherein the first main feature comprises the corridor;

and wherein the processor is further configured to identify the first main feature by positioning a rectangle to define boundaries of the corridor.

40. The system of claim 33, wherein the first image type comprises a landscape image type indicative of representation by the first two-dimensional image of a landscape comprising a background and a floor;

wherein the first main feature comprises a concave horizontal corner defining an intersection between the background and the floor;

and wherein the processor is further configured to identify the first main feature by positioning a horizontal line proximate the concave horizontal corner.

41. The system of claim 33, wherein the first two-dimensional image comprises an image plane that bounds a field of view;

wherein the first image type comprises a selection from the group consisting of an interior image type and an exterior image type;

wherein the first main feature comprises a vertical corner of a structure;

wherein the processor is further configured to identify the first main feature by positioning a vertical line proximate the vertical corner;

wherein the processor is further configured to generate the first plane by;

defining an ellipse comprising a major axis within the image plane, wherein, at a first major end of the ellipse, the major axis intersects a first edge of the field of view to define a first intersection and, at a second major end of the ellipse, the major axis intersects a second edge of the field of view to define a second intersection; and

defining the first plane such that the first plane passes through the first intersection and the vertical line;

and wherein the processor is further configured to define the second plane by defining the second plane such that the second plane passes through the second intersection and the vertical line.

42. The system of claim 33, wherein the first two-dimensional image comprises an image plane that bounds a field of view;

wherein the first image type comprises a selection from the group consisting of a people image type and a corridor image type;

wherein the first main feature is selected from the group consisting of a face of a person and a corridor;

wherein the processor is further configured to identify the first main feature by positioning a rectangle proximate the first main feature, the rectangle comprising a first vertical edge and a second vertical edge;

wherein the processor is further configured to generate the first plane by;

defining an ellipse comprising a major axis within the image plane, wherein, at a first major end of the ellipse, the major axis intersects a first edge of the field of view to define a first intersection and, at a second major end of the ellipse, the major axis intersects a second edge of the field of view to define a second intersection; and

defining the first plane such that the first plane passes through the first intersection and the first vertical edge;

wherein the processor is further configured to define the second plane by defining the second plane such that the second plane is tangent to the ellipse and passes through the first intersection and the second intersection;

wherein the processor is further configured to define a third plane such that the third plane passes through the second vertical edge and the second intersection;

and wherein the processor is further configured to apply the two-dimensional image to the first plane and the second plane by applying the first two-dimensional image to the third plane to define the first three-dimensional object.

43. The system of claim 33, wherein the first two-dimensional image comprises an image plane that bounds a field of view;

wherein the first image type is a landscape image type;

wherein the first main feature comprises a concave horizontal corner defining an intersection between a background and a floor;

wherein the processor is further configured to identify the first main feature by positioning a horizontal line proximate the concave horizontal corner;

wherein the processor is further configured to generate the first plane by defining the first plane such that the first plane extends horizontally and passes through the horizontal line;

and wherein the processor is further configured to defining the second plane by defining the second plane such that the second plane extends vertically and passes through the horizontal line.

44. The system of claim 33, further comprising:

an input device configured to receive a first user input;

wherein the processor is further configured to;

in response to receipt of the first user input, position the virtual camera at a second camera position offset from the first camera position; and

generate a second three-dimensional view of the first three-dimensional object; and

and wherein the display screen is further configured to display the second three-dimensional view.

45. The system of claim 44, wherein the input device is further configured to receive a second user input indicating a desire to move the virtual camera to a third virtual camera position;

wherein the processor is further configured to;

in response to receipt of the second user input, determine that, at the third virtual camera position, a field of view of the virtual camera would not be entirely occluded by the first three-dimensional object; and

in response to determining that, at the third virtual camera position, the field of view would not be entirely occluded by the first three-dimensional object, determine that the virtual camera will not be moved to the third virtual camera position.