Ground plane detection for placement of augmented reality objects

• US 10,026,209 B1
• Filed: 12/21/2017
• Issued: 07/17/2018
• Est. Priority Date: 12/21/2017
• Status: Active Grant

• Current Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Original Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Inventors: Dagley, Geoffrey, Hoover, Jason, Tang, Qiaochu, Wylie, Stephen, Vasisht, Sunil, Price, Micah
• Primary Examiner(s): Thompson, James A
• Application Number:
  US15/850,234
• Time in Patent Office: 208 Days
• Field of Search: None
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2210/12 Bounding box
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

In some implementations, a system may include a camera, a display, one or more memories, and one or more processors communicatively coupled to the one or more memories. The system may identify a horizontal plane in an image being captured by the camera and presented on the display, may determine a size of the horizontal plane, and may determine that the size of the horizontal plane satisfies a threshold. The system may designate the horizontal plane as a ground plane based on determining that the size of the horizontal plane satisfies the threshold. The system may output an indication that the horizontal plane has been designated as the ground plane.

Claims

1. A system, comprising:

a camera;

a display;

one or more memories; and

one or more processors, communicatively coupled to the one or more memories, to;

identify a horizontal plane in an image being captured by the camera and presented on the display;

determine a perimeter of the horizontal plane;

determine that a first dimension of the perimeter of the horizontal plane satisfies a first threshold;

determine that a second dimension of the perimeter of the horizontal plane satisfies a second threshold;

designate the horizontal plane as a ground plane based on determining that the first dimension satisfies the first threshold and the second dimension satisfies the second threshold; and

output an indication that the horizontal plane has been designated as the ground plane.

2. The system of claim 1, where the one or more processors are further to:

superimpose a guide marker on the image;

determine three-dimensional (3D) coordinates of an intersection point associated with the guide marker,

the intersection point being a point where a projection of the guide marker into 3D space intersects with the horizontal plane designated as the ground plane;

determine two-dimensional (2D) coordinates associated with presentation of an augmented reality object on the image based on the 3D coordinates of the intersection point; and

superimpose a representation of the augmented reality object on the image based on the 2D coordinates.

3. The system of claim 1, where the indication is output for at least one of:

presentation on the display, or

storage in the one or more memories.

4. The system of claim 1, where the one or more processors are further to:

extend the horizontal plane in one or more directions based on determining that a size associated with the perimeter of the horizontal plane satisfies a size threshold.

5. The system of claim 1, where the one or more processors are further to:

prevent additional horizontal plane detection based on determining that a size associated with the perimeter of the horizontal plane satisfies a size threshold.

6. The system of claim 1, where the one or more processors are further to:

determine that an area of the horizontal plane satisfies an area threshold; and

where the one or more processors, when designating the horizontal plane as the ground plane, are to;

designate the horizontal plane as the ground plane based on determining that the area of the horizontal plane satisfies the area threshold.

7. The system of claim 1, where the one or more processors are further to:

superimpose, based on outputting the indication, a representation of an augmented reality object on the image relative to the horizontal plane.

8. A method, comprising:

identifying, by a device, a horizontal plane in an image being captured by the device and presented on a display of the device;

determining, by the device, a perimeter of the horizontal plane;

determining, by the device, that a first dimension of the perimeter of the horizontal plane satisfies a first threshold;

determining, by the device, that a second dimension of the perimeter of the horizontal plane satisfies a second threshold;

designating, by the device, the horizontal plane as a ground plane based on determining that the first dimension satisfies the first threshold and the second dimension satisfies the second threshold; and

superimposing, by the device, a representation of an augmented reality object on the image relative to the horizontal plane designated as the ground plane.

9. The method of claim 8, further comprising:

superimposing a guide marker on the image;

determining three-dimensional (3D) coordinates of an intersection point associated with the guide marker,

the intersection point being a point where a projection of the guide marker into 3D space intersects with the horizontal plane designated as the ground plane;

determining two-dimensional (2D) coordinates associated with presentation of the augmented reality object on the image based on the 3D coordinates of the intersection point; and

superimposing the representation of the augmented reality object on the image based on the 2D coordinates.

10. The method of claim 9, where the guide marker includes at least one of:

a guide line,

a guide box, or

a bounding shape.

11. The method of claim 8, further comprising:

presenting, for display by the device, an indication that the horizontal plane has been designated as the ground plane.

12. The method of claim 8, further comprising:

extending the horizontal plane in one or more directions based on determining that a size associated with the perimeter of the horizontal plane satisfies a size threshold.

13. The method of claim 8, further comprising:

preventing additional horizontal plane detection based on determining that a size associated with the perimeter of the horizontal plane satisfies a size threshold.

14. The method of claim 8, further comprising:

determining that an area of the horizontal plane satisfies an area threshold;

and

where designating the horizontal plane as the ground plane comprises;

designating the horizontal plane as the ground plane based on determining that the area of the horizontal plane satisfies the area threshold.

15. The method of claim 8, further comprising:

merging multiple detected horizontal planes to form the horizontal plane.

16. A non-transitory computer-readable medium storing instructions, the instructions comprising:

one or more instructions that, when executed by one or more processors of a device, cause the one or more processors to;

identify a horizontal plane in an image being captured by the device and presented on a display of the device;

determine a perimeter of the horizontal plane;

determine that a first dimension of the perimeter of the horizontal plane satisfies a first threshold;

determine that a second dimension of the perimeter of the horizontal plane satisfies a second threshold;

designate the horizontal plane as a ground plane based on determining that the first dimension satisfies the first threshold and the second dimension satisfies the second threshold; and

output an indication that the horizontal plane has been designated as the ground plane.

17. The non-transitory computer-readable medium of claim 16, where the one or more instructions, when executed by the one or more processors, cause the one or more processors to:

superimpose a guide marker on the image;

determine three-dimensional (3D) coordinates of an intersection point associated with the guide marker,

the intersection point being a point where a projection of the guide marker into 3D space intersects with the horizontal plane designated as the ground plane;

determine two-dimensional (2D) coordinates associated with presentation of an augmented reality object on the image based on the 3D coordinates of the intersection point; and

superimpose a representation of the augmented reality object on the image based on the 2D coordinates.

18. The non-transitory computer-readable medium of claim 17, where the guide marker includes at least one of a guide line, a guide box, or a bounding shape.

19. The non-transitory computer-readable medium of claim 16, where the indication is output for presentation on the display of the device.

20. The non-transitory computer-readable medium of claim 16, where the one or more instructions, when executed by the one or more processors, cause the one or more processors to:

merge multiple detected horizontal planes to form the horizontal plane.