Ground plane detection for placement of augmented reality objects

• US 10,540,796 B2
• Filed: 06/18/2018
• Issued: 01/21/2020
• Est. Priority Date: 12/21/2017
• Status: Active Grant

• Current Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Original Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Inventors: Dagley, Geoffrey, Hoover, Jason, Tang, Qiaochu, Wylie, Stephen, Vasisht, Sunil, Price, Micah
• Primary Examiner(s): Thompson, James A
• Application Number:
  US16/011,030
• Publication Number:
  US 20190197750A1
• Time in Patent Office: 582 Days
• Field of Search: None
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2210/12 Bounding box
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

In some implementations, a system may include a camera, a display, one or more memories, and one or more processors communicatively coupled to the one or more memories. The system may identify a horizontal plane in an image being captured by the camera and presented on the display, may determine a size of the horizontal plane, and may determine that the size of the horizontal plane satisfies a threshold. The system may designate the horizontal plane as a ground plane based on determining that the size of the horizontal plane satisfies the threshold. The system may output an indication that the horizontal plane has been designated as the ground plane.

Claims

1. A device, comprising:

one or more memories; and

one or more processors, communicatively coupled to the one or more memories, to;

identify feature points in an image;

detect, based on identifying the feature points, multiple horizontal planes in the image;

estimate a vertical distance between the device and a particular horizontal plane of the multiple horizontal planes;

designate, based on estimating the vertical distance, the particular horizontal plane as a ground plane; and

provide, for display, an indication responsive to the particular horizontal plane being designated as the ground plane.

2. The device of claim 1, where the one or more processors are further to:

merge two or more detected horizontal planes to form the particular horizontal plane; and

where the one or more processors, when estimating the vertical distance between the device and the particular horizontal plane, are to;

estimate, based on merging the two or more detected horizontal planes, the vertical distance between the device and the particular horizontal plane.

3. The device of claim 1, where the one or more processors are further to:

determine an area value of a perimeter of the particular horizontal plane; and

determine that dimensions of the particular horizontal plane satisfy a threshold based on determining the area value; and

where the one or more processors, when designating the particular horizontal plane as the ground plane, are to;

designate, based on determining that the dimensions of the particular horizontal plane satisfy the threshold, the particular horizontal plane as the ground plane.

4. The device of claim 3, where the one or more processors are further to:

provide a user interface via which a user can at least one of select, adjust, reset, configure, or reconfigure the threshold.

5. The device of claim 3, where the one or more processors are further to:

apply artificial intelligence to determine the threshold.

6. The device of claim 1, where the one or more processors are further to:

determine a size of a perimeter of the particular horizontal plane; and

determine whether the size of the perimeter satisfies a threshold; and

where the one or more processors, when designating the particular horizontal plane as the ground plane, are to;

designate, based on determining that the size of the perimeter satisfies the threshold, the particular horizontal plane as the ground plane.

7. The device of claim 1, where the one or more processors are further to:

determine multiple measurements,

the multiple measurements being a function of multiple dimensions of a perimeter of the particular horizontal plane; and

determine whether the multiple measurements satisfy a threshold; and

where the one or more processors, when designating the particular horizontal plane as the ground plane, are to;

designate, based on determining that the multiple measurements satisfy the threshold, the particular horizontal plane as the ground plane.

8. A method, comprising:

identifying, by a device, feature points in an image;

detecting, by the device and based on identifying the feature points, multiple horizontal planes in the image;

estimating, by the device, a vertical distance between the device and a particular horizontal plane of the multiple horizontal planes;

designating, by the device and based on estimating the vertical distance, the particular horizontal plane as a ground plane; and

providing, by the device and for display, an indication responsive to the particular horizontal plane being designated as the ground plane.

9. The method of claim 8, further comprising:

providing a user interface for receiving an input designating one or more of the multiple horizontal planes; and

applying machine learning to modify a threshold used to designate the particular horizontal plane as the ground plane.

10. The method of claim 8, further comprising:

preventing a detection of an additional horizontal plane based on determining that multiple dimensions, of a perimeter of the particular horizontal plane, satisfy a threshold.

11. The method of claim 8, further comprising:

superimposing a representation of an augmented reality object on the image relative to the particular horizontal plane.

12. The method of claim 8, further comprising:

superimposing a representation of an augmented reality object on the image using the ground plane as a reference point.

13. The method of claim 8, further comprising:

superimposing a representation of an augmented reality object on the image using a guide marker.

14. The method of claim 8, further comprising:

superimposing a representation of an augmented reality object on the image based on performing image processing to determine a position for placement of the augment reality object.

15. A non-transitory computer-readable medium storing instructions, the instructions comprising:

one or more instructions that, when executed by one or more processors of a device, cause the one or more processors to;

identify feature points in an image;

detect, based on identifying the feature points, multiple horizontal planes in the image;

estimate a vertical distance between the device and a particular horizontal plane of the multiple horizontal planes;

designate, based on estimating the vertical distance, the particular horizontal plane as a ground plane; and

provide, for display, an indication responsive to the particular horizontal plane being designated as the ground plane.

16. The non-transitory computer-readable medium of claim 15, where the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:

identify an object in the image; and

place a representation of an augmented reality object on the object in the image.

17. The non-transitory computer-readable medium of claim 15, where the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:

identify multiple objects in the image;

prioritize the multiple objects,

a first object of the multiple objects having a first priority, and

a second object of the multiple objects having a second priority; and

superimpose a representation of an augmented reality object on the first object based on the first object having the first priority.

18. The non-transitory computer-readable medium of claim 15, where the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:

identify multiple objects in the image;

prioritize the multiple objects,

a first object of the multiple objects having a first priority, and

a second object of the multiple objects having a second priority; and

prevent a representation of an augmented reality object from being superimposed on the second object based on the second object having the second priority.

19. The non-transitory computer-readable medium of claim 15, where the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:

determine a transparency level for an augment reality object; and

superimpose the augment reality object on the image using the transparency level.

20. The non-transitory computer-readable medium of claim 15, where the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:

determine an orientation of an augment reality object based on the particular horizontal plane; and

superimpose the augment reality object on the image based on the orientation.