Placement of augmented reality objects using a guide marker

• US 10,002,442 B1
• Filed: 12/21/2017
• Issued: 06/19/2018
• Est. Priority Date: 12/21/2017
• Status: Active Grant

• Current Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Original Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Inventors: Dagley, Geoffrey, Hoover, Jason, Tang, Qiaochu, Wylie, Stephen, Vasisht, Sunil, Price, Micah
• Primary Examiner(s): Thompson, James A
• Application Number:
  US15/850,387
• Time in Patent Office: 180 Days
• Field of Search: None
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2210/12 Bounding box
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

In some implementations, a system may include a camera, a display, one or more memories, and one or more processors communicatively coupled to the one or more memories. The system may output a guide marker for presentation on the display. The guide marker may be superimposed on an image being captured by the camera and presented on the display. The system may determine three-dimensional (3D) coordinates of an intersection point associated with the guide marker. The intersection point may be a point where a projection of the guide marker into 3D space intersects with a horizontal plane identified in the image. The system may determine two-dimensional (2D) coordinates for presentation of an augmented reality object on the display based on the 3D coordinates of the intersection point. The system may superimpose a representation of the augmented reality object on the image based on the 2D coordinates.

Claims

1. A system, comprising:

a camera;

a display;

one or more memories; and

one or more processors, communicatively coupled to the one or more memories, to;

output a guide marker for presentation on the display, the guide marker being superimposed on an image being captured by the camera and presented on the display, the guide marker being output on the display to allow a user to position the guide marker at a base of a target object within the image;

determine two-dimensional (2D) coordinates of an intersection point associated with the guide marker on the display;

determine three-dimensional (3D) coordinates of the intersection point associated with the guide marker, the intersection point being a point where a projection of the guide marker into 3D space intersects with a horizontal plane identified in the image;

determine 3D coordinates associated with placement of an augmented reality object based on the 3D coordinates of the intersection point;

determine 2D coordinates for presentation of the augmented reality object on the display based on at least two or more of;

the 2D coordinates that represent the intersection point associated with the guide marker on the display, the 3D coordinates associated with placement of the augmented reality object, or the 3D coordinates of the intersection point; and

superimpose a representation of the augmented reality object on the image based on the 2D coordinates for presentation of the augmented reality object on the display.

2. The system of claim 1, where the 2D coordinates for presentation of the augmented reality object are determined by applying an offset to the 2D coordinates that represent the position of the guide marker on the display.

3. The system of claim 2, where the offset is determined based on comparing the 3D coordinates associated with placement of the augmented reality object and the 3D coordinates of the intersection point.

4. The system of claim 2, where the offset represents a distance between the augmented reality object and the horizontal plane.

5. The system of claim 2, where the offset is determined based on a height of the target object in the image.

6. The system of claim 1, where the one or more processors are further to:

determine the 3D coordinates of the intersection point based on an indication that the guide marker is aligned with an intersection of the target object in the image and the horizontal plane.

7. The system of claim 1, where the horizontal plane is designated as a ground plane.

8. The system of claim 1, where the one or more processors are further to:

determine a size for the augmented reality object based on a size of the target object.

9. A method, comprising:

outputting, by a device, a guide marker for presentation on a display of the device, the guide marker being superimposed on an image being captured by the device and presented on the display of the device, the guide marker being output on the display to allow a user to position the guide marker at a base of a target object within the image;

determining, by the device, two-dimensional (2D) coordinates of an intersection point associated with the guide marker on the display;

determining, by the device, three-dimensional (3D) coordinates of the intersection point associated with the guide marker, the intersection point being a point where a projection of the guide marker into 3D space intersects with a horizontal plane identified in the image;

determining, by the device, 3D coordinates associated with placement of an augmented reality object based on the 3D coordinates of the intersection point;

determining, by the device, 2D coordinates for presentation of the augmented reality object on the display of the device based on at least two or more of;

the 2D coordinates that represent the intersection point associated with the guide marker on the display, the 3D coordinates associated with placement of the augmented reality object, or the 3D coordinates of the intersection point; and

superimposing, by the device, a representation of the augmented reality object on the image based on the 2D coordinates for presentation of the augmented reality object on the display.

10. The method of claim 9, where the 2D coordinates for presentation of the augmented reality object are generated by applying an offset to the 2D coordinates that represent the position of the guide marker on the display of the device.

11. The method of claim 10, where the offset represents at least one of:

a distance between the augmented reality object and the horizontal plane, or a height of the target object in the image.

12. The method of claim 9, where the 3D coordinates associated with placement of the augmented reality object are generated by applying an offset to the 3D coordinates of the intersection point.

13. The method of claim 12, where the offset represents at least one of:

a distance between the augmented reality object and the horizontal plane, or a height of the target object in the image.

14. The method of claim 9, where the horizontal plane is designated as a ground plane.

15. The method of claim 9, further comprising:

determining a size for the augmented reality object based on a size of the target object.

16. A non-transitory computer-readable medium storing instructions, the instructions comprising:

one or more instructions that, when executed by one or more processors of a device, cause the one or more processors to;

output a guide marker for presentation on a display of the device, the guide marker being superimposed on an image being captured by the device and presented on the display, the guide marker being output on the display to allow a user to position the guide marker at a base of a target object within the image;

determine two-dimensional (2D) coordinates of an intersection point associated with the guide marker on the display;

determine three-dimensional (3D) coordinates of the intersection point associated with the guide marker, the intersection point being a point where a projection of at least a portion of the guide marker into 3D space intersects with a horizontal plane identified in the image;

determine 3D coordinates associated with placement of an augmented reality object based on the 3D coordinates of the intersection point;

determine 2D coordinates for presentation of the augmented reality object on the display based on at least two or more of;

the 2D coordinates that represent the intersection point associated with the guide marker on the display, the 3D coordinates associated with placement of the augmented reality object, or the 3D coordinates of the intersection point; and

superimpose a representation of the augmented reality object on the image based on the 2D coordinates for presentation of the augmented reality object on the display.

17. The non-transitory computer-readable medium of claim 16, where the 2D coordinates for presentation of the augmented reality object are determined by applying an offset to the 2D coordinates that represent the position of the guide marker on the display.

18. The non-transitory computer-readable medium of claim 17, where the offset is determined based on at least one of:

comparing the 3D coordinates associated with placement of the augmented reality object and the 3D coordinates of the intersection point, a distance between the augmented reality object and the horizontal plane, or a height of the target object in the image.

19. The non-transitory computer-readable medium of claim 17, where the offset represents at least one of:

a distance between the augmented reality object and the horizontal plane, or a height of the target object in the image.

20. The non-transitory computer-readable medium of claim 16, where the horizontal plane is designated as a ground plane.