System and method for providing improved augmented reality content

• US 10,665,023 B2
• Filed: 12/26/2017
• Issued: 05/26/2020
• Est. Priority Date: 12/26/2017
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Patel, Mehul, Chapman, Steven M., Havey, Benjamin F., Popp, Joseph
• Primary Examiner(s): Hoang, Peter
• Application Number:
  US15/854,593
• Publication Number:
  US 20190197781A1
• Time in Patent Office: 882 Days
• Field of Search: None
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10148 Varying focus
• CPC Class Codes: G06T 2210/36 Level of detail
• CPC Class Codes: G06T 7/571 from focus

Abstract

Systems, devices, and methods are disclosed for generating improved AR content. An electronic device includes circuitry coupled to a memory storing instructions that, when executed, cause the circuitry to obtain frame data for a frame captured using a camera. The frame data includes a level of focus for one or more frame objects in the frame. The circuitry is caused to associate an augmented reality object with at least one of the one or more frame objects. The circuitry is caused to determine a fit factor between a level of focus of the augmented reality object and level of focus of the at least one of the frame objects associated with the augmented reality object. Additionally, the circuitry is caused to, if the fit factor does not satisfy a threshold, apply a decrease to the level of focus of the augmented reality object in order to generate an increased fit factor.

Claims

1. An electronic device for generating improved augmented reality content, the electronic device comprising:

a camera configured to capture a first frame;

an augmented reality component coupled to the camera and a memory, wherein the memory stores instructions that, when executed, cause the augmented reality component to;

obtain frame data for the first frame, the frame data comprising a three-dimensional marker associated with a location in the first frame;

using the three-dimensional marker, determine an expected perceived depth of a frame object in the first frame;

insert an augmented reality object into the first frame at the location, wherein a focus level of the augmented reality object is set to a default depth value;

based on the location in the first frame, group the augmented reality object with the frame object in the first frame;

modify the focus level of the augmented reality object to an approximate perceived depth, wherein the approximate perceived depth of the augmented reality object corresponds with the expected perceived depth of the frame object in the first frame; and

while the camera is capturing a second frame, display the first frame and the augmented reality object to increase a fit between the focus level of the augmented reality object and the frame object are grouped with the augmented reality object.

2. The electronic device of claim 1, wherein the augmented reality object is displayed with the fit increased while the camera is capturing the second frame.

3. The electronic device of claim 1, wherein the approximate perceived depth of the augmented reality object is set to be within a range of values.

4. The electronic device of claim 1, wherein the augmented reality object is grouped with the frame object based on an estimated proximity of the augmented reality object to the frame object.

5. The electronic device of claim 1, wherein the first frame comprises a plurality of pixels, and wherein each of the plurality of pixels is associated with an individual focus level.

6. The electronic device of claim 1, wherein the frame data comprises a three-dimensional mesh of the first frame captured using a plurality of depth sensors.

7. The electronic device of claim 1, wherein the augmented reality component is further caused to:

scan data associated with the first frame, using computer vision; and

generate a three-dimensional model of the first frame, wherein the three-dimensional model of the first frame includes the three-dimensional marker.

8. A method for generating improved augmented reality content, the method comprising:

generating a focus map using frame data for a first frame, the frame data comprising a frame object associated with a location in the first frame;

using the focus map, determining an expected perceived depth of the frame object in the first frame;

inserting an augmented reality object into the first frame at the location, wherein a focus level of the augmented reality object is set to a default depth value;

based on the location in the first frame, grouping the augmented reality object with the frame object in the first frame;

modifying the focus level of focus of the augmented reality object to an approximate perceived depth, wherein the approximate perceived depth of the augmented reality object corresponds with the expected perceived depth of the frame object in the first frame; and

while a second frame is captured, displaying the first frame and the augmented reality object to cause a modification to the focus level of the augmented reality object.

9. The method of claim 8, wherein modifying the focus level of the augmented reality object increases a match between the expected perceived depth of the frame object and the approximate perceived depth of the augmented reality object.

10. The method of claim 8, wherein the focus level of focus of the augmented reality object converges within a range defined between a first threshold focus level and a second threshold focus level.

11. The method of claim 8, wherein grouping the augmented reality object with the frame object in the first frame comprises using an estimated proximity of the augmented reality object to the frame object in the first frame.

12. The method of claim 8, wherein grouping the augmented reality object with the frame object comprises using a first estimated depth for the augmented reality object and a second estimated depth for the frame object.

13. The method of claim 12, further comprising grouping the augmented reality object with a first frame object or a second frame object in addition to the frame object in the first frame.

14. The method of claim 8, wherein the focus level of the augmented reality object is determined using interpolation.

15. The method of claim 8, wherein the focus level of the augmented reality object is determined using extrapolation.

16. An electronic device for generating improved augmented reality content, the electronic device comprising circuitry coupled to a memory, wherein the memory stores instructions that, when executed, cause the circuitry to:

obtain frame data for a first frame, the frame data comprising a three-dimensional marker associated with a location in the first frame;

using the three-dimensional marker, determine an expected perceived depth of a frame object in the first frame;

insert an augmented reality object into the first frame at the location, wherein a focus level of the augmented reality object is set to a default depth value;

based on the location in the first frame, associate the augmented reality object with the frame object in the first frame;

determine a fit factor between the focus level of the augmented reality object and the expected perceived depth of the frame object in the first frame;

responsive to the fit factor not satisfying a threshold, modify the focus level of the augmented reality object to an approximate perceived depth, wherein the approximate perceived depth of the augmented reality object corresponds with the expected perceived depth of the frame object in the first frame; and

generate a second fit factor between the modified focus level of the augmented reality object and the approximate perceived depth.

17. The electronic device of claim 16, wherein the memory further stores instructions that, when executed, cause the circuitry to:

responsive to modification of the focus level of the augmented reality object, determine whether the second fit factor satisfies the threshold; and

responsive to the second fit factor not satisfying the threshold, apply a decrease to the modified focus level of the augmented reality object.

18. The electronic device of claim 16, wherein a degree of modification of the focus level of the augmented reality object is determined using interpolation.