Scene modification for augmented reality using markers with parameters

• US 10,026,228 B2
• Filed: 02/25/2015
• Issued: 07/17/2018
• Est. Priority Date: 02/25/2015
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Yuen, Kathy, Anderson, Glen J.
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): Li, Grace Q
• Application Number:
  US14/631,332
• Publication Number:
  US 20160247320A1
• Time in Patent Office: 1,238 Days
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 2219/2021 Shape modification
• CPC Class Codes: G11B 27/00 Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

Scene modification is described for augmented reality using markers with parameters. In one example, a method includes capturing a scene by a camera, the scene having a marker, analyzing the captured scene to identify the marker, determining a location of the marker in the captured scene, determining an augmented reality parameter associated with the identified marker, modifying the captured scene at the marker location based on the augmented reality parameter, and rendering the modified scene.

Claims

1. A method of modifying an image comprising:

capturing an image of a scene by a camera, the scene having a surface and one or more physical markers on the surface;

analyzing the captured scene to identify the physical marker;

determining a location of the physical marker in the captured image;

reading an observable pattern on the physical marker;

using the read pattern to determine an augmented reality parameter associated with the pattern on the identified physical marker, wherein the parameter modifies an appearance of the surface;

modifying the image of the surface at the physical marker location based on the augmented reality parameter, further comprising modifying the image by superimposing a video over the image, wherein the scene has a video frame marker, wherein a parameter associated with the video frame marker is a time frame of a video sequence, and wherein superimposing the video over the image comprises aligning a subject of the video sequence at the time frame of the parameter with a location of the video frame marker in the captured image; and

rendering the modified image.

2. The method of claim 1, wherein the augmented reality parameter indicates at least one of a texture, color and topography, and wherein modifying comprises applying the respective texture, color, or topography to the surface.

3. The method of claim 1, wherein the augmented reality parameter is an elevation and wherein modifying comprises applying the elevation to the image of the surface at the physical marker location.

4. The method of claim 1, wherein the scene has a plurality of boundary markers, the method further comprising:

identifying the boundary markers;

reading respective observable patterns on the boundary markers;

using the read boundary marker patterns to determine that the boundary markers identify boundary locations; and

wherein modifying comprises applying a video sequence to the image, and positioning and scaling the video sequence so that the video is within a boundary as determined by the locations of the boundary markers.

5. The method of claim 1, wherein the surface is a printed two-dimensional image, wherein the augmented reality parameter is a three-dimensional topography of the image and wherein modifying further comprises applying three-dimensional topography to the two-dimensional printed image surface in the image at the physical marker location.

6. The method of claim 5, wherein moving a three dimensional position comprises moving an edge of the image closer to a viewing position in a 3D space.

7. The method of claim 5, further comprising cropping the image after moving the captured scene in the 3D space.

8. The method of claim 1 wherein the scene has a further effect marker, the method comprising:

identifying the effect marker;

reading an observable pattern on the effect marker;

using the read effect marker pattern to determine that the parameter associated with the effect marker comprises a non-visual augmented reality effect, and wherein modifying comprises applying the effect to the image at the location of the effect marker.

9. The method of claim 1, wherein the augmented reality parameter is a texture and wherein modifying further comprises applying the texture to the entire surface in the image.

10. The method of claim 1, wherein the scene further has path markers, the method comprising:

analyzing the captured scene to identify the path markers;

determining locations of the path markers in the captured image;

generating a sequence of images to show a character traversing a path in the scene defined by the path markers.

11. The method of claim 1, wherein the surface is a table, wherein the parameters associated with the patterns on the physical markers modify elevations and textures of the appearance of the image of the surface, the scene further having path markers.

12. The method of claim 1 wherein the pattern is a printed quick response code.

13. The method of claim 1, wherein the pattern is an appearance of a doll.

14. An apparatus comprising:

a memory to store an image of a scene captured by a camera, the scene having a surface and one or more physical markers on the surface, each physical marker having an observable pattern, and the memory to store a parameter associated with each of the observable patterns, wherein the parameter modifies an appearance of the surface; and

a processor having a rendering module to identify the physical marker in the image, to read the respective observable pattern on the marker, to retrieve the stored parameter associated with the pattern, and to modify the image of the surface based on the retrieved parameter, further comprising the processor to modify the image by superimposing a video over the image, wherein the scene has a video frame marker, wherein a parameter associated with the video frame marker is a time frame of a video sequence, and wherein superimposing the video over the image comprises the processor aligning a subject of the video sequence at the time frame of the parameter with a location of the video frame marker in the captured image.

15. The apparatus of claim 14, wherein the memory further stores a second image of a scene captured by a second camera and wherein the processor further comprises a coordination module to combine the first and the second images before being modified by the rendering module.

16. The apparatus of claim 15, wherein the second image is part of a video sequence and wherein the coordination module applies the video sequence to the first image using the first image as a backdrop.

17. A non-transitory machine-readable medium having instructions that when operated on by the machine cause the machine to perform operations comprising:

capturing an image of a scene by a camera, the scene having a surface and one or more physical markers on the surface;

analyzing the captured scene to identify the physical marker;

determining a location of the physical marker in the captured image;

reading an observable pattern on the physical marker;

using the read pattern to determine an augmented reality parameter associated with the pattern on the identified physical marker, wherein the parameter modifies an appearance of the surface;

modifying the image of the surface at the physical marker location based on the augmented reality parameter, further comprising modifying the image by superimposing a video over the image, wherein the scene has a video frame marker, wherein a parameter associated with the video frame marker is a time frame of a video sequence, and wherein superimposing the video over the image comprises aligning a subject of the video sequence at the time frame of the parameter with a location of the video frame marker in the captured image; and

rendering the modified image.