Augmented reality system

• US 9,652,046 B2
• Filed: 07/24/2015
• Issued: 05/16/2017
• Est. Priority Date: 01/06/2011
• Status: Expired due to Fees

• Current Assignee: David Elmekies
• Original Assignee: David Elmekies
• Inventors: Elmekies, David
• Primary Examiner(s): Hannett, James
• Application Number:
  US14/808,715
• Publication Number:
  US 20150332515A1
• Time in Patent Office: 662 Days
• Field of Search: None
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04W 4/026 using orientation information, e.g. compass

Abstract

The present invention is directed to a video display and user control for the video display. In the preferred embodiment, multiple video feeds are combined into a single video display, where one feed, preferably live, includes an object which is used by a computer to select a second and display a second feed, wherein the location and orientation of the second feed changes based on the location and orientation of the object in the first feed. At least a portion of the display gives the appearance of depth, such as through use of superimposed animation. The live image is controlled by the user'"'"'s movement, causing at least a part of the display to move in concert with the user'"'"'s motion, thereby creating an enhanced three dimensional effect for a superimposed display.

Claims

1. A method for a processor-controlled video display to concurrently display a live video feed in combination with a processor-identified replacement entry, said entry replacing an object in said live video feed and including an augmented reality effect, comprising the steps of:

receiving an on-going live feed of content in a field of vision for display and delivering said content to said processor;

said processor recognizing an object and its boundaries, based at least in part on analyzing said live field and distinguishing colors, its movement and orientation in said field of vision;

obtaining an entry corresponding to said object together with display instructions from a data library, said entry representative of a 3D replacement for said object in said field of vision;

masking said object in said display and mapping said entry in its stead thereby creating a combined feed, and delivering said combined feed to a video display for presentation; and

receiving input from a tracker, said tracker tracking movement and orientation of said object in said field of vision, and said processor synchronizing position, movement, and orientation of said entry harmoniously with tracked changes in position, movement, and orientation of said object in said field of vision.

2. The method of claim 1, wherein said combined feed appears on a mobile device.

3. The method of claim 1, wherein the augmented reality effect is adjusted based on the changing position or orientation of the object.

4. The method of claim 1, wherein the movement of the object in the live video feed is controlled by a viewer.

5. The method of claim 1, wherein said processor-identified entry displays as at least one of a moving or rotating image.

6. The method of claim 1, wherein said display is capturable as a still image or video footage by a user.

7. A method for a processor-controlled video display to display merged live and computer-generated content comprising the steps of:

receiving a live video feed including an image of an object from a video camera;

recognizing said object and its boundaries in said video feed at least in part by analyzing color patterns in said video feed;

tracking any movement, rotation, or change in position or orientation of said object in said feed;

said processor forming a merged feed for display by masking said object in said video feed and mapping in its stead computer-generated augmented reality appearing content, said content and display instructions retrieved from a data store; and

delivering said merged feed to said video display for display;

wherein the movement, position, and orientation of said content is harmoniously adjusted in said merged feed based on any changing position and orientation of said object in said live feed.

8. The method of claim 7, wherein said merged feed is capturable as a still image or as video footage by a user.

9. The method of claim 7, wherein said video display is on a mobile device.

10. The method of claim 7, wherein said augmented reality effect is adjusted based on the changing position or orientation of the object in said live feed.

11. The method of claim 7, wherein the movement of said object in the live video feed is controlled by a viewer.

12. The method of claim 7, wherein said content displays as at least one of a moving or rotating image.

13. A method for a processor to direct a video display to display a live video feed with replacement content in the position of an element in said live feed comprising the steps of:

obtaining a live video feed from a video camera;

recognizing a element and its position and boundaries and linear and rotational movement in said live feed;

tracking movement of said element'"'"'s movement, rotation, position and orientation;

obtaining an entry associated with said element from a data store, wherein said entry represents 3D content for a replacement for said element including an augmented reality effect; and

directing a video display to display said live feed masking said element with said replacement mapped in the position of said element, thereby creating a combined feed;

wherein movement, rotation, position, and orientation of said replacement are harmoniously synchronized to any changing movement, rotation, position, and orientation of said element in said live feed and said element and its boundaries are identified based at least in part on color identification.

14. The method of claim 13, wherein said combined feed appears on a mobile device.

15. The method of claim 13, wherein said augmented reality effect is adjusted based on the changing position or orientation of said element.

16. The method of claim 13, wherein said augmented reality effect further includes a 3D effect.

17. The method of claim 13, wherein said replacement displays as at least one of a moving or rotating image.

18. The method of claim 13, wherein said display is capturable as a still image or video footage by a user.