Systems and methods for contextually augmented video creation and sharing

• US 9,754,416 B2
• Filed: 12/23/2014
• Issued: 09/05/2017
• Est. Priority Date: 12/23/2014
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Kamhi, Gila, Hurwitz, Barak, Kouperman, Vladimir, Nistel, Kobi
• Primary Examiner(s): Wu, Xiao
• Assistant Examiner(s): ELBINGER, STEVEN Z
• Application Number:
  US14/581,468
• Publication Number:
  US 20160180590A1
• Time in Patent Office: 987 Days
• Field of Search: None
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

An augmented reality (AR) device includes a 3D video camera to capture video images and corresponding depth information, a display device to display the video data, and an AR module to add a virtual 3D model to the displayed video data. A depth mapping module generates a 3D map based on the depth information, a dynamic scene recognition and tracking module processes the video images and the 3D map to detect and track a target object within a field of view of the 3D video camera, and an augmented video rendering module renders an augmented video of the virtual 3D model dynamically interacting with the target object. The augmented video is displayed on the display device in real time. The AR device may further include a context module to select the virtual 3D model based on context data comprising a current location of the augmented reality device.

Claims

1. An augmented reality device, comprising:

a three-dimensional (3D) video camera to capture video data comprising video images and corresponding depth information;

a display device to display the video data; and

an augmented reality module to add a virtual 3D model to the displayed video data captured by the 3D video camera, the augmented reality module comprising instructions to;

generate a 3D map based on the depth information;

process the video images and the 3D map to detect and track a target object within a field of view of the 3D video camera;

select the virtual 3D model based on a mood of a user and context data comprising a current geographic location of the augmented reality device, wherein the selected virtual 3D model is associated with the current geographic location; and

render an augmented video of the virtual 3D model dynamically interacting with the target object based on the mood of the user, and provide the augmented video for display on the display device.

2. The augmented reality device of claim 1, wherein the context data further comprises user profile information including one or more data fields comprising birthdate, anniversary, animal preference, historic person preference, current celebrity preference, and genre preference.

3. The augmented reality device of claim 2, wherein the augmented reality module further comprises instructions to select the virtual 3D model based on the user profile information.

4. The augmented reality device of claim 1, wherein the virtual 3D model comprises:

context data to associate the virtual 3D model with at least one of a location, person, and event; and

target object data associated with the context data, the target object data to define a specific object or object type for the augmented reality module to detect and track as the target object.

5. The augmented reality device of claim 4, wherein the virtual 3D model further comprises:

one or more graphic objects to animate in the rendered augmented video; and

interaction rules to define an interaction between the one or more graphic objects and the target object in the rendered augmented video.

6. The augmented reality device of claim 1, wherein:

the augmented reality module further comprises instructions to communicate through a communication network; and

the augmented reality module further comprises instructions to, automatically or in response to user input, provide the augmented video through the communication network to a video-sharing service or social media service.

7. A method for rendering augmented video, the method comprising:

capturing, by a mobile device, video data comprising video images and corresponding depth information;

generating a three-dimensional (3D) map based on the depth information;

detecting and tracking a target object in the video images;

determining context information including a location of the mobile device; and

selecting a virtual 3D model based on a mood of the user and the context information, wherein the location of the mobile device is provided by an external source;

rendering, based on the 3D map, an augmented video comprising the virtual 3D model dynamically interacting with the target object based on the mood of the user; and

displaying the augmented video in real time on a display device of the mobile device.

8. The method of claim 7, wherein the context information further includes a user profile comprising one or more data fields including birthdate, anniversary, animal preference, historic person preference, current celebrity preference, and genre preference.

9. The method of claim 8, further comprising selecting the virtual 3D model based on the user profile.

10. The method of claim 7, wherein the virtual 3D model comprises:

context data to associate the virtual 3D model with at least one of a location, person, and event; and

target object data associated with the context data, the target object data to define a specific object or object type to be detected and tracked as the target object.

11. The method of claim 10, wherein the virtual 3D model further comprises:

one or more graphic objects to animate in the rendered augmented video; and

interaction rules to define an interaction between the one or more graphic objects and the target object in the rendered augmented video.

12. The method of claim 7, wherein the augmented video comprises an augmented micro video, and wherein the method further comprises automatically communicating the augmented micro video through a communication network to at least one of a video-sharing service and a social media service.

13. At least one non-transitory computer-readable storage medium having stored thereon instructions that, when executed by a processor, cause the processor to perform operations comprising:

capturing, by an augmented reality device, video data comprising video images and corresponding depth information;

generating a three-dimensional (3D) map based on the depth information;

detecting and tracking a target object in the video images;

determining context information including a location of the augmented reality device;

selecting a virtual 3D model based on a mood of a user and the context information;

rendering, based on the 3D map, an augmented video comprising the virtual 3D model dynamically interacting with the target object based on the mood of the user; and

displaying the augmented video in real time on a display device of the augmented reality device.

14. The at least one non-transitory computer-readable storage medium of claim 13, wherein the context information further includes a user profile comprising one or more data fields including birthdate, anniversary, animal preference, historic person preference, current celebrity preference, and genre preference.

15. The at least one non-transitory computer-readable storage medium of claim 14, wherein the operations further comprise selecting the virtual 3D model based on the user profile.

16. The at least one non-transitory computer-readable storage medium of claim 13, wherein the virtual 3D model comprises:

context data to associate the virtual 3D model with at least one of a location, person, and event; and

target object data associated with the context data, the target object data to define a specific object or object type to be detected and tracked as the target object.

17. The at least one non-transitory computer-readable storage medium of claim 16, wherein the virtual 3D model further comprises:

one or more graphic objects to animate in the rendered augmented video; and

interaction rules to define an interaction between the one or more graphic objects and the target object in the rendered augmented video.

18. The at least one non-transitory computer-readable storage medium of claim 13, wherein the augmented video comprises an augmented micro video, and wherein the operations further comprise automatically communicating the augmented micro video through a communication network to at least one of a video-sharing service and a social media service.