SYSTEMS AND METHODS FOR CONTEXTUALLY AUGMENTED VIDEO CREATION AND SHARING

• US 20160180590A1
• Filed: 12/23/2014
• Published: 06/23/2016
• Est. Priority Date: 12/23/2014
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: KAMHI, GILA, HURWITZ, BARAK, COOPERMAN, VLADIMIR, NISTEL, KOBI
• Granted Patent: US 9,754,416 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

An augmented reality (AR) device includes a 3D video camera to capture video images and corresponding depth information, a display device to display the video data, and an AR module to add a virtual 3D model to the displayed video data. A depth mapping module generates a 3D map based on the depth information, a dynamic scene recognition and tracking module processes the video images and the 3D map to detect and track a target object within a field of view of the 3D video camera, and an augmented video rendering module renders an augmented video of the virtual 3D model dynamically interacting with the target object. The augmented video is displayed on the display device in real time. The AR device may further include a context module to select the virtual 3D model based on context data comprising a current location of the augmented reality device.

Claims

1. An augmented reality device, comprising:

a three-dimensional (3D) video camera to capture video data comprising video images and corresponding depth information;

a display device to display the video data; and

an augmented reality module to add a virtual 3D model to the displayed video data, the augmented reality module comprising;

a depth mapping module to generate a 3D map based on the depth information;

a dynamic scene recognition and tracking module to process the video images and the 3D map to detect and track a target object within a field of view of the 3D video camera; and

an augmented video rendering module to render an augmented video of the virtual 3D model dynamically interacting with the target object, and to provide the augmented video for display on the display device.

2. The augmented reality device of claim 1, further comprising a context module to select the virtual 3D model based on context data comprising a current location of the augmented reality device, wherein the selected virtual 3D model is associated with the current location.

3. The augmented reality device of claim 2, wherein the context data further comprises user profile information including one or more data field comprising birthdate, anniversary, animal preference, historic person preference, current celebrity preference, and genre preference.

4. The augmented reality device of claim 3, wherein the context module is further configured to select the virtual 3D model based on the user profile information.

5. The augmented reality device of claim 2, wherein the context module is further configured to select the virtual 3D model based on a mood indicator selected by a user.

6. The augmented reality device of claim 1, wherein the virtual 3D model comprises:

context data to associate the virtual 3D model with at least one of a location, person, and event; and

target object data associated with the context data, the target object data to define a specific object or object type for the dynamic scene recognition and tracking module to detect and track as the target object.

7. The augmented reality device of claim 6, wherein the virtual 3D model further comprises:

one or more graphic objects to animate in the rendered augmented video; and

interaction rules to define an interaction between the one or more graphic objects and the target object in the rendered augmented video.

8. The augmented reality device of claim 1, further comprising:

a communication module to communicate through a communication network; and

a social media module to, automatically or in response to user input, provide the augmented video through the communication network to a video-sharing service or social media service.

9. A method for rendering augmented video, the method comprising:

capturing, by a mobile device, video data comprising video images and corresponding depth information;

generating a three-dimensional (3D) map based on the depth information;

detecting and tracking a target object in the video images;

rendering, based on the 3D map, an augmented video comprising a virtual 3D model dynamically interacting with the target object; and

displaying the augmented video in real time on a display device of the mobile device.

10. The method of claim 9, further comprising:

determining context information including a location of the mobile device; and

selecting the virtual 3D model based on the context information.

11. The method of claim 10, wherein the context information further includes a user profile comprising one or more data field including birthdate, anniversary, animal preference, historic person preference, current celebrity preference, and genre preference.

12. The method of claim 11, further comprising selecting the virtual 3D model based on the user profile.

13. The method of claim 11, wherein the context information further includes an indication of a user'"'"'s mood, and wherein the method further comprises selecting the virtual 3D model based on the indication of the user'"'"'s mood.

14. The method of claim 9, wherein the virtual 3D model comprises:

context data to associate the virtual 3D model with at least one of a location, person, and event; and

target object data associated with the context data, the target object data to define a specific object or object type for the dynamic scene recognition and tracking module to detect and track as the target object.

15. The method of claim 14, wherein the virtual 3D model further comprises:

one or more graphic objects to animate in the rendered augmented video; and

interaction rules to define an interaction between the one or more graphic objects and the target object in the rendered augmented video.

16. The method of claim 9, wherein the augmented video comprises an augmented micro video, and wherein the method further comprises automatically communicating the augmented micro video through a communication network to at least one of a video-sharing service and a social media service.

17. At least one computer-readable storage medium having stored thereon instructions that, when executed by a processor, cause the processor to perform operations comprising:

capturing, by an augmented reality device, video data comprising video images and corresponding depth information;

generating a three-dimensional (3D) map based on the depth information;

detecting and tracking a target object in the video images;

rendering, based on the 3D map, an augmented video comprising a virtual 3D model dynamically interacting with the target object; and

displaying the augmented video in real time on a display device of the augmented reality device.

18. The least one computer-readable storage medium of claim 17, the operations further comprising:

determining context information including a location of the augmented reality device; and

selecting the virtual 3D model based on the context information.

19. The least one computer-readable storage medium of claim 18, wherein the context information further includes a user profile comprising one or more data field including birthdate, anniversary, animal preference, historic person preference, current celebrity preference, and genre preference.

20. The least one computer-readable storage medium of claim 19, and wherein the operations further comprise selecting the virtual 3D model based on the user profile.

21. The least one computer-readable storage medium of claim 19, wherein the context information further includes an indication of a user'"'"'s mood, and wherein the operations further comprise selecting the virtual 3D model based on the indication of the user'"'"'s mood.

22. The least one computer-readable storage medium of claim 17, wherein the virtual 3D model comprises:

context data to associate the virtual 3D model with at least one of a location, person, and event; and

target object data associated with the context data, the target object data to define a specific object or object type for the dynamic scene recognition and tracking module to detect and track as the target object.

23. The least one computer-readable storage medium of claim 22, wherein the virtual 3D model further comprises:

one or more graphic objects to animate in the rendered augmented video; and

interaction rules to define an interaction between the one or more graphic objects and the target object in the rendered augmented video.

24. The least one computer-readable storage medium of claim 17, wherein the augmented video comprises an augmented micro video, and wherein the operations further comprise automatically communicating the augmented micro video through a communication network to at least one of a video-sharing service and a social media service.