Visually passing data through video

• US 9,224,322 B2
• Filed: 08/03/2012
• Issued: 12/29/2015
• Est. Priority Date: 08/03/2012
• Status: Active Grant

• Current Assignee: APX Labs, Inc.
• Original Assignee: APX Labs, Inc.
• Inventors: Ballard, Brian, Martellaro, John A., Jenkins, Jeffrey E.
• Primary Examiner(s): Johnson, M Good
• Application Number:
  US13/566,573
• Publication Number:
  US 20140035951A1
• Time in Patent Office: 1,243 Days
• Field of Search: 345/633
• US Class Current: 1/1
• CPC Class Codes: G09G 2360/18 Use of a frame buffer in a display terminal, inclusive of the display panel
• CPC Class Codes: G09G 3/003 to produce spatial visual effects

Abstract

A method and a system involve the insertion of digital data into a number of video frames of a video stream, such that the video frames contain both video content and the inserted digital data. The video, including the inserted digital data is then visually conveyed to and received by an augmented reality device without the use of a network connection. In the augmented reality device, the digital data is detected, processed and used to provide computer-generated data and/or information. The computer-generated data and/or information is then presented on a display associated with the augmented reality device or otherwise reproduced through the augmented reality device, where the computer-generated data and/or information supplements the video content so as to enhance the viewing experience of the augmented reality device user.

Claims

1. A method of conveying digital data to an augmented reality device through external video, the method comprising:

encoding digital data into each of a plurality of video frames associated with the external video, such that each of the plurality of video frames includes both viewable video content and the encoded digital data;

displaying the external video externally from the augmented reality device including each of the plurality of video frames such that the external video including each of the plurality of video frames is available to be visually sensed and captured by the augmented reality device,

wherein the digital data represents data and/or information that is included with the viewable video content and visually detectable by the augmented reality device, and

wherein the digital data is encoded within the external video such that a user with their naked eye cannot detect it.

2. The method of claim 1, wherein encoding digital data into each of a plurality of video frames associated with the external video comprises:

integrating the digital data with the viewable video content.

3. The method of claim 1, wherein encoding digital data into each of a plurality of video frames associated with the external video comprises:

encoding the digital data as one or more codes into one or more peripheral portions of each of the plurality of video frames.

4. The method of claim 3, wherein encoding the digital data as one or more codes into one or more peripheral portions of each of the plurality of video frames comprises:

encoding the digital data as one or more QR codes.

5. The method of claim 3, wherein encoding the digital data as one or more codes into one or more peripheral portions of each of the plurality of video frames comprises:

encoding the digital data as one or more bar codes.

6. The method of claim 3, wherein encoding the digital data as one or more codes into one or more peripheral portions of each of the plurality of video frames comprises:

inserting the digital data as one or more block codes.

7. The method of claim 1, wherein encoding digital data into each of a plurality of video frames associated with the video comprises:

encoding digital data into each and every video frame associated with the external video.

8. The method of claim 1 further comprising:

encoding into at least one video frame associated with the external video, a first marker indicating the presence of the digital data in the plurality of video frames.

9. The method of claim 8 further comprising:

encoding into at least one video frame associated with the external video, a second marker, wherein the first marker further indicates a first one of the plurality of video frames containing the digital data, and wherein the second marker indicates a last one of the plurality of video frames containing the digital data.

10. The method of claim 1, wherein inserting digital data into each of a plurality of video frames associated with the external video comprises:

encoding different digital data into each of the plurality of video frames.

11. A method of receiving digital data in an augmented reality device through external video, the method comprising:

visually capturing a plurality of video frames of the external video, wherein each of the plurality of video frames includes viewable video content displayed externally from the augmented reality device and digital data that has been encoded therein and visually detectable by the augmented reality device;

processing the digital data encoded into each of the plurality of visually captured video frames and generating therefrom data and/or information that is included with the viewable video content; and

presenting, through the augmented reality device, the data and/or information that is included with the viewable video content,

wherein the digital data is encoded within the external video such that a user with their naked eye cannot detect it.

12. The method of claim 11 further comprising:

detecting the digital data in each of the plurality of video frames; and

storing the digital data in memory.

13. The method of claim 11, wherein the plurality of video frames containing the digital data is less than all of the video frames associated with the external video, the method further comprising:

capturing video frames that contain the digital data and capturing video frames that do not contain the digital data; and

determining which video frames contain the digital data based on a first predefined marker indicating the presence of the digital data.

14. The method of claim 13, wherein determining which video frames contain the digital data based on a second predefined marker, the first marker indicating a first video frame containing the digital data and the second marker indicating a last video frame containing data.

15. The method of claim 13, wherein each of the plurality of video frames containing digital data include a marker indicating the presence of the digital data.

16. The method of claim 11, wherein presenting, through the augmented reality device, the data and/or information that is included with the viewable video content comprises:

rendering the data and/or information that is included with the viewable video content on a display of the augmented reality device.

17. The method of claim 16, wherein the data and/or information is text.

18. The method of claim 16, wherein the data and/or information is graphics.

19. The method of claim 16, wherein the data and/or information is animation.

20. The method of claim 16, wherein the data and/or information is additional video.

21. The method of claim 11, wherein presenting, through the augmented reality device, the data and/or information that is included with the viewable video content comprises:

reproducing sound through a sound reproduction component of the augmented reality device.

22. The method of claim 11, wherein presenting, through the augmented reality device, the data and/or information that is included with the viewable video content comprises:

downloading the data/information into the augmented reality device over a network connection.

23. An augmented reality device comprising:

a video sensor configured to visually capture external video, wherein the external video comprises a plurality of video frames, each including viewable video content displayed externally from the augmented reality device and digital data encoded therein that is visually detectable by the augmented reality device;

a visual processor configured to process the digital data that was encoded into each of the plurality of captured video frames and to generate therefrom data and/or information; and

a rendering module configured to present, through the augmented reality device, the data and/or information,

wherein the digital data is encoded within the video such that a user with their naked eye cannot detect it.

24. The augmented reality device of claim 23, wherein the video sensor is a camera.

25. The augmented reality device of claim 23, wherein the plurality of video frames containing the digital data is less than all of the video frames associated with the external video, and wherein the visual processor is further configured to capture video frames that contain the digital data, capture video frames that do not contain the digital data, and determine which video frames contain the digital data and which video frames do not contain the digital data based on a predefined marker indicating the presence of the digital data.

26. The augmented reality device of claim 25, wherein the visual processor is further configured to determine which video frames contain the digital data and which video frames do not contain the digital data based on a second predefined marker, the predefined marker indicating a first video frame containing the digital data and the second predefined marker indicating a last video frame containing data.

27. The augmented reality device of claim 25, wherein the visual processor is further configured to determine which video frames contain the digital data and which video frames do not contain digital data by detecting the predefined marker in each of the plurality of video frames that contain the digital data.

28. The augmented reality device of claim 23 further comprising a display, and wherein the rendering module is further configured to render the data and/or information on the display of the augmented reality device.

29. The augmented reality device of claim 28, wherein the data and/or information is text.

30. The augmented reality device of claim 28, wherein the data and/or information is graphics.

31. The augmented reality device of claim 28, wherein the data and/or information is animation.

32. The augmented reality device of claim 28, wherein the data and/or information is additional video.

33. The augmented reality device of claim 23 further comprising a sound reproduction component, wherein the rendering module is further configured to reproduce sound through the sound reproduction component.

34. The augmented reality device of claim 23 further comprising a network services interaction module configured to provide a network connection for the augmented reality device, over which, the data and/or information that is included with the viewable video content is downloaded.