Interactive binocular video display

• US 9,591,349 B2
• Filed: 12/23/2014
• Issued: 03/07/2017
• Est. Priority Date: 12/23/2014
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Assayag, Michel, Taite, Shahar, Wasserblat, Moshe, Rider, Tomer, Pereg, Oren, Sivak, Alexander
• Primary Examiner(s): MOTSINGER, SEAN T
• Application Number:
  US14/581,548
• Publication Number:
  US 20160182940A1
• Time in Patent Office: 805 Days
• US Class Current: 1/1
• CPC Class Codes: H04N 21/25841 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; arrangements for identifying locations of receiving stations in broadcast systems H04H60/51; location of the user terminal in data switching networks H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4524 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; systems specially adapted for using geographical information in broadcast systems H04H60/70; protocols in which the network application is adapted for the location of the user terminal in communication control or processing H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/6131 involving transmission via a mobile phone network wireless downlink channel access H04W74/006
• CPC Class Codes: H04N 21/6181 involving transmission via a mobile phone network arrangements for providing broadcast or conference services to substation in data switching networks in combination with wireless systems H04L12/189; wireless uplink channel access H04W74/004
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program

Abstract

Various systems and methods for providing a repositionable video display on a mobile device, to emulate the effect of user-controlled binoculars, are described herein. In one example, one or more high resolution video sources (such as UltraHD video cameras) obtain video that is wirelessly broadcasted to mobile devices. The mobile device processes the broadcast based on the approximate location of the spectator'"'"'s mobile device, relative to a scene within the field of view of the mobile device. The location of the mobile device may be derived from a combination of network monitoring, camera inputs, object recognition, and the like. Accordingly, the spectator can obtain a virtual magnification of a scene from an external video source displayed on the spectator'"'"'s mobile device.

Claims

1. A mobile computing device to provide a virtual binocular display, the mobile computing device comprising:

a wireless networking transceiver to receive a selected stream of video data from a video streaming server via a wireless network;

a processor to process video content from the stream of the video data, wherein the selected stream of video data is selected based on a location of the mobile computing device in a viewing area located in proximity to a scene captured in the video data;

a display processor to output a select enhanced portion of the video content on a display screen of the mobile computing device, the select enhanced portion determined based on an orientation of the mobile computing device relative to the scene captured in the video data; and

a camera to capture a digital image of the scene;

wherein operations performed by the processor to process the video content from the stream of the video data include operations to;

perform image recognition of at least one object in the digital image of the scene; and

identify the select enhanced portion of the video content based on a location of the at least one object in the select enhanced portion of the high-definition video.

2. The mobile computing device of claim 1, wherein the processor operates to generate a graphical user interface, the graphical user interface to receive user commands to effect control of the output of the select enhanced portion of the video content on the display screen, wherein the select enhanced portion of the video content provides a zoomed and panned portion of the video content from the stream of the video data.

3. The mobile computing device of claim 1, wherein the processor operates to change the select enhanced portion of the video content on the display screen based on at least one motion input detected by the mobile computing device.

4. The mobile computing device of claim 1, wherein the wireless networking transceiver operates to receive the stream of the video data via operation of the wireless network according to an IEEE 802.11 wireless network protocol, wherein the selected stream of the video data is selected based on a location of the mobile computing device relative to at least one access point of the wireless network located within the viewing area, and wherein an identification of the location of the mobile computing device relative to the access point of the wireless network is communicated to the video streaming server to indicate the location of the mobile computing device.

5. The mobile computing device of claim 1, wherein the at least one object includes at least one indicator, and wherein the processor operates to correlate the at least one indicator detected in the digital image of the scene to the orientation of the mobile computing device relative to the scene captured in the video data, based on the location of the at least one object in the select enhanced portion of the high-definition video.

6. The mobile computing device of claim 5, wherein the processor operates to correlate the digital image of at least the portion of the scene including the at least one indicator to the select enhanced portion of the video content.

7. The mobile computing device of claim 6, wherein the processor operates to compare and match the digital image of at least the portion of the scene captured by the camera with an image of the scene captured in the video data, to determine the orientation of the mobile computing device relative to the scene.

8. A method of displaying streaming video at a mobile device, the method comprising operations executing with hardware components at the mobile device, the operations including:

capturing, with the mobile device, a digital image of a scene;

performing an image recognition of at least one object in the digital image of the scene;

accessing a selected high-definition video stream at the mobile device, the high-definition video stream providing high-definition video of the scene captured in a direction of view away from the mobile device and towards an area of interest of the scene;

determining an orientation of the mobile device relative to the area of interest, based on the at least one object;

selecting a particular zoomed portion of the high-definition video based on determining the orientation of the mobile device relative to the area of interest, the particular zoomed portion of the high-definition video providing a reduced area within the high-definition video; and

displaying the particular zoomed portion of the high-definition video of the scene on the mobile device.

9. The method of claim 8, further comprising:

panning the high-definition video of the scene to the particular zoomed portion of the high-definition video, the panning performed using the orientation of the mobile device relative to the area of interest; and

zooming the high-definition video of the scene to the particular zoomed portion of the high-definition video, the zooming performed using the orientation of the mobile device relative to the area of interest.

10. The method of claim 8, further comprising:

providing location information of the mobile device to a remote server, the location information indicating a location of the mobile device in an observation area adjacent to the scene; and

receiving the high-definition video stream from one of a plurality of high-definition video streams, the high-definition video stream selected by the remote server based the location information of the mobile device.

11. The method of claim 8, wherein the at least one object is a marker, and the method further comprising:

detecting, with the mobile device, the marker in the digital image of the scene; and

wherein the particular zoomed portion of the high-definition video is further selected based on a position of the marker detected by the mobile device within the high-definition video of the scene.

12. The method of claim 11, wherein correlating the orientation of the mobile device to the particular zoomed portion of the high-definition video includes matching the marker in the digital image of the scene to a marker in an image of the high-definition video of the scene, to simulate a perspective of the scene from the mobile device in the selecting of the particular zoomed portion of the high-definition video.

13. The method of claim 12, wherein the marker is provided by at least one infrared light positioned in the scene.

14. The method of claim 12, wherein the marker is a machine-readable object positioned in the scene, the machine-readable object providing an identifier associated with a location relative to the scene.

15. The method of claim 8, further comprising:

displaying an information overlay on the particular zoomed portion of the high-definition video, the information overlay providing information about at least one real-world item portrayed in the particular zoomed portion of the high-definition video.

16. The method of claim 8, further comprising:

processing at least one user input received in connection with a graphical user interface output with the mobile device, the at least one user input used to;

select the high-definition video stream from among a plurality of high-definition video streams;

perform panning of the high-definition video of the scene to another area of interest of the scene;

or perform zooming of the high-definition video of the scene to generate a zoomed display of another portion of the area of interest of the scene.

17. At least one machine-readable storage medium including instructions for outputting a video display on a mobile computing device, comprising instructions, that when executed, cause the mobile computing device to:

establish a connection with a wireless network;

capture a digital image of an area of interest from a perspective of a first side of the mobile computing device, using a camera located on the first side of the mobile computing device;

perform an image recognition of at least one object in the digital image;

process a high-definition video stream of the area of interest, the high-definition video stream received via the connection with the wireless network;

determine a perspective of the mobile computing device relative to an orientation of a video scene provided in the high-definition video stream based upon a correlation of the at least one object in the digital image with a representation of the at least one object in the high-definition video stream;

select a portion of the video scene of the high-definition video stream based upon the determined perspective of the mobile computing device; and

output digital video on a display screen located on a second side of the mobile computing device, wherein the second side is opposite of the first side, wherein the output of the digital video includes the portion of the video scene of the high-definition video stream, and wherein the portion of the video scene of the high-definition video stream is zoomed and panned to resemble the perspective of the mobile computing device.

18. The machine-readable storage medium of claim 17, wherein the at least one object is a marker, and wherein the instructions, when executed, further cause the mobile computing device to:

process the digital image of the area of interest captured with the camera;

detect the marker in the image of the area of interest; and

correlate the marker in the area of interest to an orientation of the portion of the video scene.

19. The machine-readable storage medium of claim 18, wherein the marker is a machine-detectable marker, the machine-detectable marker including an infrared light or a machine-detectable object.

20. The machine-readable storage medium of claim 17, wherein the instructions, when executed, further cause the mobile computing device to:

transmit location information of the mobile computing device to a remote video server; and

receive the high-definition video stream from the remote video server in response to the location information, the high-definition video stream selected from a plurality of high-definition video streams.

21. The machine-readable storage medium of claim 20, wherein the location information of the mobile computing device includes wireless network information, the wireless network information including an SSID of a wireless access point for the wireless network.

22. The machine-readable storage medium of claim 20, wherein the location information of the mobile computing device includes an identifier read by the mobile computing device from a QR code, RFID tag, or Bluetooth Low Energy beacon.

23. The machine-readable storage medium of claim 17, wherein the instructions, when executed, further cause the mobile computing device to:

detect and process a user gesture input to the mobile computing device; and

change a displayed portion of the video scene of the high-definition video stream based on the user gesture input.