SYCHRONIZED DISPLAY OF STREET VIEW MAP AND VIDEO STREAM

• US 20160300386A1
• Filed: 04/13/2015
• Published: 10/13/2016
• Est. Priority Date: 04/13/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bostick, James E., Ganci, John M. Jr., Rakshit, Sarbajit K., Trim, Craig M.
• Granted Patent: US 10,083,532 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/30 on the same track as the main recording

Abstract

Approaches presented herein enable displaying of a street view or map in the context of a video stream object. Specifically, captured video is analyzed for a plurality of geo-coordinate and direction metadata associated with frames of the video. The video may also be analyzed for visual indicia of location or direction. A user watching the video selects an object therein, which may be then identified by its coordinates and labeled for the user. A map or street view corresponding to the selected object is shown synchronized to the video (e.g., in a side-by-side view), based on the geo-coordinate and direction metadata associated with the video frames. The synchronized video and map speed may be manipulated. The video may also be broken up by frames, and the user permitted to navigate beyond the bounds of the video, where frames are played when the user navigates to areas within the video.

Claims

1. A computer-implemented method for displaying location imagery in context with a video stream, the method comprising:

extracting locational metadata from a set of video data, the video data comprising a plurality of frames;

identifying a set of geographic locations associated with the plurality of frames based on the locational metadata;

generating a set of location imagery associated with the set of geographic locations, the location imagery comprising a set of map features;

displaying the set of location imagery with the plurality of frames; and

synchronizing the set of location imagery to the plurality of frames based on the set of geographic locations, the synching comprising;

playing a set of frames of the plurality of frames; and

displaying, in time with the playing, the generated location imagery associated with the geographic location of each of the played set of frames.

2. The method of claim 1, the method further comprising:

receiving a selection by a user of an object within the video;

identifying the object based on at least one of;

video frame, geographic location, and visual indicia;

searching for descriptive information associated with the object; and

providing a description of the object in the location imagery based on the descriptive information.

3. The method of claim 1, the method comprising:

receiving, from a user, a navigation instruction in the location imagery; and

synchronizing a set of frames of the plurality of frames to the navigation of the location imagery.

4. The method of claim 3, wherein the plurality of frames are displayed in an order based on the navigation received from the user, and wherein the synchronizing pauses when a navigation of the location imagery is received from the user to a portion of location imagery that does not have an associated frame.

5. The method of claim 1, wherein the geographic locations are identified by geographic coordinate metadata and direction metadata of the video.

6. The method of claim 1, wherein the location imagery is at least one of:

a map view, a street view, an aerial view, a helicopter view, and a satellite view.

7. The method of claim 1, wherein the location imagery is displayed with the plurality of frames as at least one of:

a side-by-side partition, an overlay, and a pop-out window.

8. A system for displaying location imagery in context with a video stream, the computer system comprising:

a memory medium comprising program instructions;

a bus coupled to the memory medium; and

a processor for executing the program instructions, the instructions causing the system to;

extract locational metadata from a set of video data, the video data comprising a plurality of frames;

identify a set of geographic locations associated with the plurality of frames based on the locational metadata;

generate a set of location imagery associated with the set of geographic locations, the location imagery comprising a set of map features;

display the set of location imagery with the plurality of frames; and

synchronize the set of location imagery to the plurality of frames based on the set of geographic locations by causing the system to;

play a set of frames of the plurality of frames; and

display, in time with the playing, the generated location imagery associated with the geographic location of each of the played set of frames.

9. The system of claim 8, the instructions further causing the system to:

receive a selection by a user of an object within the video;

identify the object based on at least one of;

video frame, geographic location, and visual indicia;

search for descriptive information associated with the object; and

provide a description of the object in the location imagery based on the descriptive information.

10. The system of claim 8, the instructions further causing the system to:

receive, from a user, a navigation instruction in the location imagery; and

synchronize a set of frames of the plurality of frames to the navigation of the location imagery.

11. The system of claim 10, wherein the plurality of frames are displayed in an order based on the navigation received from the user, and wherein the synchronization pauses when a navigation of the location imagery is received from the user to a portion of location imagery that does not have an associated frame.

12. The system of claim 8, wherein the geographic locations are identified by geographic coordinate metadata and direction metadata of the video.

13. The system of claim 8, wherein the location imagery is at least one of:

a map view, a street view, an aerial view, a helicopter view, and a satellite view.

14. The system of claim 8, wherein the location imagery is displayed with the plurality of frames as at least one of:

a side-by-side partition, an overlay, and a pop-out window.

15. A computer program product for displaying location imagery in context with a video stream, the computer program product comprising a computer readable storage device, and program instructions stored on the computer readable storage device, to:

extract locational metadata from a set of video data, the video data comprising a plurality of frames;

identify a set of geographic locations associated with the plurality of frames based on the locational metadata;

generate a set of location imagery associated with the set of geographic locations, the location imagery comprising a set of map features;

display the set of location imagery with the plurality of frames; and

synchronize the set of location imagery to the plurality of frames based on the set of geographic locations by causing the system to;

play a set of frames of the plurality of frames; and

display, in time with the playing, the generated location imagery associated with the geographic location of each of the played set of frames.

16. The computer program product of claim 15, the computer readable storage device further comprising instructions to:

receive a selection by a user of an object within the video;

identify the object based on at least one of;

video frame, geographic location, and visual indicia;

search for descriptive information associated with the object; and

provide a description of the object in the location imagery based on the descriptive information.

17. The computer program product of claim 15, the computer readable storage device further comprising instructions to:

receive, from a user, a navigation instruction in the location imagery; and

synchronize a set of frames of the plurality of frames to the navigation of the location imagery.

18. The computer program product of claim 17, wherein the plurality of frames are displayed in an order based on the navigation received from the user, and wherein the synchronization pauses when a navigation of the location imagery is received from the user to a portion of location imagery that does not have an associated frame.

19. The computer program product of claim 15, wherein the geographic locations are identified by geographic coordinate metadata and direction metadata of the video.

20. The computer program product of claim 15, wherein the location imagery is at least one of:

a map view, a street view, an aerial view, a helicopter view, and a satellite view.