Systems and methods for compressing geotagged video

• US 10,452,715 B2
• Filed: 06/30/2012
• Issued: 10/22/2019
• Est. Priority Date: 06/30/2012
• Status: Active Grant

• Current Assignee: Sonic IP Incorporated (Endeavor Group Holdings Inc.)
• Original Assignee: DivX, LLC (f/k/a DivX CF Holdings LLC) (SoftBank Group Corp.)
• Inventors: Soroushian, Kourosh, Braness, Jason
• Primary Examiner(s): Hess, Michael J
• Application Number:
  US13/539,337
• Publication Number:
  US 20140003501A1
• Time in Patent Office: 2,670 Days
• Field of Search: None
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06F 16/73 Querying
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/75 Clustering; Classification
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/787 using geographical or spatial information, e.g. location
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 21/23109 by placing content in organized collections, e.g. EPG data repository details of retrieval of video data and associated meta data in video databases G06F16/739
• CPC Class Codes: H04N 21/422 Input-only peripherals , i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] input devices also receiving signals from specially adapted client devices H04N21/4104
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/85406 involving a specific file format, e.g. MP4 format

Abstract

Systems and methods for compressing and sharing geotagged video in accordance with embodiments of the invention are disclosed. One embodiment includes receiving a captured video sequence, where at least one geographic location is associated with the captured video sequence, selecting a segment of the captured video sequence, identifying a set of relevant video segments from a geotagged video database based on the at least one geotag associated with the captured video sequence, determining the video segment from the set of relevant video segments that is the best match by comparing the similarity of the content in the video segments to the content of the selected segment from the captured video sequence, encoding the selected segment, where the selected segment is encoded using predictions that include references to the video segment that is the best match, and storing the encoded video segment in the geotagged video database.

Claims

1. A method of encoding a video sequence using a geotagged video database, comprising:

receiving a captured video sequence including at least one video segment at an encoding server, where at least one geotag indicating geographic capture location information is associated with the captured video sequence;

selecting a segment from the at least one video segment in the captured video sequence using the encoding server;

determining geographic capture location information associated with the selected segment from the at least one geotag associated with the captured video sequence using the encoding server;

identifying a set of relevant video segments from a geotagged video database using the encoding server based on the geographic capture location information associated with the selected video segment, location information from geotags associated with video segments stored in the geotagged video database, and a velocity of a video capture device during capture of the selected segment;

comparing content of the selected segment to content of each video segment in the set of segments using the encoding server to determine a similarity of content between the selected segment and each video segment in the set of relevant video segments by;

performing feature matching with respect to at least one frame in the selected video segment and at least one frame from a video segment within the set of relevant video segments; and

comparing the photometric similarity of the at least one frame in the selected video segment and the at least one frame from the video segment within the set of relevant video segments;

determining a most relevant video segment from the set of relevant video segments that has content that is most similar to the content of the selected segment from the captured video sequence using the encoding server based upon the similarity of content in each video segment within the set of relevant video segments to content of the selected segment from the captured video sequence using the encoding server;

encoding the selected segment from the captured video sequence using the encoding server, where the selected segment is encoded using predictions that include references to the most relevant video segment from the geotagged video database, wherein the most relevant video segment from the geotagged video database was captured at a different time from the captured video sequence; and

storing the encoded video segment in the geotagged video database using the encoding server.

2. The method of claim 1, wherein the geotags associated with the video segments stored in the geotagged video database are metadata including at least one piece of information selected from the group consisting of latitude and longitude coordinates, altitude, bearing, distance, velocity, tilt, accuracy data, time of day, date, and place name.

3. The method of claim 1, wherein the at least one geotag associated with the captured video sequence comprises latitude and longitude coordinates, altitude, bearing, and tilt.

4. The method of claim 1, wherein the at least one geotag associated with the captured video sequence comprises a plurality of geotags associated with individual frames in the captured video sequence.

5. The method of claim 1, wherein:

selecting a segment of the captured video sequence comprises selecting an intra-frame from the captured video sequence;

the set of relevant video segments from the geotagged video database comprises a set of frames selected from the group consisting of intra-frames and anchor frames from the video segments stored in the geotagged video database; and

determining the most relevant video segment from the set of relevant video segments that is the best match comprises comparing the similarity of the selected intra-frame to the frames in the set of frames.

6. The method of claim 1, wherein identifying a set of relevant video segments from a geotagged video database using the encoding server based on the geographic capture location information associated with the selected video segment and the location information from geotags associated with video segments in the geotagged video database further comprises:

determining a capture location for the selected video segment based on geographic location information indicated by the at least one geotag associated with the captured video sequence; and

searching the geotagged video database for video segments having geotags indicating a capture location in proximity to the capture location of the selected video segment.

7. The method of claim 6, wherein identifying a set of relevant video segments from a geotagged video database using the encoding server based on the geographic capture location information associated with the selected video segment and the location information from geotags associated with video segments in the geotagged video database further comprises:

determining view information for the selected video segment includes a capture altitude, a bearing and a tilt for the selected video segment based on information in the at least one geotag associated with the captured video sequence; and

searching the geotagged video database for video segments having geotags indicating that the video segments having similar view information to the view information for the scene captured from the capture location at the capture altitude, bearing and tilt.

8. The method of claim 7, wherein identifying a set of relevant video segments from a geotagged video database using the encoding server based on the geographic capture location information associated with the selected video segment and the location information from geotags associated with video segments in the geotagged video database further comprises:

determining a capture time of the selected video segment based on information in the at least one geotag associated with the captured video sequence; and

searching the geotagged video database for video segments having geotags indicating a capture time similar to the capture time of the selected video segment.

9. The method of claim 1, wherein:

the geotagged video database includes metadata indicating the video recording device that captured a video segment;

a plurality of video segments in the geotagged video database were captured using the same recording device used to capture the received video sequence; and

wherein identifying a set of relevant video segments from a geotagged video database using the encoding server based on the geographic capture location information associated with the selected video segment and the location information from geotags associated with video segments in the geotagged video database further comprises searching for relevant video segments captured by the same recording device used to capture the received video sequence using the metadata in the geotagged video database indicating the video recording devices that captured video segments.

10. The method of claim 1, wherein determining the most relevant video segment from the set of relevant video segments considers both similarity of content measured during feature matching and photometric similarity.

11. The method of claim 1, wherein determining the most relevant video segment from the set of relevant video segments considers the number of video segments on which the encoding of a video segment depends.

12. The method of claim 1, wherein determining the most relevant video segment from the set of relevant video segments is based upon video segments in the set of video segments captured by the same recording device that captured the captured video sequence.

13. The method of claim 1, wherein:

the most relevant video segment from the geotagged video database is a different resolution from a resolution of the captured video sequence;

encoding the selected segment from the captured video sequence using the encoding server further comprises resampling the most relevant video segment from the geotagged video database to the resolution of the captured video sequence; and

encoding the selected segment using predictions that include references to the most relevant video segment from the geotagged video database comprises encoding the selected segment using predictions that include references to the resampled video segment.

14. The method of claim 13, further comprising generating metadata describing the resampling process used to resample the most relevant video segment from the geotagged video database and storing the metadata in a container file including the encoded segment from the captured video sequence.

15. The method of claim 1, wherein storing the encoded video segment in the geotagged video database further comprises storing the encoded video segment in a separate container file using the encoding server.

16. The method of claim 1, wherein storing the encoded video segment in the geotagged video database further comprises storing the encoded video segment in a container file that includes at least one video segment on which the encoding of the video segment depends.

17. The method of claim 16, wherein the encoded video sequence is multiplexed into the container file so that each particular reference frame referenced by a particular frame in the video sequence is located in the container file prior to the particular frame in the video sequence that references the particular reference frame.

18. The method of claim 1, wherein storing the encoded video segment in the geotagged video database comprises storing the encoded video segment on a server and writing an entry including the location of the encoded video segment into the geotagged video database.

19. The method of claim 1, wherein the captured video sequence comprises a plurality of video sequences wherein each of the plurality of video sequences is captured by a different one of a plurality of coordinated cameras where the plurality of coordinated cameras are coordinated to capture images at a same time, and wherein the video segments in the geotagged video database are captured by at least one camera that is not coordinated with the plurality of coordinated cameras.

20. The method of claim 1, wherein the captured video sequence and at least one video segment in the geotagged video database are captured using different cameras.

21. The method of claim 1, wherein the captured video sequence and at least one video segment in the geotagged video database are captured using the same camera.

22. A video sharing server system, comprising:

an encoding server; and

a geotagged video database including a plurality of video sequences tagged with geotags indicating geographic locations;

wherein the encoding server is configured to;

receive a captured video sequence including at least one video segment and associated with the captured video sequence at least one geotag, where the at least one geotag indicates geographic capture location information indicating at least one geographic location;

select a segment from the at least one video segment in the captured video sequence;

determine geographic capture location information associated with the selected segment from the at least one geotag associated with the captured video sequence;

identify a set of relevant video segments from a geotagged video database using the geographic capture location information associated with the selected video segment, location information from geotags associated with video segments stored in the geotagged video database, and a velocity of a video capture device during capture of the selected segment;

compare content of the selected segment to content of each video segment in the set of segments to determine a similarity of content between the selected segment and each video segment in the set of relevant video segments by;

performing feature matching with respect to at least one frame in the selected video segment and at least one frame from a video segment within the set of relevant video segments; and

comparing the photometric similarity of the at least one frame in the selected video segment and the at least one frame from the video segment within the set of relevant video segments;

determine a most relevant video segment from the set of relevant video segments that has content that is most similar to the content of the selected segment from the captured video sequence based on the similarity of content between the selected segment and each video segment in the set of relevant video segments;

encode the selected segment from the captured video sequence using predictions that include references to the most relevant video segment from the geotagged video database, wherein the most relevant video segment from the geotagged video database was captured at a different time from the captured video sequence; and

store the encoded video segment in the geotagged video database.

23. A non-transitory machine readable medium containing processor instructions, where execution of the instructions by a processor causes the processor to perform a process that comprises:

receiving a captured video sequence including at least one video segment and is associated with the captured video sequence at least one geotag, where the at least one geotag indicates geographic capture location information indicating at least one geographic location;

selecting a segment from the at least one video segment in the captured video sequence using the encoding server;

obtaining a set of relevant video segments from a geotagged video database using the geographic capture location information associated with the selected video segment, location information from geotags associated with video segments stored in the geotagged video database, and a velocity of a video capture device during capture of the selected segment;

comparing content of the selected segment to content of each video segment in the set of segments to determine a similarity of content between the selected segment and each video segment in the set of relevant video segments by;

performing feature matching with respect to at least one frame in the selected video segment and at least one frame from a video segment within the set of relevant video segments; and

comparing the photometric similarity of the at least one frame in the selected video segment and the at least one frame from the video segment within the set of relevant video segments;

determining a most relevant video segment from the set of relevant video segments that has content that is most similar to the content of the captured video sequence based on the similarity of content between the selected segment and each video segment in the set of relevant video segments; and

encoding the selected segment from the captured video sequence using predictions that include references to the video segment from the geotagged video database that is the best match, wherein the most relevant video segment from the geotagged video database was captured at a different time from the captured video sequence.