Data processing systems

• US 10,643,665 B2
• Filed: 04/29/2016
• Issued: 05/05/2020
• Est. Priority Date: 04/29/2015
• Status: Active Grant

• Current Assignee: TomTom International BV (TomTom NV)
• Original Assignee: TomTom International BV (TomTom NV)
• Inventors: de Jong, Frank, Kleinjan, Eveline Anna, Pauwels, Peter-Frans, Stanisic, Slobodan, Hetherington, Douglas, Spence, Gavin, Hermsen, Walter, Zilio, Davide, Obogeanu-Hempel, Erno Marius, Obradovic, Nebojsa
• Primary Examiner(s): Tran, Thai Q
• Assistant Examiner(s): Park, Sunghyoun
• Application Number:
  US15/570,678
• Publication Number:
  US 20180190323A1
• Time in Patent Office: 1,467 Days
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G11B 19/20 Driving; Starting; Stopping; Control thereof
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure

Abstract

Methods and systems are disclosed related to the processing of video and sensor data recorded by a video camera. For example, a first embodiment is directed to the storage of sensor data in a metadata portion of a digital media file, a second embodiment is directed to the storage of highlight data in a metadata portion of a digital media file, and a third embodiment is directed to the creation of highlight data based on sensor data.

Claims

1. A method of storing data collected by a digital video camera having one or more sensor devices associated therewith, the method comprising:

receiving an first input to cause the camera to start recording;

opening a digital media container on a first memory based on receipt of the first input, the opening including reserving a predetermined amount of memory before a payload portion of the digital media container into which a metadata portion of the container is to be added when closing the container to create the media file;

writing video image data based on data received from an image sensor of the camera to the payload portion of the digital media container;

storing sensor data based on data received from the one or more sensor devices in a second memory, wherein the second memory is not used for storing video image data;

upon receiving a second input, causing the camera to stop recording;

after the camera has stopped recording, acquiring, from the second memory, the sensor data and incorporating the sensor data into a metadata portion in the predetermined amount of memory that was previously reserved before the payload portion of the digital media container on the first memory; and

closing the digital media container to create a digital media file stored in the first memory.

2. The method of claim 1, wherein the first input and/or second input is a manual input by a user, such as the user actuating a user input of the camera or a remote control device operatively connected to the camera.

3. The method of claim 1, wherein the first input and/or second input is automatically generated based on data from the one or more of the sensor devices.

4. The method of claim 1, wherein the camera comprises a video processing device, and the video image data comprises an encoded video stream generated using an encoder of the video processing device from the data received from the image sensor.

5. The method of claim 1, wherein the first memory is different from the second memory, said first memory being a memory removable from the camera, and said second memory being a non-removable memory within the camera.

6. The method of claim 1, wherein the one or more sensor devices measure at least one of:

(i) movements or other physical parameters of the camera, user of the camera and/or item of sports equipment and/or vehicle to which the camera is mounted;

(ii) environmental conditions around the user and/or camera; and

(iii) physiological properties of the user.

7. The method of claim 1, wherein the sensor data received from the one or more sensor devices is stored in the second memory in association with data indicative of a time at which the data was determined by the sensor data.

8. The method of claim 1, wherein the sensor data is contemporaneous with the video image data, such that the sensor data comprises data collected from the one or more sensor devices during the time in which the video image data is recorded.

9. The method of claim 1, further comprising:

storing highlight data identifying one or more times of interest in the video image data in the second memory; and

adding the highlight data stored in the second memory to the metadata portion of the digital media container based on receipt of the second input.

10. The method of claim 9, wherein the highlight data comprises one or more highlights, each highlight being a time period in the video image data.

11. The method of claim 10, wherein the time period of a highlight comprises a time window based on a time of a tag, said tag being a single time in the video image data.

12. The method of claim 11, wherein the time of the tag for a highlight is determined based on received data indicative of a manual input by a user, such as the user actuating a user input of the camera or a remote control device operatively connected to the camera.

13. The method of claim 11, wherein the time of the tag for a highlight is automatically determined based on an analysis of the sensor device received from the one or more sensor devices.

14. The method of claim 13, wherein the analysis of the sensor data is performed after receipt of the second input to cause the camera to stop recording, so as to generate the highlight data, and the sensor data and the highlight data are added to the metadata portion of the digital media container.

15. The method of claim 13, wherein the camera is associated with a plurality of sensor devices, and wherein the analysis of the sensor data comprises analysing a plurality of datasets from the plurality of sensor devices, each of the datasets comprises a plurality of data values for a plurality of times during a time period in which the video image data is collected.

16. The method of claim 15, wherein the analysis of the datasets comprises:

identifying extrema in each of the plurality of datasets;

determining, for each of the identified extrema, if the time of an extremum is within a predetermined time of the time of another of the extrema; and

generating a plurality of clusters based on the determination, each cluster comprising a plurality of extrema, wherein the time of each extremum in the cluster is within the predetermined time of the time of another extremum in the cluster.

17. The method of claim 16, wherein each cluster has a start time and an end time, which together define a duration for the cluster, and wherein the time window for a highlight created from a cluster is based on the duration for the cluster.

18. The method of claim 1, further comprising:

determining, based on one or more characteristics of the video image data to be written in the payload portion of the digital media container, a size of the predetermined amount of memory.

19. The method of claim 18, wherein the characteristics of the video image data include a format of the video image data.

20. A system for storing data collected by a digital video camera having one or more sensor devices associated therewith, the system comprising one or more processing resources configured to:

receive a first input to cause the camera to start recording;

open a digital media container on a first memory based on receipt of the first input, the opening including reserving a predetermined amount of memory before a payload portion of the digital media container into which a metadata portion of the container is to be added when closing the container to create the media file;

write video image data based on data received from an image sensor of the camera to the payload portion of the digital media container;

store sensor data based on data received from the one or more sensor devices in a second memory, wherein the second memory is not used for storing video image data;

upon receiving a second input, causing the camera to stop recording;

after the camera has stopped recording, acquiring, from the second memory, the sensor data and incorporating the sensor data into a metadata portion in the predetermined amount of memory that was previously reserved before the payload portion of the digital media container on the first memory; and

close the digital media container to create a digital media file stored in the first memory.

21. A non-transitory computer readable medium comprising computer software comprising a set of instructions that, when executed by one or more processors of a digital video camera, cause the digital video camera to perform a method of storing data collected by a digital video camera having one or more sensor devices associated therewith, the method comprising:

receiving an first input to cause the camera to start recording;

opening a digital media container on a first memory based on receipt of the first input, the opening including reserving a predetermined amount of memory before a payload portion of the digital media container into which a metadata portion of the container is to be added when closing the container to create the media file;

writing video image data based on data received from an image sensor of the camera to the payload portion of the digital media container;

storing sensor data based on data received from the one or more sensor devices in a second memory, wherein the second memory is not used for storing video image data;

upon receiving a second input, causing the camera to stop recording;

after the camera has stopped recording, acquiring, from the second memory, the sensor data and incorporating the sensor data into a metadata portion in the predetermined amount of memory that was previously reserved before the payload portion of the digital media container on the first memory; and

closing the digital media container to create a digital media file stored in the first memory.