Techniques for synchronizing audio and video data in an image signal processing system
First Claim
1. A method for synchronizing audio and video data on an electronic device comprising:
- using an image signal processor to receive a plurality of image frames corresponding to the video data, the plurality of image frames being acquired using a digital image sensor of an electronic device;
detecting the start of each image frame of the plurality of image frames;
upon detecting the start of each image frame, reading a respective current timestamp value from a time code register configured to provide timestamp values corresponding to a current time;
associating each respective current timestamp value with a respective set of metadata associated with the respective image frame, wherein each respective current timestamp value corresponds to the start of each respective image frame; and
aligning each image frame of the plurality of image frames with a corresponding audio sample from a set of audio data acquired concurrently with the video data by the electronic device using the respective current timestamp value associated with the respective set of metadata to synchronize the playback of each image frame.
1 Assignment
0 Petitions
Accused Products
Abstract
The present disclosure provides techniques for performing audio-video synchronization using an image signal processing system. In one embodiment, a time code register provides a current time stamp when sampled. The value of the time code register may be incremented at regular intervals based on a clock of the image signal processing system. At the start of a current frame acquired by an image sensor, the time code register is sampled, and a timestamp is stored into a timestamp register associated with the image sensor. The timestamp is then read from the time stamp register and written to a set of metadata associated with the current frame. The timestamp stored in the frame metadata may then be used to synchronize the current frame with a corresponding set of audio data.
-
Citations
26 Claims
-
1. A method for synchronizing audio and video data on an electronic device comprising:
-
using an image signal processor to receive a plurality of image frames corresponding to the video data, the plurality of image frames being acquired using a digital image sensor of an electronic device; detecting the start of each image frame of the plurality of image frames; upon detecting the start of each image frame, reading a respective current timestamp value from a time code register configured to provide timestamp values corresponding to a current time; associating each respective current timestamp value with a respective set of metadata associated with the respective image frame, wherein each respective current timestamp value corresponds to the start of each respective image frame; and aligning each image frame of the plurality of image frames with a corresponding audio sample from a set of audio data acquired concurrently with the video data by the electronic device using the respective current timestamp value associated with the respective set of metadata to synchronize the playback of each image frame. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A system comprising:
-
an image signal processor comprising an interface configured to receive a plurality of image frames acquired using a digital image sensor, wherein the plurality of image frames correspond to a set of video data; an audio sub-system comprising an input device configured to receive audio data concurrently with the image data acquired by the digital image sensor; and audio-video synchronization logic configured to, for each of the plurality of image frames, identify the start of the image frame, determine a timestamp value corresponding to the start of the image frame, and write the timestamp value to metadata associated with the image frame; wherein the audio-video synchronization logic is configured to synchronize the plurality of image frames with the audio data by correlating the timestamps values stored in the metadata associated with each of the plurality of image frames to corresponding timestamps associated with the audio data. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. One or more non-transitory tangible computer-readable storage media having instructions encoded thereon for execution by a processor, the instructions comprising:
-
code to detect the beginning of a plurality of image frames from a set of video data acquired by an image processing system of an electronic device using a digital image sensor; code to associate a timestamp value with metadata associated with each image frame of the plurality of image frames, wherein each respective timestamp value corresponds to the beginning of each respective image frame; code to align each respective image frame with a corresponding audio sample from a set of audio data acquired by the electronic device using each respective timestamp value. - View Dependent Claims (16, 17, 18)
-
-
19. An electronic device comprising:
-
a first image sensor; a first image sensor interface; an image signal processing sub-system configured to receive a first set of video data acquired using the first image sensor via the first sensor interface, wherein the first set of video data comprises a first plurality of image frames; an audio processing sub-system configured to receive a first set of audio data acquired concurrently with the first set of video data; and audio-video synchronization circuitry comprising; a time code register configured to provide a current time value; a first timestamp register associated with the first image sensor; and logic configured to, for each image frame of the first set of video data, detect the start of an image frame of the first set of video data, sample the time code register upon detecting the start of the image frame to acquire a timestamp value corresponding to the start of the image frame, store the timestamp value in the first timestamp register, write the timestamp value from the first timestamp register to a set of metadata associated with the image frame, and synchronize the first set of video data and the first set of audio data by correlating the timestamp values in the metadata of each image frame of first set of the video data to corresponding timestamps associated with the first set of audio data. - View Dependent Claims (20, 21, 22, 23, 24, 25, 26)
-
Specification