Audio/video methods and systems

• US 9,940,969 B2
• Filed: 07/08/2014
• Issued: 04/10/2018
• Est. Priority Date: 07/24/2009
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Lord, John D.
• Primary Examiner(s): ISLAM, MOHAMMAD K
• Application Number:
  US14/325,957
• Publication Number:
  US 20150003802A1
• Time in Patent Office: 1,372 Days
• Field of Search: 713 2, 713176, 713150, 4554221, 4555561, 715716, 726 26, 726 28, 348169, 348461
• CPC Class Codes: G11B 27/022 Electronic editing of analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/913 for scrambling ; for copy protection
• CPC Class Codes: H04N 9/8042 involving data reduction
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

Audio and or video data is structurally and persistently associated with auxiliary sensor data (e.g., relating to acceleration, orientation or tilt) through use of a unitary data object, such as a modified MPEG file or data stream. In this form, different rendering devices can employ co-conveyed sensor data to alter the audio or video content. Such use of the sensor data may be personalized to different users, e.g., through preference data. For example, accelerometer data can be associated with video data, allowing some users to view a shake-stabilized version of a video, and other users to view the video with such motion artifacts undisturbed. In like fashion, camera parameters, such as focal plane distance, can be co-conveyed with audio/video content—allowing the volume to be diminished (or not, again depending on user preference) when a camera captures audio/video from a distant subject. Some arrangements employ multiple image sensors and/or multiple audio sensors—each also collecting auxiliary data. A great number of other features and arrangements are also detailed.

Claims

1. A method comprising the acts:

in an originating hardware unit, receiving video information captured during entertainment content production, and transforming the video information for representation in a unitary data object, said transforming including compressing the video information;

in the originating unit, receiving audio information captured during entertainment content production, and transforming the audio information for representation in said unitary data object;

in the originating unit, receiving sensor information captured during entertainment content production, said sensor information comprising at least one of acceleration, orientation or tilt data, said sensor information being different than said video information and being different than said audio information, and transforming the sensor information for representation in said unitary data object; and

providing the unitary data object, comprising said compressed video, audio and sensor information, for rendering by first and second remote end user rendering units;

wherein the sensor information is persistently associated with the audio and compressed video information in a structured fashion by said unitary data object, and is thereby adapted for use by a processor in each of said first and second remote rendering units to selectively alter the audio and/or video information based on the sensor information—

or not, in accordance with end user or device preferences, so that different end users may view the video content with, or without, motion stabilization or motion-related sound effects.

2. The method of claim 1 that includes:

transforming the video information for representation in a video portion of the unitary data object;

transforming the audio information for representation in an audio portion of the unitary data object; and

transforming the sensor information for representation in said audio portion of the unitary data object.

3. The method of claim 1 that includes providing the unitary data object for rendering by plural remote end user rendering units via an online user generated content site.

15. The method of claim 1 that further includes:

the first remote end user rendering unit rendering the video without motion stabilization; and

the second remote end user rendering unit rendering the video with motion stabilization, employing the sensor information included in the unitary data object.

16. The method of claim 1 that further includes:

the first remote end user rendering unit rendering the audio without motion-related sound effects; and

the second remote end user rendering unit rendering the audio with motion-related sound effects, employing the sensor information included in the unitary data object.

17. The method of claim 1 in which the sensor information comprises a parameter related to acceleration.

18. The method of claim 1 in which the sensor information comprises a parameter related to orientation.

19. The method of claim 1 in which the sensor information comprises a parameter related to tilt.

4. A method comprising the acts:

in an originating hardware unit, receiving video information captured during entertainment content production, and transforming the video information for representation in a unitary data object, said transforming including compressing the video information;

in the originating unit, receiving audio information captured during entertainment content production, and transforming the audio information for representation in said unitary data object;

in the originating unit, receiving sensor information captured during entertainment content production, said sensor information comprising data indicating focal length of a camera that captured the video information, said sensor information being different than said video information and being different than said audio information; and

providing the unitary data object, comprising said compressed video, audio and sensor information, for rendering by first and second remote end user rendering units;

wherein the sensor information is persistently associated with the audio and video information in a structured fashion by said unitary data object, and is thereby adapted for use by a processor in each of said first and second remote rendering units to selectively alter the audio information based on the sensor information—

or not, in accordance with end user or device preferences, so that different end users may view the video content with, or without, dimensional audio.

5. The method of claim 4 that includes:

transforming the video information for representation in a video portion of the unitary data object;

transforming the audio information for representation in an audio portion of the unitary data object; and

transforming the sensor information for representation in said audio portion of the unitary data object.

6. The method of claim 5 that includes providing the unitary data object for rendering by plural remote end user rendering units via an online user generated content site.

7. A mobile phone device including a processor, a memory, a screen, an audio output, and a wireless interface, the memory containing software instructions that configure the device to receive a unitary data object via said wireless interface, the unitary data object representing entertainment content comprising compressed video information, audio information, and sensor information, said sensor information comprising data indicating motion of a camera used to capture the video information and/or indicating distance of said camera from a camera subject, said sensor information being different than said video information and being different than said audio information, said software instructions enabling a user of the device to specify whether the entertainment content should be altered in accordance with said sensor information, when the entertainment content is rendered to the user.

8. The mobile phone device of claim 7 in which said software instructions enable the user to specify whether the entertainment content should be motion-stabilized in accordance with said sensor information, or not, when the entertainment content is rendered to the user.

9. The mobile phone device of claim 7 in which said software instructions enable the user to specify whether the audio of the entertainment content should be volume-controlled in accordance with said distance of the camera to the camera subject, or not, when the entertainment content is rendered to the user.

10. The mobile phone device of claim 7 in which said instructions configure the device to selectably employ sensor information indicating motion of the camera used to capture the video information, to alter the entertainment content.

11. The mobile phone device of claim 7 in which said instructions configure the device to selectably apply motion stabilization to the video information, using said sensor information.

12. The mobile phone device of claim 7 in which said instructions configure the device to selectably apply motion-related sound effects to the audio, using said sensor information.

13. The mobile phone device of claim 7 in which said instructions configure the device to selectably employ sensor information indicating distance of the camera from the camera subject, to alter the entertainment content.

14. The mobile phone device of claim 7 in which said instructions configure the device to selectably apply dimensional audio effects to the audio, using said sensor information.