AUDIO/VIDEO METHODS AND SYSTEMS

• US 20110069229A1
• Filed: 07/23/2010
• Published: 03/24/2011
• Est. Priority Date: 07/24/2009
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Lord, John D.
• Granted Patent: US 8,773,589 B2
• Time in Patent Office: Days
• US Class Current: 348/461
• CPC Class Codes: G11B 27/022 Electronic editing of analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/913 for scrambling ; for copy protection
• CPC Class Codes: H04N 9/8042 involving data reduction
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

Audio and or video data is structurally and persistently associated with auxiliary sensor data (e.g., relating to acceleration, orientation or tilt) through use of a unitary data object, such as a modified MPEG file or data stream. In this form, different rendering devices can employ co-conveyed sensor data to alter the audio or video content. Such use of the sensor data may be personalized to different users, e.g., through preference data. For example, accelerometer data can be associated with video data, allowing some users to view a shake-stabilized version of a video, and other users to view the video with such motion artifacts undisturbed. In like fashion, camera parameters, such as focal plane distance, can be co-conveyed with audio/video content—allowing the volume to be diminished (or not, again depending on user preference) when a camera captures audio/video from a distant subject. Some arrangements employ multiple image sensors and/or multiple audio sensors—each also collecting auxiliary data. A great number of other features and arrangements are also detailed.

Claims

1. A method comprising the acts:

receiving video information, and transforming the video information for representation in a video portion of a unitary data object;

receiving audio information, and transforming the audio information for representation in an audio portion of the unitary data object;

receiving sensor information, comprising at least one parameter relating to acceleration, orientation or tilt, and transforming the sensor information for representation in the unitary data object; and

transmitting the unitary data object to a data receiver, or storing the unitary data object on a computer readable storage medium, so that the sensor information is structurally associated with the audio and video information by the unitary data object, and thereby adapted for use by a processor in altering the audio or video information.

2. The method of claim 1 that includes using a processor to alter the audio or video information in accordance with at least some of the sensor information.

3. The method of claim 1 that includes transforming the sensor information for representation in the video portion of the unitary data object.

4. The method of claim 1 in which the unitary data object comprises an MPEG data stream or an MPEG data file.

5. The method of claim 1 that includes transforming the sensor information to a frequency range near the bottom or top of the human hearing range.

6. The method of claim 1 that includes representing the sensor information as a signal that is steganographically hidden in the audio or video information.

7. The method of claim 1 wherein the sensor data comprises acceleration data.

8. The method of claim 1 wherein the sensor data comprises orientation data.

9. The method of claim 1 wherein the sensor data comprises tilt data.

10. The method of claim 1 that further includes receiving camera data, and transforming for representation in the audio portion of the unitary data object, wherein the camera data comprises at least one parameter relating to focus, zoom, aperture size, depth-of-field, exposure time, ISO setting, and/or focal depth.

11. A method comprising the acts:

receiving a unitary data object;

recovering audio data from an audio portion of the unitary data object; and

recovering sensor data from the audio portion of the unitary data object, the sensor data comprising at least one parameter relating to acceleration, orientation or tilt;

wherein at least one of said recovering acts is performed by a hardware processor.

12. The method of claim 11 in which said recovering sensor data comprises applying a steganographic decoding process to audio data to extract the sensor data therefrom.

13. The method of claim 11 that includes altering the recovered audio data in accordance with at least some of said sensor data.

14. The method of claim 11 that further includes recovering video data from a video portion of the unitary data object, and altering the recovered video in accordance with at least some of said sensor data.

15. A method comprising the acts:

receiving a unitary data object;

recovering audio data from an audio portion of the unitary data object; and

recovering camera data from the audio portion of the unitary data object, the camera data comprising at least one parameter relating to focus, zoom, aperture size, depth-of-field, exposure time, ISO setting, and/or focal depth;

wherein at least one of said recovering acts is performed by a hardware processor.

16. The method of claim 15 that includes altering the recovered audio data in accordance with at least some of said sensor data.

17. The method of claim 15 that further includes recovering video data from a video portion of the unitary data object, and altering the recovered video in accordance with at least some of said sensor data.

18. A method comprising the acts:

receiving a unitary data object;

recovering both video data and sensor data from the unitary data object, the sensor data comprising at least one parameter relating to acceleration, orientation or tilt; and

processing the video data in accordance with at least some of the sensor data, to yield altered video data for rendering to a user;

wherein at least one of said acts is performed by a hardware processor.

19. The method of claim 18 that includes:

obtaining user preference data;

processing the video data with at least some of the sensor data in accordance with the user preference data; and

rendering the processed video to a user.

20. The method of claim 19 that includes shake-compensating the video by use of the sensor data.

21. The method of claim 19 that includes obtaining the user preference data from the user through a user interface.

22. A computer readable storage medium containing non-transitory software instructions causing a processor programmed thereby to:

recover both video data and sensor data from a received unitary data object, the sensor data comprising at least one parameter relating to acceleration, orientation or tilt; and

process the video data in accordance with at least some of the sensor data, to yield altered video data for rendering to a user.

23. A method comprising the acts:

receiving a unitary data object;

recovering both video data and camera data from the unitary data object, the camera data comprising at least one parameter relating to focus, zoom, aperture size, depth-of-field, exposure time, ISO, and/or lens focal-length; and

processing the video data in accordance with at least some of the camera data to yield altered video data for rendering to a user;

wherein at least one of said acts is performed by a hardware processor.

24. A method comprising the acts:

receiving a unitary data object;

recovering both audio data and sensor data from the unitary data object, the sensor data comprising at least one parameter relating to acceleration, orientation or tilt; and

processing the audio data in accordance with at least some of the sensor data to yield altered audio data for rendering to a user;

wherein at least one of said acts is performed by a hardware processor.

25. A method comprising the acts:

receiving a unitary data object;

recovering both audio data and camera data from the unitary data object, the camera data comprising at least one parameter relating to focus, zoom, aperture size, depth-of-field, exposure time, ISO, and/or lens focal-length; and

processing the audio data in accordance with at least some of the camera data to yield altered audio data for rendering to a user;

wherein at least one of said acts is performed by a hardware processor.

26. The method of claim 25 in which the processing comprises altering an amplitude of audio in accordance with camera focus data to yield dimensional audio.

27. A method comprising:

collecting sensor information from a sensor conveyed by a movable object in a sporting event;

collecting video information from the sporting event using a camera at a location remote from said movable object;

creating a unitary data object that includes both data corresponding to the collected sensor information, and data corresponding to said collected video information; and

storing the unitary data object in a computer readable storage medium, or transmitting the unitary data object to a data receiver.

28. The method of claim 27 that includes collecting the sensor information from a sensor conveyed by a puck, a ball, a horse, a player, or a car.

29. The method of claim 27 wherein the sensor data comprises at least one of acceleration data, orientation data, location data, and/or tilt data.

30. A mobile phone comprising a processor, and at least first and second sensors, the processor being configured to create a unitary data object including information sensed by the first sensor and information sensed by the second sensor, wherein the first sensor comprises an image or audio sensor, and the second sensor comprises an acceleration sensor, an orientation sensor, or a tilt sensor.

31. The mobile phone of claim 30 wherein the other sensor comprises an acceleration sensor.

32. The mobile phone of claim 30 wherein the other sensor comprises an orientation sensor.

33. The mobile phone of claim 30 wherein the other sensor comprises a tilt sensor.