METHOD AND APPARATUS TO PROVIDE A PHYSICAL STIMULUS TO A USER, TRIGGERED BY A MOTION DETECTION IN A VIDEO STREAM

• US 20110044604A1
• Filed: 03/04/2009
• Published: 02/24/2011
• Est. Priority Date: 03/10/2008
• Status: Abandoned Application

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Braspenning, Ralph, Brokken, Dirk
• Application Number:
  US12/920,874
• Publication Number:
  US 20110044604A1
• Time in Patent Office: Days
• US Class Current: 386/239
• CPC Class Codes: A63F 2300/302 specially adapted for receiving control signals not targeted to a display device or game input means, e.g. vibrating driver's seat, scent dispenser
• CPC Class Codes: A63F 2300/6018 where the game content is authored by the player, e.g. level editor or by game device at runtime, e.g. level is created from music data on CD
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

An audio stream (14) and video stream (12) from a conventional audiovisual source (10) are processed by processor (20). A motion processor (30) establishes at least one motion feature and outputs it to the stimulus controller (32) which generates a stimulus in stimulus generator (34). The stimulus generator (34) may be a Galvanic Vestibular Stimulus generator.

Claims

1. A method for reproducing a video data stream representing a sequence of images for a user the method comprising:

extracting at least one motion feature representing motion from the video data stream (12); and

generating (88) stimulus data from the motion feature; and

applying (89) a non audio-visual physical stimulus to a user (36) based on the stimulus data.

2. A method according to claim 1 wherein the step of extracting a motion feature comprises

estimating the dominant motion of the scene by calculating (60) motion data of each of a plurality of blocks of pixels

analyzing (62,64) the distribution of the motion data;

and

if there is a dominant peak in the distribution of motion data identifying (68) the motion of that peak as a motion feature.

3. A method according to claim 1 wherein the step of extracting a motion feature comprises:

motion segmenting (84) the foreground from the background; and

calculating (86) the respective motion of foreground and background as a motion feature.

4. A method according to claim 1 wherein the step of applying (89) a non audio-visual stimulus applies a Galvanic Vestibular Stimulus to the user.

5. A method according to claim 1 wherein the step of applying (89) a non audio-visual stimulus includes applying a tactile stimulation of the skin of the user.

6. A method according to claim 1 wherein applying (89) a non audio-visual stimulus includes physically moving the user'"'"'s body or part thereof.

7. A method according to claim 1 wherein the video data stream (12) is accompanied by an audio stream (14) further comprising:

receiving the audio stream and the extracted motion data;

modifying the audio data in the audio stream based on the extracted motion data; and

outputting the modified audio data through an audio reproduction unit.

8. A computer program product arranged to enable a computer connected to a stimulus generator for applying a non audio-visual stimulus to a user to carry out a method according to claim 1.

9. Apparatus for reproducing a video data stream representing a sequence of images for a user comprising:

a motion processor (30) arranged to extract at least one motion feature representing motion from the video data stream

a stimulus generator (34) arranged to provide a non-audiovisual stimulus;

wherein the motion processor (30) is arranged to drive the stimulus generator based on the extracted motion feature.

10. Apparatus according to claim 9 wherein the stimulus generator (34) is a Galvanic Vestibular stimulus generator integrated into a headphone.

11. Apparatus according to claim 9 wherein the stimulus generator (34) includes at least one mechanical actuator (62) built into a body-contact object (60) for applying a tactile stimulation of the skin of the user.

12. Apparatus according to claim 9 wherein the stimulus generator (34) includes an actuator (64) arranged to physically move a ground surface or furniture for applying a non audio-visual stimulus includes physically moving the user'"'"'s body or part thereof.

13. Apparatus according to claim 9 wherein the motion processor is arranged to estimate the dominant motion of the scene by calculating motion data of each of a plurality of blocks of pixels to analyze the distribution of the motion data;

and if there is a dominant peak in the distribution of motion data to identify the motion of that peak as the motion feature.

14. Apparatus according to claim 9 wherein the motion processor (30) is arranged to motion segment the foreground from the background and to calculate the respective motion of foreground and background as the motion feature.

15. Apparatus according to claim 9 further comprising:

an audio processor (48) arranged to receive an audio data stream and to receive the extracted motion feature from the effects processor and to modify the received audio data in the audio stream based on the extracted motion feature; and

an audio reproduction unit for outputting the modified audio data.