Information processing apparatus

• US 8,619,197 B2
• Filed: 11/04/2009
• Issued: 12/31/2013
• Est. Priority Date: 05/11/1999
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kondo, Tetsujiro, Okumura, Yuji, Kokubo, Tetsushi, Teshigawara, Satoshi
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): EDWARDS, TYLER B
• Application Number:
  US12/612,090
• Publication Number:
  US 20100045818A1
• Time in Patent Office: 1,518 Days
• Field of Search: 348699-701, 37524008-24017
• US Class Current: 348/700
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 9/00 Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131
• CPC Class Codes: G09B 9/05 the view from a vehicle being simulated G09B9/052, G09B9/058 take precedence
• CPC Class Codes: G09B 9/12 Motion systems for aircraft simulators

Abstract

Relationship information is previously generated and stored by learning based on camera motion estimation information expressing motion of a video camera, which is detected by a desired image signal picked up by the video camera, and camera motion information expressing physical motion of the video camera, which was obtained by a sensor for detecting physical motion at the same time when the desired image signal was picked up by the video camera. Camera motion estimation information with respect to an inputted image signal is detected from the inputted image signal. Camera motion prediction information with respect to the inputted image signal is generated, based on the camera motion estimation information detected and the relationship information. Based on the camera motion prediction information, a vibration signal for vibrating an object is generated. As a result, vibration data can be easily generated at low costs, based on existing video assets.

Claims

1. A learning apparatus comprising:

camera motion estimation information detection means for detecting camera motion estimation information from a desired image signal picked up by a video camera; and

coefficient generation means for generating a conversion coefficient for generating camera motion prediction information for learning expressing motion of the video camera which picked up an arbitrary image signal, from the arbitrary image signal, based on (a) sensor signal expressing physical motion of the video camera, which is obtained by a sensor for detecting physical motion, at the same time when the desired image signal was picked up, and (b) the camera motion estimation information;

wherein the desired image signal is a signal obtained for learning processing that is performed automatically by a learning section of said apparatus.

2. The apparatus according to claim 1, wherein the camera motion estimation includes a motion vector detection section for detecting a motion vector from the inputted image signal, a motion center detection section for detecting a motion center, based on the motion vector, and a camera motion estimation information detection section for detecting the camera motion estimation information, based on the motion vector and the motion center.

3. The apparatus according to claim 2, wherein the motion vector detection section detects the motion vector, with respect to a plurality of preset representative points of the inputted image signal.

4. The apparatus according to claim 2, comprising an acceleration sensor as the sensor.

5. The apparatus according to claim 2, comprising an angular speed sensor as the sensor.

6. The apparatus according to claim 2,

wherein the camera motion estimation information detection means generates the camera motion estimation information which is constructed by information of plural kinds of motions.

7. A learning method comprising the steps of:

detecting camera motion estimation information from a desired image signal picked up by a video camera; and

generating a conversion coefficient for generating camera motion prediction information for learning expressing motion of the video camera which picked up an arbitrary image signal, from the arbitrary image signal, based on (a) sensor signal expressing physical motion of the video camera, which is obtained by a sensor for detecting physical motion, at the same time when the desired image signal was picked up, and (b) the camera motion estimation information;

wherein the desired image signal is a signal obtained for learning processing that is performed automatically by a learning section.

8. The method according to claim 7, wherein the step of detecting the camera motion estimation information includes a step of detecting a motion vector from the inputted image signal, a step of detecting a motion center, based on the motion vector, and a step of detecting the camera motion estimation information, based on the motion vector and the motion center.

9. The method according to claim 8, wherein in the step of detecting the motion vector, the motion vector is detected with respect to a plurality of preset representative points of the inputted image signal.

10. The method according to claim 8, wherein in the step of generating the conversion coefficient, the conversion coefficient is generated, based on camera motion information expressing physical motion of the video camera, which was obtained by an acceleration sensor as the sensor.

11. The method according to claim 8, wherein in the step of generating the conversion coefficient, the conversion coefficient is generated, based on camera motion information expressing physical motion of the video camera, which was obtained by an angular speed sensor as the sensor.

12. The method according to claim 8,

wherein in the step of detecting the camera motion estimation information, the camera motion estimation information which is constructed by information of plural kinds of motions.

13. A non-transitory computer recording medium encoded with a computer program for letting a computer execute learning processing, the program comprising the steps of:

detecting camera motion estimation information from a desired image signal picked up by a video camera; and

generating a conversion coefficient for generating camera motion prediction information for learning expressing motion of the video camera which picked up an arbitrary image signal, from the arbitrary image signal, based on (a) sensor signal expressing physical motion of the video camera, which is obtained by a sensor for detecting physical motion, at the same time when the desired image signal was picked up, and (b) the camera motion estimation information;

wherein the desired image signal is a signal obtained for learning processing that is performed automatically by a learning section.

14. The medium according to claim 13, wherein the step of detecting the camera motion estimation information includes a step of detecting a motion vector from the inputted image signal, a step of detecting a motion center, based on the motion vector, and a step of detecting the camera motion estimation information, based on the motion vector and the motion center.

15. The medium according to claim 13, wherein in the step of detecting the motion vector, the motion vector is detected with respect to a plurality of preset representative points of the inputted image signal.

16. The medium according to claim 13, wherein in the step of generating the conversion coefficient, the conversion coefficient is generated, based on camera motion information expressing physical motion of the video camera, which was obtained by an acceleration sensor as the sensor.

17. The medium according to claim 13, wherein in the step of generating the conversion coefficient, the conversion coefficient is generated, based on camera motion information expressing physical motion of the video camera, which was obtained by an angular speed sensor as the sensor.

18. The medium according to claim 13,

wherein in the step of generating the camera motion estimation information, camera motion estimation information which is constructed by information of plural kinds of motions.

19. An apparatus comprising:

memory means for storing relationship information generated by learning based on camera motion estimation information for learning expressing motion of a video camera, which is detected by a desired image signal picked up by the video camera, and camera motion information for learning expressing physical motion of the video camera, which was obtained by a sensor for detecting physical motion at the same time when the desired image signal was picked up by the video camera;

camera motion estimation information detection means for detecting camera motion estimation information with respect to an inputted image signal, from the inputted image signal; and

camera motion prediction information generation means for generating camera motion prediction information with respect to the inputted image signal, based on the camera motion estimation information of the inputted image signal detected by the camera motion estimation information detection means and the relationship information, wherein a motion signal causes a user to experience a live feeling as if said user were riding in an image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on (a) one or more motion vectors in the desired image signal and (b) the camera motion estimation information of the inputted image signal is generated based on a motion vector in the input image signal.

20. A learning apparatus comprising:

camera motion estimation information detection means for detecting camera motion estimation information from a desired image signal picked up by a video camera; and

coefficient generation means for generating a conversion coefficient for generating camera motion prediction information expressing motion of the video camera which picked up an arbitrary image signal, from the arbitrary image signal, based on (a) sensor signal expressing physical motion of the video camera, which is obtained by a sensor for detecting physical motion, at the same time when the desired image signal was picked up, and (b) the camera motion estimation information, wherein a motion signal causes a user to experience a live feeling as if said user were riding in an image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on (1) one or more motion vectors in the desired image signal and (2) the camera motion estimation information of an inputted image signal is generated based on a motion vector in the input image signal.

21. An information processing method comprising the steps of;

generating relationship information generated by learning based on (a) camera motion estimation information expressing motion of a video camera, which is detected by a desired image signal picked up by the video camera, and (b) camera motion information expressing physical motion of the video camera, which was obtained by a sensor for detecting physical motion at the same time when the desired image signal was picked up by the video camera;

detecting camera motion estimation information with respect to an inputted image signal, from the inputted image signal; and

generating camera motion prediction information with respect to the inputted image signal, based on (1) the camera motion estimation information detected and (2) the relationship information, wherein a motion signal causes a user to experience a live feeling as if said user were riding in an image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on (i) one or more motion vectors in the desired image signal and (ii) the camera motion estimation information of the inputted image signal is generated based on a motion vector in the input image signal.

22. A learning method comprising the steps of:

detecting camera motion estimation information from a desired image signal picked up by a video camera; and

generating a conversion coefficient for generating camera motion prediction information expressing motion of the video camera which picked up an arbitrary image signal, from the arbitrary image signal, based on (a) sensor signal expressing physical motion of the video camera, which is obtained by a sensor for detecting physical motion, at the same time when the desired image signal was picked up, and (b) the camera motion estimation information, wherein a motion signal causes a user to experience a live feeling as if said user were riding in an image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on (1) one or more motion vectors in the desired image signal and (2) the camera motion estimation information of an inputted image signal is generated based on a motion vector in the input image signal.

23. A non-transitory computer program recording medium encoded with a computer program for letting a computer execute information processing, the program comprising the steps of:

generating relationship information generated by learning based on (a) camera motion estimation information expressing motion of a video camera, which is detected by a desired image signal picked up by the video camera, and (b) camera motion information expressing physical motion of the video camera, which was obtained by a sensor for detecting physical motion at the same time when the desired image signal was picked up by the video camera;

detecting camera motion estimation information with respect to an inputted image signal, from the inputted image signal; and

generating camera motion prediction information with respect to the inputted image signal, based on (1) the camera motion estimation information detected and (2) the relationship information, wherein a motion signal causes a user to experience a live feeling as if said user were riding in an image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on (i) one or more motion vectors in the desired image signal and (ii) the camera motion estimation information of the inputted image signal is generated based on a motion vector in the input image signal.

24. A non-transitory recording medium encoded with a computer program for letting a computer execute learning processing, the program comprising the steps of:

detecting camera motion estimation information from a desired image signal picked up by a video camera; and

generating a conversion coefficient for generating camera motion prediction information expressing motion of the video camera which picked up an arbitrary image signal, from the arbitrary image signal, based on (a) sensor signal expressing physical motion of the video camera, which is obtained by a sensor for detecting physical motion, at the same time when the desired image signal was picked up, and (b) the camera motion estimation information, wherein a motion signal causes a user to experience a live feeling as if said user were riding in an image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on (1) one or more motion vectors in the desired image signal and (2) the camera motion estimation information of an inputted image signal is generated based on a motion vector in the input image signal.

25. An information processing method comprising the steps of:

detecting from an inputted image signal camera motion information with respect to the inputted image signal; and

detecting camera motion prediction information with respect to the inputted image signal, based on (a) the detected camera motion information and (b) relationship information generated by learning based on (1) camera motion estimation information that expresses motion of a video camera, which is detected by a desired image signal picked up by the video camera, and (2) camera motion information that expresses physical motion of the video camera, obtained by a sensor for detecting physical motion at the same time that the desired image signal was picked up, wherein a motion signal causes a user to experience a live feeling as if said user were riding in an image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on (i) one or more motion vectors in the desired image signal and (ii) the camera motion estimation information of the inputted image signal is generated based on a motion vector in the input image signal.

26. A non-transitory computer program recording medium encoded with a computer program for controlling a computer to execute information processing by:

detecting from an inputted image signal camera motion information with respect to the inputted image signal; and

detecting camera motion prediction information with respect to the inputted image signal, based on (a) the detected camera motion information and (b) relationship information generated by learning based on (1) camera motion estimation information that expresses motion of a video camera, which is detected by a desired image signal picked up by the video camera, and (2) camera motion information that expresses physical motion of the video camera, obtained by a sensor for detecting physical motion at the same time that the desired image signal was picked, wherein a motion signal causes a user to experience a live feeling as if said user were riding in an image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on (i) one or more motion vectors in the desired image signal and (ii) the camera motion estimation information of the inputted image signal is generated based on a motion vector in the input image signal.