INFORMATION PROCESSING APPARATUS

• US 20100045818A1
• Filed: 11/04/2009
• Published: 02/25/2010
• Est. Priority Date: 05/11/1999
• Status: Active Grant

• Current Assignee: Satoshi Teshigawara, Tetsujiro Kondo, Tetsushi Kokubo, Yuji Okumura
• Original Assignee: Satoshi Teshigawara, Tetsujiro Kondo, Tetsushi Kokubo, Yuji Okumura
• Inventors: Kokubo, Tetsushi, Kondo, Tetsujiro, Okumura, Yuji, Teshigawara, Satoshi
• Granted Patent: US 8,619,197 B2
• Time in Patent Office: Days
• US Class Current: 348/222.100
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 9/00 Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131
• CPC Class Codes: G09B 9/05 the view from a vehicle being simulated G09B9/052, G09B9/058 take precedence
• CPC Class Codes: G09B 9/12 Motion systems for aircraft simulators

Abstract

Relationship information is previously generated and stored by learning based on camera motion estimation information expressing motion of a video camera, which is detected by a desired image signal picked up by the video camera, and camera motion information expressing physical motion of the video camera, which was obtained by a sensor for detecting physical motion at the same time when the desired image signal was picked up by the video camera. Camera motion estimation information with respect to an inputted image signal is detected from the inputted image signal. Camera motion prediction information with respect to the inputted image signal is generated, based on the camera motion estimation information detected and the relationship information. Based on the camera motion prediction information, a vibration signal for vibrating an object is generated. As a result, vibration data can be easily generated at low costs, based on existing video assets.

Claims

1. (canceled)

2. An apparatus comprising:

memory means for storing relationship information generated by learning based on camera motion estimation information for learning expressing motion of a video camera, which is detected by a desired image signal picked up by the video camera, and camera motion information for learning expressing physical motion of the video camera, which was obtained by a sensor for detecting physical motion at the same time when the desired image signal was picked up by the video camera;

camera motion estimation information detection means for detecting camera motion estimation information with respect to an inputted image signal, from the inputted image signal;

camera motion prediction information generation means for generating camera motion prediction information with respect to the inputted image signal, based on the camera motion estimation information of the inputted image signal detected by the camera motion estimation information detection means and the relationship information; and

vibration signal generation means for generating a vibration signal for vibrating an object,

wherein the desired image signal is a signal obtained for learning processing that is performed automatically by a learning section of said apparatus.

3. The apparatus according to claim 2, further comprising drive means for driving the object, based on the vibration signal.

4. The apparatus according to claim 2, further comprising a chair as the object.

5. The apparatus according to claim 4, further comprising display means for displaying the input image signal in synchronization with the drive means.

6-13. -13. (canceled)

14. A learning apparatus comprising:

camera motion estimation information detection means for detecting camera motion estimation information from a desired image signal picked up by a video camera; and

coefficient generation means for generating a conversion coefficient for generating camera motion prediction information for learning expressing motion of the video camera which picked up an arbitrary image signal, from the arbitrary image signal, based on sensor signal expressing physical motion of the video camera, which is obtained by a sensor for detecting physical motion, at the same time when the desired image signal was picked up, and the camera motion estimation information;

wherein the desired image signal is a signal obtained for learning processing that is performed automatically by a learning section of said apparatus.

15. The apparatus according to claim 14, wherein the camera motion estimation includes a motion vector detection section for detecting a motion vector from the inputted image signal, a motion center detection section for detecting a motion center, based on the motion vector, and a camera motion estimation information detection section for detecting the camera motion estimation information, based on the motion vector and the motion center.

18. The apparatus according to claim 15, wherein the motion vector detection section detects the motion vector, with respect to a plurality of preset representative points of the inputted image signal.

19. The apparatus according to claim 15, comprising an acceleration sensor as the sensor.

20. The apparatus according to claim 15, comprising an angular speed sensor as the sensor.

21. The apparatus according to claim 15,

wherein the camera motion estimation information detection means generates the camera motion estimation information which is constructed by information of plural kinds of motions.

16-17. -17. (canceled)

22-34. -34. (canceled)

35. A learning method comprising the steps of:

detecting camera motion estimation information from a desired image signal picked up by a video camera; and

generating a conversion coefficient for generating camera motion prediction information for learning expressing motion of the video camera which picked up an arbitrary image signal, from the arbitrary image signal, based on sensor signal expressing physical motion of the video camera, which is obtained by a sensor for detecting physical motion, at the same time when the desired image signal was picked up, and the camera motion estimation information;

wherein the desired image signal is a signal obtained for learning processing that is performed automatically by a learning section of said apparatus.

36. The method according to claim 35, wherein the step of detecting the camera motion estimation information includes a step of detecting a motion vector from the inputted image signal, a step of detecting a motion center, based on the motion vector, and a step of detecting the camera motion estimation information, based on the motion vector and the motion center.

39. The method according to claim 36, wherein in the step of detecting the motion vector, the motion vector is detected with respect to a plurality of preset representative points of the inputted image signal.

40. The method according to claim 36, wherein in the step of generating the conversion coefficient, the conversion coefficient is generated, based on camera motion information expressing physical motion of the video camera, which was obtained by an acceleration sensor as the sensor.

41. The method according to claim 36, wherein in the step of generating the conversion coefficient, the conversion coefficient is generated, based on camera motion information expressing physical motion of the video camera, which was obtained by an angular speed sensor as the sensor.

42. The method according to claim 36,

wherein in the step of detecting the camera motion estimation information, the camera motion estimation information which is constructed by information of plural kinds of motions.

37-38. -38. (canceled)

43. (canceled)

44. A computer-readable storage medium storing a program that when executed by a computer causes the computer to execute information processing, the program comprising the steps of:

generating relationship information generated by learning based on camera motion estimation information for learning expressing motion of a video camera, which is detected by a desired image signal picked up by the video camera, and camera motion information for learning expressing physical motion of the video camera, which was obtained by a sensor for detecting physical motion at the same time when the desired image signal was picked up by the video camera;

detecting camera motion estimation information with respect to an inputted image signal, from the inputted image signal;

generating camera motion prediction information with respect to the inputted image signal, based on the camera motion estimation information of the inputted image signal detected and the relationship information; and

generating a vibration signal for vibrating an object

wherein the desired image signal is a signal obtained for learning processing that is performed automatically by a learning section of a learning apparatus.

45. The medium according to claim 44, wherein the program further comprises a step of driving the object, based on the vibration signal.

46. The medium according to claim 45, wherein a chair is used as the object in the step of driving the object.

47. The medium according to claim 45, wherein the program further comprises a step of displaying the input image signal in synchronization with the driving.

48-55. -55. (canceled)

56. A recording medium which records a program for letting a computer execute learning processing, the program comprising the steps of:

detecting camera motion estimation information from a desired image signal picked up by a video camera; and

generating a conversion coefficient for generating camera motion prediction information for learning expressing motion of the video camera which picked up an arbitrary image signal, from the arbitrary image signal, based on sensor signal expressing physical motion of the video camera, which is obtained by a sensor for detecting physical motion, at the same time when the desired image signal was picked up, and the camera motion estimation information;

wherein the desired image signal is a signal obtained for learning processing that is performed automatically by a learning section of said apparatus.

57. The medium according to claim 56, wherein the step of detecting the camera motion estimation information includes a step of detecting a motion vector from the inputted image signal, a step of detecting a motion center, based on the motion vector, and a step of detecting the camera motion estimation information, based on the motion vector and the motion center.

60. The medium according to claim 56, wherein in the step of detecting the motion vector, the motion vector is detected with respect to a plurality of preset representative points of the inputted image signal.

61. The medium according to claim 56, wherein in the step of generating the conversion coefficient, the conversion coefficient is generated, based on camera motion information expressing physical motion of the video camera, which was obtained by an acceleration sensor as the sensor.

62. The medium according to claim 56, wherein in the step of generating the conversion coefficient, the conversion coefficient is generated, based on camera motion information expressing physical motion of the video camera, which was obtained by an angular speed sensor as the sensor.

63. The medium according to claim 56,

wherein in the step of generating the camera motion estimation information, camera motion estimation information which is constructed by information of plural kinds of motions.

58-59. -59. (canceled)

64. An apparatus comprising:

memory means for storing relationship information generated by learning based on camera motion estimation information for learning expressing motion of a video camera, which is detected by a desired image signal picked up by the video camera, and camera motion information for learning expressing physical motion of the video camera, which was obtained by a sensor for detecting physical motion at the same time when the desired image signal was picked up by the video camera;

camera motion estimation information detection means for detecting camera motion estimation information with respect to an inputted image signal, from the inputted image signal; and

camera motion prediction information generation means for generating camera motion prediction information with respect to the inputted image signal, based on the camera motion estimation information of the inputted image signal detected by the camera motion estimation information detection means and the relationship information, wherein the motion signal causes a user to experience a live feeling as if said user were riding in said image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on one or more motion vectors in the desired image signal and the camera motion estimation information of the inputted image signal is generated based on a motion vector in the input image signal.

65. A learning apparatus comprising:

camera motion estimation information detection means for detecting camera motion estimation information from a desired image signal picked up by a video camera; and

coefficient generation means for generating a conversion coefficient for generating camera motion prediction information expressing motion of the video camera which picked up an arbitrary image signal, from the arbitrary image signal, based on sensor signal expressing physical motion of the video camera, which is obtained by a sensor for detecting physical motion, at the same time when the desired image signal was picked up, and the camera motion estimation information, wherein the motion signal causes a user to experience a live feeling as if said user were riding in said image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on one or more motion vectors in the desired image signal and the camera motion estimation information of an inputted image signal is generated based on a motion vector in the input image signal.

66. An information processing method comprising the steps of:

generating relationship information generated by learning based on camera motion estimation information expressing motion of a video camera, which is detected by a desired image signal picked up by the video camera, and camera motion information expressing physical motion of the video camera, which was obtained by a sensor for detecting physical motion at the same time when the desired image signal was picked up by the video camera;

detecting camera motion estimation information with respect to an inputted image signal, from the inputted image signal; and

generating camera motion prediction information with respect to the inputted image signal, based on the camera motion estimation information detected and the relationship information, wherein the motion signal causes a user to experience a live feeling as if said user were riding in said image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on one or more motion vectors in the desired image signal and the camera motion estimation information of the inputted image signal is generated based on a motion vector in the input image signal.

67. A learning method comprising the steps of:

detecting camera motion estimation information from a desired image signal picked up by a video camera; and

generating a conversion coefficient for generating camera motion prediction information expressing motion of the video camera which picked up an arbitrary image signal, from the arbitrary image signal, based on sensor signal expressing physical motion of the video camera, which is obtained by a sensor for detecting physical motion, at the same time when the desired image signal was picked up, and the camera motion estimation information, wherein the motion signal causes a user to experience a live feeling as if said user were riding in said image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on one or more motion vectors in the desired image signal and the camera motion estimation information of an inputted image signal is generated based on a motion vector in the input image signal.

68. A program recording medium which records a program for letting a computer execute information processing, the program comprising the steps of:

generating relationship information generated by learning based on camera motion estimation information expressing motion of a video camera, which is detected by a desired image signal picked up by the video camera, and camera motion information expressing physical motion of the video camera, which was obtained by a sensor for detecting physical motion at the same time when the desired image signal was picked up by the video camera;

detecting camera motion estimation information with respect to an inputted image signal, from the inputted image signal; and

generating camera motion prediction information with respect to the inputted image signal, based on the camera motion estimation information detected and the relationship information, wherein the motion signal causes a user to experience a live feeling as if said user were riding in said image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on one or more motion vectors in the desired image signal and the camera motion estimation information of the inputted image signal is generated based on a motion vector in the input image signal.

69. A recording medium which records a program for letting a computer execute learning processing, the program comprising the steps of:

detecting camera motion estimation information from a desired image signal picked up by a video camera; and

generating a conversion coefficient for generating camera motion prediction information expressing motion of the video camera which picked up an arbitrary image signal, from the arbitrary image signal, based on sensor signal expressing physical motion of the video camera, which is obtained by a sensor for detecting physical motion, at the same time when the desired image signal was picked up, and the camera motion estimation information, wherein the motion signal causes a user to experience a live feeling as if said user were riding in said image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on one or more motion vectors in the desired image signal and the camera motion estimation information of an inputted image signal is generated based on a motion vector in the input image signal.

70. An information processing method comprising the steps of:

detecting from an inputted image signal camera motion information with respect to the inputted image signal; and

detecting camera motion prediction information with respect to the inputted image signal, based on the detected camera motion information and relationship information generated by learning based on camera motion estimation information that expresses motion of a video camera, which is detected by a desired image signal picked up by the video camera, and camera motion information that expresses physical motion of the video camera, obtained by a sensor for detecting physical motion at the same time that the desired image signal was picked up, wherein the motion signal causes a user to experience a live feeling as if said user were riding in said image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on one or more motion vectors in the desired image signal and the camera motion estimation information of the inputted image signal is generated based on a motion vector in the input image signal.

71. A program recording medium on which is recorded a program for controlling a computer to execute information processing by:

detecting from an inputted image signal camera motion information with respect to the inputted image signal; and

detecting camera motion prediction information with respect to the inputted image signal, based on the detected camera motion information and relationship information generated by learning based on camera motion estimation information that expresses motion of a video camera, which is detected by a desired image signal picked up by the video camera, and camera motion information that expresses physical motion of the video camera, obtained by a sensor for detecting physical motion at the same time that the desired image signal was picked, wherein the motion signal causes a user to experience a live feeling as if said user were riding in said image picked up by said video camera;

wherein the camera motion estimation information for learning is generated based on one or more motion vectors in the desired image signal and the camera motion estimation information of the inputted image signal is generated based on a motion vector in the input image signal.

72. An apparatus comprising:

memory means for storing relationship information generated by learning based on camera motion estimation information expressing motion of a video camera, which is detected by a desired image signal picked up by the video camera, and camera motion information expressing physical motion of the video camera, which was obtained by a sensor for detecting physical motion at the same time when the desired image signal was picked up by the video camera;

camera motion estimation information detection means for detecting camera motion estimation information with respect to an inputted image signal, from the inputted image signal;

camera motion prediction information generation means for generating camera motion prediction information with respect to the inputted image signal, based on the camera motion estimation information detected by the camera motion estimation information detection means and the relationship information;

wherein the desired image signal is a signal for learning;

camera motion prediction information generation means for generating camera motion prediction information with respect to an inputted image signal, based on the camera motion estimation information of the inputted image signal detected by the camera motion estimation information detection means and relationship information; and

vibration signal generation means for generating a vibration signal for vibrating an object, by calculating camera motion prediction information.

73. An information processing method comprising the steps of:

generating relationship information generated by learning based on camera motion estimation information expressing motion of a video camera, which is detected by a desired image signal picked up by the video camera, and camera motion information expressing physical motion of the video camera, which was obtained by a sensor for detecting physical motion at the same time when the desired image signal was picked up by the video camera;

detecting camera motion estimation information with respect to an inputted image signal, from the inputted image signal;

generating camera motion prediction information with respect to the inputted image signal, based on the camera motion estimation information detected and the relationship information;

wherein the desired image signal is a signal for learning;

generating camera motion prediction information with respect to an inputted image signal, based on camera motion estimation information of the inputted image signal detected by camera motion estimation information and relationship information; and

generating a vibration signal for vibrating an object, based on the vibration signal.

74. A program recording medium which records a program for letting a computer execute information processing, the program comprising the steps of:

generating relationship information generated by learning based on camera motion estimation information expressing motion of a video camera, which is detected by a desired image signal picked up by the video camera, and camera motion information expressing physical motion of the video camera, which was obtained by a sensor for detecting physical motion at the same time when the desired image signal was picked up by the video camera;

detecting camera motion estimation information with respect to an inputted image signal, from the inputted image signal;

generating camera motion prediction information with respect to the inputted image signal, based on the camera motion estimation information detected and the relationship information;

wherein the desired image signal is a signal for learning;

generating camera motion prediction information with respect to an inputted image signal, based on camera motion estimation information of the inputted image signal detected by the camera motion estimation information detection means and relationship information; and

generating a vibration signal for vibrating an object, based on the vibration signal.