Interactive video based games using objects sensed by tv cameras

• US 8,068,095 B2
• Filed: 11/08/2010
• Issued: 11/29/2011
• Est. Priority Date: 08/22/1997
• Status: Expired due to Fees

• Current Assignee: Motion Games LLC
• Original Assignee: Motion Games LLC
• Inventors: Pryor, Timothy R.
• Primary Examiner(s): ABDULSELAM, ABBAS I
• Application Number:
  US12/941,304
• Publication Number:
  US 20110059798A1
• Time in Patent Office: 386 Days
• Field of Search: 345/158, 345/173, 345/204, 345/419, 463/37
• US Class Current: 345/157
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06F 2203/0331 Finger worn pointing device
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method and apparatus for interactive TV camera based games in which position or orientation of points on a player or of an object held by a player are determined and used to control a video display. Both single camera and stereo camera pair based embodiments are disclosed, preferably using stereo photogrammetry where multi-degree of freedom information is desired. Large video displays, preferably life-size may be used where utmost realism of the game experience is desired.

Claims

1. An interactive video gaming system using changes in position of an object held by a player in order to provide changing video game images on a display, comprising:

at least one object adapted to be held by at least one player of the game;

an electro-optical sensing system having an output providing information concerning at least one feature of said at least one object; and

computer means for analyzing said output and determining the location of one or more object features to sub-pixel resolution, said features recognized by at least one characteristic, and for further determining a change in said displayed video game image using said location.

2. An interactive video gaming system according to claim 1, wherein said location includes the range of said feature from said electro-optical sensing system.

3. An interactive video gaming system according to claim 1, wherein said feature location is determined in 3 dimensions.

4. An interactive video gaming system according to claim 3, wherein said location is determined using a single TV camera.

5. An interactive video gaming system according to claim 1, wherein said electro-optical system has a field of view encompassing a range of player movements needed to play the game.

6. An interactive video gaming system according to claim 1, wherein the angle of view of said electro-optical system is greater than 30 degrees.

7. An interactive video gaming system according to claim 1, wherein the speed of said electro-optical system is at least 30 Hz.

8. An interactive video gaming system according to claim 1, wherein the image of said feature is round.

9. An interactive video gaming system according to claim 1, wherein said characteristic is feature shape.

10. An interactive video gaming system according to claim 1, wherein said feature is round.

11. An interactive video gaming system according to claim 1, wherein a plurality of characteristics are used to recognize said feature.

12. An interactive video gaming system according to claim 1, wherein said feature is a high contrast.

13. An interactive video gaming system according to claim 1, wherein said feature is self-luminous.

14. An interactive video gaming system according to claim 13, wherein said feature is a battery driven LED.

15. An interactive video gaming system according to claim 1, wherein said system further contains an interrupt member for use by a player to signal the computer.

16. An interactive video gaming system according to claim 15, wherein said interrupt member is provided on said object.

17. An interactive video gaming system according to claim 15, wherein said interrupt member employs voice recognition.

18. An interactive video gaming system according to claim 15, wherein said interrupt member is sensed by said sensing system.

19. An interactive video gaming system according to claim 15, wherein said interrupt member is a trigger.

20. An interactive video gaming system according to claim 1, wherein said display is in 3 dimensions.

21. An interactive video gaming system according to claim 1, wherein said electro-optical system is comprised by a TV camera.

22. An interactive video gaming system according to claim 21, wherein said camera further obtains an image of at least one player which may be displayed on said display.

23. An interactive video gaming system according to claim 1, wherein said electro-optical system operates by triangulation.

24. An interactive video gaming system according to claim 1, wherein said electro-optical system employs a laser light source.

25. An interactive video gaming system according to claim 1, wherein said electro-optical system is comprised a stereo pair of cameras.

26. An interactive video gaming system according to claim 1, wherein said object is a gun.

27. An interactive video gaming system according to claim 1, wherein said object is a doll.

28. An interactive video gaming system according to claim 1, wherein said object is a sword.

29. An interactive video gaming system according to claim 1, wherein said object is an airplane.

30. An interactive video gaming system according to claim 1, wherein said object is a steering wheel.

31. An interactive video gaming system using changes in position of an object held by a player in order to provide changing video game images on a display, comprising:

at least one object adapted to be held by at least one player of the game;

an electro-optical sensing system having an output providing information concerning at least one feature of said at least one object;

computer means for analyzing said output and determining the location of one or more object features, said features recognized by at least one characteristic, and for further determining a change in said displayed video game image using said location; and

an internet connection over which changed game information can be transmitted to a display of a remotely located player.

32. An interactive video gaming system according to claim 31, wherein said location includes the range of said feature from said electro-optical sensing system.

33. An interactive video gaming system according to claim 31, wherein said feature location is determined in 3 dimensions.

34. An interactive video gaming system according to claim 33, wherein said location is determined using a single TV camera.

35. An interactive video gaming system according to claim 31, wherein said electro-optical system has a field of view encompassing a range of player movements needed to play the game.

36. An interactive video gaming system according to claim 31, wherein the angle of view of said electro-optical system is greater than 30 degrees.

37. An interactive video gaming system according to claim 31, wherein the speed of said electro-optical system is at least 30 Hz.

38. An interactive video gaming system according to claim 31, wherein said location of said feature is determined to sub-pixel resolution.

39. An interactive video gaming system according to claim 31, wherein the image of said feature is round.

40. An interactive video gaming system according to claim 31, wherein said characteristic is feature shape.

41. An interactive video gaming system according to claim 31, wherein said feature is round.

42. An interactive video gaming system according to claim 31, wherein a plurality of characteristics are used to recognize said feature.

43. An interactive video gaming system according to claim 31, wherein said feature is a high contrast.

44. An interactive video gaming system according to claim 31, wherein said feature is self-luminous.

45. An interactive video gaming system according to claim 44, wherein said feature is a battery driven LED.

46. An interactive video gaming system according to claim 31, wherein said system further contains an interrupt member for use by a player to signal the computer.

47. An interactive video gaming system according to claim 46, wherein said interrupt member is provided on said object.

48. An interactive video gaming system according to claim 46, wherein said interrupt member employs voice recognition.

49. An interactive video gaming system according to claim 46, wherein said interrupt member is sensed by said sensing system.

50. An interactive video gaming system according to claim 46, wherein said interrupt member is a trigger.

51. An interactive video gaming system according to claim 31, wherein said display is in 3 dimensions.

52. An interactive video gaming system according to claim 31, wherein said electro-optical system is comprised by a TV camera.

53. An interactive video gaming system according to claim 52, wherein said camera further obtains an image of at least one player which may be displayed on said display.

54. An interactive video gaming system according to claim 31, wherein said electro-optical system operates by triangulation.

55. An interactive video gaming system according to claim 31, wherein said electro-optical system employs a laser light source.

56. An interactive video gaming system according to claim 31, wherein said electro-optical system is comprised a stereo pair of cameras.

57. An interactive video gaming system according to claim 31, wherein said object is a gun.

58. An interactive video gaming system according to claim 31, wherein said object is a doll.

59. An interactive video gaming system according to claim 31, wherein said object is a sword.

60. An interactive video gaming system according to claim 31, wherein said object is an airplane.

61. An interactive video gaming system according to claim 31, wherein said object is a steering wheel.

62. A method for interactive video gaming, comprising the steps of:

providing a computer;

providing a display on which changing game images are displayed, said display controlled by said computer;

providing an electro-optical sensing system to observe a field of view encompassing the region in which at least one player plays the game;

providing an object for use by a player in playing the game;

electro-optically sensing at least one feature of at least one object adapted to be held by a player of the game to provide information concerning said object; and

analyzing said information to determine a change in location of said feature at least 30 times per second, and using said changed location, changing the game image displayed.

63. A method according to claim 62, wherein said electro-optical sensing system employs at least one TV camera.

64. A method according to claim 62, wherein a large display of 42 inches diagonal or greater provides realism to the game.

65. A method according to claim 64, wherein a stereo pair of cameras is employed.

66. A method according to claim 62, wherein said feature is recognized by its shape.

67. A method according to claim 62, wherein said feature is recognized by its size.

68. A method according to claim 62, wherein said feature is recognized by its color.

69. A method according to claim 62, wherein said feature is recognized by its brightness.

70. A method according to claim 62, wherein said feature is recognized by its contrast.

71. A method according to claim 62, wherein said feature is recognized by its size.

72. A method according to claim 71, wherein said feature is recognized by its diameter or area.

73. A method according to claim 62, wherein said feature is recognized by a plurality of characteristics.

74. A method according to claim 62, wherein said location includes the range of said object.

75. A method according to claim 62, wherein a centroid of said feature is determined in order to determine its location.

76. A method according to claim 62, wherein said location is determined in three dimensions.

77. A method according to claim 62, wherein a plurality of players play said game and said field of view encompasses the region encompassing the play of said plurality of players.

78. A method according to claim 62, wherein said computer further analyses said information to determine an orientation of said object.

79. A method according to claim 62, wherein information concerning background conditions from current information concerning said object.

80. A method according to claim 62, wherein said computer further analyses said information to determine an orientation of said object.

81. A method according to claim 62, wherein knowledge of previous locations is used to predict a current location.

82. A method according to claim 62, wherein information is processed in regions in which motion has occurred.

83. A method according to claim 62, wherein a player holds an object in each hand.

84. A method according to claim 62, wherein there are at least two players and each holds an object.

85. A method according to claim 62, wherein a player holds an object using both hands.