Method and apparatus for use in determining an activity level of a user in relation to a system

• US 7,782,297 B2
• Filed: 05/08/2006
• Issued: 08/24/2010
• Est. Priority Date: 07/27/2002
• Status: Expired due to Term

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America (Sony Group Corp.)
• Inventors: Mao, Xiadong, Marks, Richard, Zalewski, Gary
• Primary Examiner(s): Tran; Henry N
• Application Number:
  US11/382,258
• Publication Number:
  US 20070015558A1
• Time in Patent Office: 1,569 Days
• Field of Search: 345156-158, 345/7, 345/8, 463/36, 463/37, 463/38, 463/46, 463/47, 463/52, 715798-801, 713/300, 713/320, 713/323, 348/77, 348/94, 348/95, 348/115, 348/135, 348/141, 348/142, 348/207.1, 348/207.11, 348/208.1, 348/208.2, 348/211.99
• US Class Current: 345/156
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/48 Starting a game, e.g. activating a game device or waiting for other players to join a multiplayer session
• CPC Class Codes: A63F 2300/1006 having additional degrees of freedom
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6072 of an input signal, e.g. pitch and rhythm extraction, voice recognition

Abstract

Methods and techniques for use in the operation of a game apparatus or other system for determining an activity level of a user in relation to the game apparatus or other system. A machine-readable medium having embodied thereon instructions for performing such methods and techniques is also included, as well as an example system for implementation.

Claims

1. A method for use in operation of a game apparatus for determining a lack of user activity in relation to the game apparatus, comprising the steps of:

obtaining information usable to determine an activity level of a user of the game apparatus, wherein the information comprises at least two of inertial tracking information, image tracking information, and acoustic tracking information;

mixing the at least two of inertial tracking information, image tracking information, and acoustic tracking information to generate position and orientation information;

processing the position and orientation information to determine whether the user is currently inactive; and

controlling the operation of the game apparatus when the step of processing the position and orientation information determines that the user is currently inactive.

2. The method as claimed in claim 1, wherein the step of processing the position and orientation information comprises determining that the user is currently inactive so long as the activity of the user does not exceed a threshold.

3. The method as claimed in claim 2, further comprising modifying the threshold during the step of processing the position and orientation information.

4. The method as claimed in claim 1, wherein the controlling the game apparatus, when the step of processing the position and orientation information determines that the user is currently inactive, comprises suspending execution of a game.

5. The method as claimed in claim 1, wherein the step of processing the position and orientation information is performed with respect to the activity of individual ones of a plurality of users to identify one or more individual users of the plurality of users who is currently inactive, and the step of controlling the operation of the game apparatus includes identifying a particular user of the plurality of users who is currently inactive.

6. The method as claimed in claim 5, further comprising determining that all of a predetermined set of the plurality of users are currently inactive.

7. The method as claimed in claim 1, wherein the information usable to determine the activity level of the user of the game apparatus includes information relating to at least one of a movement of at least a portion of the user'"'"'s body, a position of the user'"'"'s body, an orientation of at least a portion of the user'"'"'s body, or an acoustic level in a vicinity of a screen upon which a progress of a game is displayable by the game apparatus.

8. The method as claimed in claim 6, wherein the step of obtaining the information usable to determine the activity level of the user of the game apparatus includes obtaining the image tracking information from an image capture device in a vicinity of a screen upon which a progress of a game is displayable by the game apparatus, where the image tracking information is one of the at least two types of information.

9. The method as claimed in claim 8, wherein the step of obtaining the information usable to determine the activity level of the user of the game apparatus includes obtaining information representative of a position of the at least one photonically detectable (“

PD”

) element within an image captured by the image capture device, the PD element being included in an article worn by the user, the PD element being distinctive by at least one of brightness, color, texture, pattern, reflectivity or illumination intensity, and the step of processing comprises determining that the user is currently inactive when a change in the position of the at least one PD element over an interval of time fails to exceed a predetermined threshold.

10. The method as claimed in claim 9, wherein the at least one PD element includes a light source.

11. The method as claimed in claim 10, wherein the at least one PD element includes a light emitting diode (“

LED”

).

12. The method as claimed in claim 9, wherein the article includes at least two PD elements, wherein the step of obtaining the information usable to determine the activity level of the user of the game apparatus includes capturing an orientation of the article relative to an x-axis of a plane of the image and a y-axis of the plane of the image.

13. The method as claimed in claim 9, wherein the article includes three PD elements defining a triangle, wherein the step of obtaining the information usable to determine an activity level of the user of the game apparatus includes capturing an orientation of the game controller further in relation to a z-axis, the z-axis being perpendicular to the plane of the image.

14. The method as claimed in claim 9, wherein the article includes at least four PD elements defining a rectangle, wherein the step of obtaining the information usable to determine the activity level of the user of the game apparatus includes capturing an orientation of the game controller further in relation to a z-axis, the z-axis being perpendicular to the plane of the image.

15. The method as claimed in claim 13 or claim 14, wherein the step of processing further includes determining an orientation of the article including orientation data relating to at least one of pitch, yaw or roll and comparing the orientation data to predetermined orientation data to determine whether the user is currently active.

16. The method as claimed in claim 15, wherein the orientation data relates to pitch, yaw and roll of the article.

17. The method as claimed in any one of claims 1 through 5, wherein the step of obtaining the information usable to determine the activity level of the user of the game apparatus includes sampling information obtained by an inertial sensor incorporated in an article worn by the user and the step of processing the position and orientation information includes processing the position and orientation information to determine that a movement of the predetermined object does not exceed a threshold within a predetermined interval of time, where the inertial tracking information is one of the at least two types of information.

18. The method as claimed in claim 17,

wherein the step of obtaining the information usable to determine the activity level of the user of the game apparatus includes obtaining information from an image capture device in a vicinity of a screen upon which a progress of a game is displayable by the game apparatus.

19. The method as claimed in claim 17, wherein the step of obtaining the information usable to determine the activity level of the user of the game apparatus includes obtaining information usable to determine acceleration of the article and the step of processing the position and orientation information includes determining an acceleration of the article.

20. The method as claimed in claim 19, wherein the step of processing the position and orientation information includes determining a velocity of the article by integrating acceleration values obtained from the position and orientation information over an interval of time, and determining that the user is currently inactive when the velocity determined by the processing does not exceed a threshold velocity value.

21. The method as claimed in claim 20, wherein the step of processing the position and orientation information includes determining a displacement of the article by first integrating acceleration values obtained from the information over an interval of time and integrating the result of the first integrating over the interval of time, and determining that the user is currently inactive when the displacement determined by the processing does not exceed a threshold displacement value.

22. The method as claimed in any one of claims 1, 2, 3 or 5, wherein the step of obtaining the information usable to determine the activity level of the user of the game apparatus includes sampling information obtained by an acoustic transducer in the vicinity of the user and the step of processing the position and orientation information includes determining that a user is currently inactive when the acoustic tracking information obtained by the acoustic transducer in the vicinity of the user does not satisfy a predetermined criterion.

23. The method as claimed in claim 22, further comprising displaying a progress of a game upon a screen, wherein the acoustic tracking information is obtained by an acoustic transducer located in a vicinity of the screen.

24. The method as claimed in claim 4, further comprising displaying a progress of a game upon a screen, the step of obtaining the information usable to determine the activity level of the user of the game apparatus comprises obtaining the acoustic tracking information obtained by an acoustic transducer located in a vicinity of the screen, and the step of processing the position and orientation information includes determining that a particular user of the plurality of users is currently inactive when the acoustic tracking information obtained by the acoustic transducer in a vicinity of the screen does not satisfy a predetermined criterion specific to the particular user.

25. The method as claimed in claim 23, wherein the predetermined criterion includes at least one criterion selected from the group consisting of pitch, audio pulse frequency, audio pulse shape, audio pulse duty cycle, beat frequency, spectral signature including a spectral content characteristic of a distinctive sound, spectral signature characteristic of a particular user'"'"'s voice, a time-varying signature characteristic of a string of one or more phonemes concatenated together and recognizable through speech recognition.

26. An apparatus for use in controlling a progress of a game, the apparatus comprising:

a memory that stores a game program; and

a processor operable to access the game program from the memory, and to execute the game program to enable a game to be interactively played by a user in accordance with interactions of a user, the processor being operable to;

obtain information usable to determine an activity level of a user of the game apparatus, wherein the information comprises at least two of inertial tracking information, image tracking information, and acoustic tracking information;

mixing the at least two of inertial tracking information, image tracking information, and acoustic tracking information to generate position and orientation information;

process the position and orientation information to determine whether the user is currently inactive; and

controlling the operation of the game apparatus when the processing the position and orientation information determines that the user is currently inactive.

27. The apparatus of claim 26, wherein the processor is further configured to determine, in response to the processing of the position and orientation information, that the user is currently inactive so long as an activity of the user does not exceed a threshold.

28. The apparatus of claim 27, wherein the processor is further configured, in response to the processing of the position and orientation information, to suspend execution of a game when it is determined that the user is currently inactive.

29. The apparatus of claim 28, wherein the processor, when controlling the operation of the game apparatus and when the processing the position and orientation information determines that the user is currently inactive, is further configured to processes the position and orientation information with respect to the activity of individual ones of a plurality of users to identify one or more individual users of the plurality of users who is currently inactive, and identify a particular user of the plurality of users who is currently inactive.

30. The apparatus of claim 26, further comprising:

an image capture device coupled with the processor; and

a game controller communicationally coupled with the processor, the game controller comprising;

a body and at least one photonically detectable (“

PD”

) element assembled with the body, the PD element being distinguishable from the body by at least one of brightness, color, texture, pattern, reflectivity or illumination intensity, wherein the processor in obtaining the information usable to determine the activity level of the user obtains image tracking information representative of a position of the at least one PD element within an image captured by the image capture device.

31. The apparatus of claim 30, further comprising:

wherein the obtaining the image tracking information comprises receiving the image recorded by the image capture device; and

the processing the image tracking information comprises determining a position of the PD element within the image, and determining positions of the PD element at different points in time to determine whether the user is currently inactive in relation to the game controller.

32. The apparatus of claim 30, wherein the body of the game controller further comprises:

a forward section to be oriented towards a screen when a progress of the game controlled by the processor in accordance with one or more signals received from the game controller is displayed upon the screen; and

at least one input device assembled with the body, the input device manipulable by a user to register an input from the user.

33. The apparatus of claim 30, wherein:

the processor is further configured to determine a position of the photonically detectable element within the image recordable by the image capture device when the forward section of the game controller is oriented at least generally towards the screen, wherein positions of the PD element at different points in time quantify movement of the body in space.