Method and system for applying gearing effects to visual tracking

• US 10,406,433 B2
• Filed: 09/28/2018
• Issued: 09/10/2019
• Est. Priority Date: 07/27/2002
• Status: Expired due to Term

• Current Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Zalewski, Gary M., Marks, Richard, Mao, Crusoe Xiaodong
• Primary Examiner(s): Amadiz, Rodney
• Application Number:
  US16/147,365
• Publication Number:
  US 20190038977A1
• Time in Patent Office: 347 Days
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/245 specially adapted to a particular type of game, e.g. steering wheels
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 13/57 Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game A63F13/56 takes precedence
• CPC Class Codes: A63F 13/63 by the player, e.g. authoring using a level editor
• CPC Class Codes: A63F 2300/1006 having additional degrees of freedom
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1062 being specially adapted to a type of game, e.g. steering wheel
• CPC Class Codes: A63F 2300/1081 Input via voice recognition
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/308 Details of the user interface
• CPC Class Codes: A63F 2300/6018 where the game content is authored by the player, e.g. level editor or by game device at runtime, e.g. level is created from music data on CD
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6072 : of an input signal, e.g. pitch and rhythm extraction, voice recognition
• CPC Class Codes: A63F 2300/64 : for computing dynamical parameters of game objects, e.g. motion determination or computation of frictional forces for a virtual car
• CPC Class Codes: A63F 2300/66 : for rendering three dimensional images

Abstract

A system is provided, including the following: an input device, the input device including, a light emitting diode (LED) array; an image capture device configured to capture images of an interactive environment in which the input device is disposed, the image capture device configured to generate captured image data that is processed to determine a movement of the input device in the interactive environment; wherein a gearing is established that adjusts an amount by which the movement of the input device is mapped to movement of an image that is rendered to a display, the gearing changing to different settings during the movement of the image as the movement of the image is rendered to the display.

Claims

1. A method, comprising:

receiving captured image data from an image capture device configured to capture images of an interactive environment in which an input device is disposed, the input device having a light emitting diode (LED) array;

processing the captured image data to determine a movement of the input device in the interactive environment;

establishing a gearing that adjusts an amount by which the movement of the input device is mapped to movement of an image that is rendered to a display;

changing the gearing to different settings during the movement of the image as the movement of the image is rendered to the display.

2. The method of claim 1, wherein processing the captured image data includes tracking the LED array in the captured image data.

3. The method of claim 2, wherein tracking the LED array includes analyzing a position and deformation of the LED array in the captured image data to determine a position and orientation of the input device in the interactive environment.

4. The method of claim 2, wherein the input device is configured to modulate the LED array to enable the tracking of the LED array in the captured image data.

5. The method of claim 1, wherein changing the gearing to different settings is in accordance with a user selectable setting.

6. The method of claim 1, further comprising, receiving inertial data generated from one or more inertial sensors of the input device, the inertial sensors include one or more of an accelerometer and a gyroscope, the movement of the input device being determined using the inertial data.

7. The method of claim 6, wherein processing the inertial data includes analyzing signals from the accelerometer or gyroscope to determine a position or orientation of the input device in the interactive environment.

8. The method of claim 1, wherein the movement of the image is a translational or rotational movement the image.

9. The method of claim 1, wherein changing the gearing to different settings occurs during the movement of the input device.

10. A system, comprising:

an input device, the input device including,

a light emitting diode (LED) array;

an image capture device configured to capture images of an interactive environment in which the input device is disposed, the image capture device configured to generate captured image data that is processed to determine a movement of the input device in the interactive environment;

wherein a gearing is established that adjusts an amount by which the movement of the input device is mapped to movement of an image that is rendered to a display, the gearing changing to different settings during the movement of the image as the movement of the image is rendered to the display.

11. The system of claim 10, wherein the captured image data is processed to track the LED array in the captured image data, including analyzing a position and deformation of the LED array in the captured image data to determine a position and orientation of the input device in the interactive environment.

12. The system of claim 11, wherein the input device is configured to modulate the LED array to enable the tracking of the LED array in the captured image data.

13. The system of claim 10, wherein the input device further includes one or more inertial sensors configured to generate inertial data, the inertial sensors include an accelerometer or a gyroscope, wherein processing the inertial data includes analyzing signals from the accelerometer or gyroscope to determine a position or orientation of the input device in the interactive environment.

14. The system of claim 10, wherein the input device includes a handle for holding by a hand of a user, and wherein the LED array is positioned on a front side of the input device that is opposite the handle.

15. The system of claim 10, wherein the input device includes one or more of the following:

a joystick, a trigger, a button, a pressure pad.

16. The system of claim 10, wherein the image capture device includes two or more cameras.

17. The system of claim 10, wherein the image capture device is proximate to the display.

18. A system, comprising:

an input device, the input device including,

a plurality of lights, and

one or more inertial sensors configured to generate inertial data;

an image capture device configured to capture images of an interactive environment in which the input device is disposed, the image capture device configured to generate captured image data that is processed with the inertial data to determine a movement of the input device in the interactive environment;

wherein a gearing is established that adjusts an amount by which the movement of the input device is mapped to movement of a virtual object that is rendered to a display, the gearing changing to different settings during the movement of the virtual object as the movement of the virtual object is rendered to the display.

19. The system of claim 18, wherein the captured image data is processed to track the plurality of lights in the captured image data, including analyzing a position of the plurality of lights in the captured image data to determine a position of the input device in the interactive environment.

20. The system of claim 18, wherein the inertial sensors include an accelerometer or a gyroscope, wherein processing the inertial data includes analyzing signals from the accelerometer or gyroscope to determine an orientation of the input device in the interactive environment.

21. The system of claim 18, wherein the input device includes a handle for holding by a hand of a user, and wherein the plurality of lights are positioned on a front side of the input device that is opposite the handle.

22. The system of claim 18, wherein the input device includes one or more of the following:

a joystick, a trigger, a button, a pressure pad.

23. The system of claim 18, wherein the image capture device includes two or more cameras.

24. The system of claim 18, wherein the image capture device is proximate to the display.