Systems for using three-dimensional object as controller in an interactive game

• US 9,511,282 B2
• Filed: 08/10/2015
• Issued: 12/06/2016
• Est. Priority Date: 12/17/2007
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Zalewski, Gary, Gillo, Tomas
• Primary Examiner(s): Chan, Allen
• Application Number:
  US14/822,829
• Publication Number:
  US 20150346948A1
• Time in Patent Office: 484 Days
• Field of Search: 463/33
• US Class Current: 1/1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/8011 Ball
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

A system for initiating and using a three-dimensional object as a controlling device when interfacing with a computer system used for interactive video game play is provided. One example system includes an interface for receiving data from a capturing device of a three-dimensional space and storage coupled with computer system. The computer system provides data to a screen and receiving user input to obtain geometric parameters of the three-dimensional object and assign actions to be performed with the three-dimensional object when moved or placed in positions in front of the capture device during interactive video game play. The geometric parameters and the assigned actions being saved to a database on the storage for access during interactive video game play or future interactive sessions.

Claims

1. A system for using a three-dimensional object as a controlling device, comprising:

a camera configured to capture data of a three-dimensional space;

a computer system being interfaced with the camera and a display screen;

the computer system providing a graphical user interface that is rendered on the display screen, the graphical user interface receiving user input to enable capture via the camera of geometric parameters of the three-dimensional object when presented in the three-dimensional space, wherein the three-dimensional object is assigned to one or more actions to be executed by the computer system when the three-dimensional object is moved or displayed in front of the camera during an interactive session that is executed by the computer system;

wherein the three-dimensional object has a shape that is identifiable by the camera;

wherein identification of the shape or changes in position of the shape of the three-dimensional object in the three-dimensional space conveys input to the computer system during the interactive session.

2. The system for using a three-dimensional object as a controlling device as recited in claim 1, wherein the computer system executes one of a program, game program, an application, a search engine, a communication program, or a combination of two or more thereof, wherein said identification of the shape or changes in position of the shape of the three-dimensional object is used to provide input.

3. The system for using a three-dimensional object as a controlling device as recited in claim 1, wherein the camera is a depth camera that is configured to capture depth data.

4. The system for using a three-dimensional object as a controlling device as recited in claim 1, wherein the depth data is used to identify a wire frame model of the three-dimensional object.

5. The system for using a three-dimensional object as a controlling device as recited in claim 1, wherein the computer system causes the camera to capture a sequence of images to enable said identification of the shape or changes in position of the shape of the three-dimensional object.

6. The system for using a three-dimensional object as a controlling device as recited in claim 1, wherein the computer program is configured to execute images of a virtual reality space for rendering on the display screen during the interactive session, wherein the three-dimensional object is a real-world object and the real-world object is represented in the virtual reality space as a virtual object, such that movement of the three-dimensional object is illustrated as movement of the virtual object.

7. A computer implemented method to interactively capture and utilize a three-dimensional object to be a controlling device for a computer system, comprising:

capturing depth data of the three-dimensional object;

processing captured depth data of the three-dimensional object to create geometric defining parameters for the three-dimensional object;

during an interactive session processed by the computer system, capturing depth data and analyzing the geometric defining parameters to identify the three-dimensional object;

rendering on a display, a virtual object, that is selected for the three-dimensional object, the virtual object being rendered on the display such that movement of the three-dimensional object causes rendering of corresponding movement by the virtual object on the display screen.

8. The computer implemented method of claim 7, wherein the virtual object is an graphical object that is associated to the three-dimensional object, such that when the three-dimensional object is identified from the depth data captured by a depth camera, the virtual objects is displayed in a virtual scene that is rendered on the display screen.

9. The computer implemented method of claim 8, wherein movements or positions of the three-dimensional object are mapped to specific inputs to be made by the virtual object in the virtual scene.

10. The computer implemented method of claim 8, further comprising,

triggering activation of actions by the virtual object during the interactive session in response to physical actions performed with the three-dimensional object.

11. The computer implemented method of claim 8, wherein the actions by the virtual objects are pre-correlated to one or more physical actions to be performed with the three-dimensional object.

12. The computer implemented method of claim 7, wherein depth data for the three-dimensional object is captured from a plurality of angles.

13. The computer implemented method of claim 7, wherein the plurality of angles allows the computer system to process the depth data and create at least part of the geometric parameters for the three-dimensional object.

14. The computer implemented method of claim 7, further comprising,

capturing depth data of one or more additional three-dimensional objects;

associating virtual objects to each of the one or more additional three-dimensional objects;

wherein identification of the one or more additional three-dimensional objects causes rendering of the respective virtual object in a virtual scene.

15. The computer implemented method of claim 7, wherein the virtual object is rendered in a virtual scene that is rendered on the display screen.

16. The computer implemented method of claim 7, wherein some movements of the three-dimensional object trigger input of one or more actions to an application rendering the interactive session.

17. The computer implemented method of claim 16, wherein some movements include one or more of moving the object, turning the object, waving the object, tapping the object, shaking the object, pointing with the object, or combination of two or more thereof.

18. The computer implemented method of claim 16, wherein the computer system causes a depth camera to capture depth data for a sequence of images to enable said identification of the three-dimensional object.

19. The computer implemented method of claim 7, wherein the three-dimensional object is a physical object that is hand-held by a user.

20. The computer implemented method of claim 7, wherein the interactive session is of a video game or an application that renders a virtual scene for rendering the virtual object, the virtual scene having one or more objects and the virtual object used to interact or interface with the one or more objects.