Controller with an integrated depth camera

• US 8,368,753 B2
• Filed: 03/17/2008
• Issued: 02/05/2013
• Est. Priority Date: 03/17/2008
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Zalewski, Gary
• Primary Examiner(s): Chang, Jungwon
• Application Number:
  US12/050,128
• Publication Number:
  US 20090231425A1
• Time in Patent Office: 1,786 Days
• Field of Search: 463/32, 463/37, 463/33, 463/39, 463/31, 715/706, 348/142, 348/42, 345/419, 345/520
• US Class Current: 348/142
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/90 Constructional details or arrangements of video game devices not provided for in groups A63F13/20 or A63F13/25, e.g. housing, wiring, connections or cabinets
• CPC Class Codes: A63F 2300/1018 Calibration; Key and button assignment
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0085 Motion estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A method for determining a position of a controller in a three-dimensional space is disclosed. The method includes an operation to calibrate a computer program to identify a group of recognized fixed points from a plurality of fixed points within the three-dimensional space using a depth camera integrated with the controller. Another operation activates use of the controller and depth camera during interface with the application. During use of the controller, image and depth data within the three-dimensional space is captured with the depth camera of the controller. In another operation the image and depth data is analyzed to find one or more of the group of recognized fixed points. In one embodiment, the previous two operations are repeated and changes of position of the controller are determined based on a change in position of the found ones of the group of recognized fixed points. The method also includes an operation to trigger an input command based on the determined change in position of the controller, wherein the input command causes an action by the application.

Claims

1. A handheld controller for a computer system, comprising;

a controller housing, the controller housing defined by an exterior that encloses an interior volume;

interface controls, the interface controls partially exposed to the exterior of the housing;

a depth camera assembly, the depth camera assembly including a depth module, an image module and processing logic, the depth module and image module defined to receive image and depth data through a camera lens; and

a communication module, the communication module configured to transmit and receive data from the processing logic of the depth camera assembly to the computer system, wherein changes to the image and depth data received by the depth camera assembly trigger input commands for the computer system;

wherein the depth camera assembly is contained within the controller housing and the camera lens is exposed to the exterior of the controller housing.

2. An interactive computing system configured to display an interactive session on a display, comprising;

a computing console, the computing console including;

a central processing unit;

a graphics processing unit; and

a communications interface, the communications interface being configured to provide communication to and from the computing console; and

a controller for the computing system, the controller including;

a depth camera, the depth camera having a depth module, an image module and processing logic; and

a communication module, the communication module being configured to receive and transmit data from the processing logic of the depth camera to the communications interface of the computing console,

wherein changes in image and depth data captured by the depth camera are translated to input commands for the computing system;

wherein the depth module is defined to capture depth data and the image module is defined to capture image data for objects within a field of view of the depth camera.

3. The computer system of claim 2, wherein the processing logic is defined to process the captured depth data and image data.

4. The computer system of claim 2, wherein the communications module is defined to transmit and receive data from the depth camera to the computing console.