Locating camera relative to a display device

• US 20110151970A1
• Filed: 12/18/2009
• Published: 06/23/2011
• Est. Priority Date: 12/18/2009
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Osman, Steven
• Granted Patent: US 8,497,902 B2
• Time in Patent Office: Days
• US Class Current: 463/30
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/40 Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/1018 Calibration; Key and button assignment
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/205 for detecting the geographical location of the game platform

Abstract

Embodiments of the present disclosure relate to a method, system, and computer-readable medium for locating a sensor of a gaming console relative to a position of a display device in a gaming system includes displaying an image on the display device, estimating the first position and first orientation of the display device relative to the portable device, once the sensor on the portable device has found the image on the display device, instructing the user to move the portable device to a new position, estimating the second position and second orientation of the display device relative to the portable device, once the sensor on the portable device has found the image on the display device, determining the size of the display device using information gathered during the estimating of the first position and orientation and the estimating of the second position and orientation, and determining the orientation of the display device relative to a sensor of a gaming console using the two findings sent to the gaming console.

Claims

1. A method for locating a sensor on a gaming console relative to a position of a display device in a gaming system, comprising:

displaying an image on the display device;

estimating the first position and first orientation of the display device relative to the portable device, once the portable device has found the image on the display device;

instructing the user to move the portable device to a new position;

estimating the second position and second orientation of the display device relative to the portable device, once the portable device has found the image on the display device;

determining the size of the display device using information gathered during the estimating of the first position and orientation and the estimating of the second position and orientation; and

determining the orientation of the display device relative to the sensor of the gaming console using information gathered during the estimating of the first position and orientation and the estimating of the second position and orientation.

2. A method according to claim 1, further comprising:

instructing the portable device to search for the image, wherein the gaming console instructs the portable device to search for the image;

searching for the image on the display device using a camera on the portable device;

searching the image on the camera of the portable device for the pattern on the display device; and

sending findings of the portable device to the gaming console, wherein the portable device wirelessly sends the findings to the gaming console.

3. The method according to claim 1, further comprising:

determining a first position of the gaming console relative to the portable device using the sensor on the gaming console;

determining a first orientation of the gaming console relative to the portable device using the sensor on the gaming console;

determining a second position of the gaming console relative to the portable device using the sensor on the gaming console;

determining a second orientation of the gaming console relative to the portable device using the sensor on the gaming console;

instructing the portable device to search for the image on the display device from the second position, wherein the gaming console instructs the portable device to search for the image;

searching for the image on the display device from the second position of the portable device using a camera on the portable device;

searching the image of the sensor on the portable device for the pattern on the display device;

sending new findings of the portable device to the gaming console, wherein the portable device wirelessly sends the findings to the gaming console.

4. A method according to claim 1, further comprising:

prompting a user to position the portable device such that a sensor on the portable device can detect an image on the display device.

5. A method according to claim 1, further comprising:

instructing the display device to display an image, wherein the gaming console instructs the display device to display the image.

6. A method according to claim 1, further comprising:

detecting the movement of the portable device using the gaming console.

7. A method according to claim 1, wherein the steps of estimating the first position and the first orientation of the portable device are accomplished using a pose estimation algorithm.

8. A method according to claim 1, wherein the steps of estimating the second position and the second orientation of the portable device are accomplished using a pose estimation algorithm.

9. A method according to claim 1, wherein the portable device includes a gamepad, a joystick, a universal remote, a remote controller, a cellular telephone equipped with a camera, and a Sony PlayStation®

Portable (PSP™

).

10. A method according to claim 1, wherein the gaming console is a video gaming console.

11. A method according to claim 1, wherein the display device includes at least one of a cathode ray tube, a flat panel display, a liquid crystal display (LCD), a plasma display panel (PDP), an organic light-emitting diode display (OLED), a field emission display (FED), a nanocrystal display, a laser TV, and an electroluminescent display (ELD).

12. A method according to claim 1, wherein the sensor of the gaming console comprises at least one of a camera, an ultrasonic emitter, an ultrasonic detector, a magnetic emitter, a magnetic coil, a light-emitting diode, or any combination thereof.

13. A method according to claim 1, wherein the sensor of the portable device comprises at least one of a camera, an ultrasonic emitter, an ultrasonic detector, a magnetic emitter, a magnetic coil, a light-emitting diode, or any combination thereof.

14. A method according to claim 1, further comprising:

instructing the user to move the portable device to a different new position, if it is determined that the new position is along an axis that passes through candidate display devices.

43. The method according to claim 1, further comprising:

determining a first position of the gaming console relative to the portable device using the sensor on the gaming console;

determining a first orientation of the gaming console relative to the portable device using the sensor on the gaming console;

determining a second position of the gaming console relative to the portable device using the sensor on the portable device;

determining a second orientation of the gaming console relative to the portable device using the sensor on the portable device;

instructing the portable device to search for the image on the display device from the second position, wherein the gaming console instructs the portable device to search for the image;

searching for the image on the display device from the second position of the portable device using a camera on the portable device;

searching the image of the sensor on the portable device for the pattern on the display device;

sending new findings of the portable device to the gaming console, wherein the portable device wirelessly sends the findings to the gaming console.

15. A system for locating a sensor of a gaming console relative to a position of a display device in a gaming system, comprising:

means for displaying an image on the display device;

means for estimating the first position and first orientation of the display device relative to the portable device, once the portable device has found the image on the display device;

means for instructing the user to move the portable device to a new position;

means for estimating the second position and second orientation of the display device relative to the portable device, once the portable device has found the image on the display device;

means for determining the size of the display device using information gathered during the estimating of the first position and orientation and the estimating of the second position and orientation; and

means for determining the orientation of the display device relative to a sensor of a gaming console using information gathered during the estimating of the first position and orientation and the estimating of the second position and orientation.

16. A system according to claim 15, further comprising:

means for instructing the portable device to search for the image, wherein the gaming console instructs the portable device to search for the image;

means for searching for the image on the display device using a camera on the portable device;

means for searching the image on the camera of the portable device for the pattern on the display device; and

means for sending findings of the portable device to the gaming console, wherein the portable device wirelessly sends the findings to the gaming console.

17. The system according to claim 15, further comprising:

means for determining a first position of the gaming console relative to the portable device using a sensor on the gaming console;

means for determining a first orientation of the gaming console relative to the portable device using the sensor on the gaming console;

means for determining a second position of the gaming console relative to the portable device using the sensor on the gaming console;

means for determining a second orientation of the gaming console relative to the portable device using the sensor on the gaming console;

means for instructing the portable device to search for the image on the display device from the second position, wherein the gaming console instructs the portable device to search for the image; and

means for searching for the image on the display device from the second position of the portable device using the sensor on the portable device;

means for searching the image of the sensor on the portable device for the pattern on the display device; and

means for sending new findings of the portable device to the gaming console, wherein the portable device wirelessly sends the findings to the gaming console.

18. A system according to claim 15, further comprising:

means for prompting a user to position the portable device such that a sensor on the portable device can detect an image on the display device.

19. A system according to claim 15, further comprising:

means for instructing the display device to display an image, wherein the gaming console instructs the display device to display the image.

20. A system according to claim 15, further comprising:

means for detecting the movement of the portable device using the gaming console.

21. A system according to claim 15, wherein the means for estimating the first position and the first orientation of the portable device uses a pose estimation algorithm.

22. A system according to claim 15, wherein the means for estimating the second position and the second orientation of the portable device uses a pose estimation algorithm.

23. A system according to claim 15, wherein the portable device includes a gamepad, a joystick, a universal remote, a remote controller, a cellular telephone equipped with a camera, and a Sony PlayStation®

Portable (PSP™

).

24. A system according to claim 15, wherein the gaming console is a video gaming console.

25. A system according to claim 15, wherein the display device comprises at least one of a cathode ray tube, a flat panel display, a liquid crystal display (LCD), a plasma display panel (PDP), an organic light-emitting diode display (OLED), a field emission display (FED), a nanocrystal display, a laser TV, and an electroluminescent display (ELD).

26. A system according to claim 15, wherein the sensor of the gaming console comprises at least one of a camera, an ultrasonic emitter, an ultrasonic detector, a magnetic emitter, a magnetic coil, a light-emitting diode, or any combination thereof.

27. A system according to claim 15, wherein the sensor of the portable device comprises at least one of a camera, an ultrasonic emitter, an ultrasonic detector, a magnetic emitter, a magnetic coil, a light-emitting diode, or any combination thereof.

28. A system according to claim 15, further comprising:

means for instructing the user to move the portable device to a different new position, if it is determined that the new position is along an axis that passes through candidate display devices.

44. The system according to claim 15, further comprising:

means for determining a first position of the gaming console relative to the portable device using the sensor on the gaming console;

means for determining a first orientation of the gaming console relative to the portable device using the sensor on the gaming console;

means for determining a second position of the gaming console relative to the portable device using the sensor on the portable device;

means for determining a second orientation of the gaming console relative to the portable device using the sensor on the portable device;

means for instructing the portable device to search for the image on the display device from the second position, wherein the gaming console instructs the portable device to search for the image; and

means for searching for the image on the display device from the second position of the portable device using the sensor on the portable device;

means for searching the image of the sensor on the portable device for the pattern on the display device; and

means for sending new findings of the portable device to the gaming console, wherein the portable device wirelessly sends the findings to the gaming console.

29. A computer-readable medium for use in a gaming system for locating a sensor of a gaming console relative to a position of a display device within the gaming system, comprising:

means for displaying an image on the display device;

means for estimating the first position and first orientation of the display device relative to the portable device, once the portable device has found the image on the display device;

means for instructing the user to move the portable device to a new position;

means for estimating the second position and second orientation of the display device relative to the portable device, once the portable device has found the image on the display device;

means for determining the size of the display device using information gathered during the estimating of the first position and orientation and the estimating of the second position and orientation; and

means for determining the orientation of the display device relative to the sensor of the gaming console using information gathered during the estimating of the first position and orientation and the estimating of the second position and orientation.

30. A computer-readable medium according to claim 29, further comprising:

means for instructing the portable device to search for the image, wherein the gaming console instructs the portable device to search for the image;

means for searching for the image on the display device using a camera on the portable device;

means for searching the image on the camera of the portable device for the pattern on the display device; and

means for sending findings of the portable device to the gaming console, wherein the portable device wirelessly sends the findings to the gaming console.

31. The computer-readable medium according to claim 29, further comprising:

means for determining a first position of the gaming console relative to the portable device using the sensor on the gaming console;

means for determining a first orientation of the gaming console relative to the portable device using the sensor on the gaming console;

means for determining a second position of the gaming console relative to the portable device using the sensor on the gaming console;

means for determining a second orientation of the gaming console relative to the portable device using the sensor on the gaming console;

means for instructing the portable device to search for the image on the display device from the second position, wherein the gaming console instructs the portable device to search for the image; and

means for searching for the image on the display device from the second position of the portable device using the sensor on the portable device;

means for searching the image of the sensor on the portable device for the pattern on the display device; and

means for sending new findings of the portable device to the gaming console, wherein the portable device wirelessly sends the findings to the gaming console.

32. A computer-readable medium according to claim 29, further comprising:

means for prompting a user to position the portable device such that a sensor on the portable device can detect an image on the display device.

33. A computer-readable medium according to claim 29, further comprising:

means for instructing the display device to display an image, wherein the gaming console instructs the display device to display the image.

34. A computer-readable medium according to claim 29, further comprising:

means for detecting the movement of the portable device using the gaming console.

35. A computer-readable medium according to claim 29, wherein the means for estimating the first position and the first orientation of the portable device uses a pose estimation algorithm.

36. A computer-readable medium according to claim 29, wherein the means for estimating the second position and the second orientation of the portable device uses a pose estimation algorithm.

37. A computer-readable medium according to claim 29, wherein the portable device includes a gamepad, a joystick, a universal remote, a remote controller, a cellular telephone equipped with a camera, and a Sony PlayStation®

Portable (PSP™

).

38. A computer-readable medium according to claim 29, wherein the gaming console is a video gaming console.

39. A computer-readable medium according to claim 29, wherein the display device comprises at least one of a cathode ray tube, a flat panel display, a liquid crystal display (LCD), a plasma display panel (PDP), an organic light-emitting diode display (OLED), a field emission display (FED), a nanocrystal display, a laser TV, and an electroluminescent display (ELD).

40. A computer-readable medium according to claim 29, wherein the sensor of the gaming console comprises at least one of a camera, an ultrasonic emitter, an ultrasonic detector, a magnetic emitter, a magnetic coil, a light-emitting diode, or any combination thereof.

41. A computer-readable medium according to claim 29, wherein the sensor of the portable device comprises at least one of a camera, an ultrasonic emitter, an ultrasonic detector, a magnetic emitter, a magnetic coil, a light-emitting diode, or any combination thereof.

42. A computer-readable medium according to claim 29, further comprising:

means for instructing the user to move the portable device to a different new position, if it is determined that the new position is along an axis that passes through candidate display devices.

45. The computer-readable medium according to claim 29, further comprising:

means for determining a first position of the gaming console relative to the portable device using the sensor on the gaming console;

means for determining a first orientation of the gaming console relative to the portable device using the sensor on the gaming console;

means for determining a second position of the gaming console relative to the portable device using the sensor on the portable device;

means for determining a second orientation of the gaming console relative to the portable device using the sensor on the portable device;

means for instructing the portable device to search for the image on the display device from the second position, wherein the gaming console instructs the portable device to search for the image; and

means for searching for the image on the display device from the second position of the portable device using the sensor on the portable device;

means for searching the image of the sensor on the portable device for the pattern on the display device; and

means for sending new findings of the portable device to the gaming console, wherein the portable device wirelessly sends the findings to the gaming console.