Input for computer device using pattern-based computer vision

• US 8,602,893 B2
• Filed: 06/02/2010
• Issued: 12/10/2013
• Est. Priority Date: 06/02/2010
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Mao, Xiaodong, Rimon, Noam
• Primary Examiner(s): CUFF, MICHAEL A
• Application Number:
  US12/792,628
• Publication Number:
  US 20110300939A1
• Time in Patent Office: 1,287 Days
• Field of Search: 463/36, 382/103
• US Class Current: 463/36
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/8047 Music games
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/0334 Foot operated pointing devices
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/19 by sensing codes defining pattern positions

Abstract

Images of a least a portion of a structurally encoded pattern formed on a surface from a camera can be analyzed to determine a position of a camera. The structurally encoded pattern can include a plurality of elements. Each element can be encoded with information identifying a location and orientation of the element with respect to the surface. A computer program can identify at least a portion of one or more pattern elements in the image and determine the position of the camera relative to the surface from the information encoded in the one or more pattern elements in the image.

Claims

1. An input apparatus for a computer device, comprising:

a computer processor configured to execute computer executable instructions, wherein the instructions are configured to cause the computer device to;

a) obtain an image of a least a portion of a structurally encoded pattern formed on a surface from a camera, wherein the structurally encoded pattern includes a plurality of elements, wherein each element is encoded with information identifying a location and orientation of the element with respect to the surface;

b) identify at least a portion of one or more pattern elements in the image; and

c) determine a position of the camera relative to the surface from the information encoded in the one or more pattern elements in the image, and wherein the computer processor is configured to use the position of the camera as a control input to a video game program.

2. The apparatus of claim 1 wherein the structurally encoded pattern includes a plurality of cells, wherein each cell contains a unique element that is different from every other element in every other cell in the pattern.

3. The apparatus of claim 1 wherein each element is not rotationally symmetric with respect to an axis normal to the plane of the surface.

4. The apparatus of claim 1 wherein the computer processor is configured to determine coordinates of a location of the camera relative to the pattern.

5. The apparatus of claim 1 wherein the computer processor is configured to determine an angular orientation of the camera relative to the pattern.

6. The apparatus of claim 1 wherein the computer processor is configured to compare the image to reference data representing the pattern.

7. The apparatus of claim 1, further comprising the camera, wherein the camera is coupled to the computer processor.

8. The apparatus of claim 7 wherein the camera is adapted to be mounted to a user'"'"'s foot.

9. The apparatus of claim 7 wherein the camera is adapted to be mounted to a user'"'"'s hand.

10. The apparatus of claim 1, further comprising a mat, wherein the structurally encoded pattern is formed on the mat.

11. An input apparatus for a computer device, comprising:

a structurally encoded pattern formed on a surface, wherein the structurally encoded pattern includes a plurality of elements, wherein each element is encoded with information identifying a location and orientation of the element with respect to the surface;

a camera operable to obtain an image of at least a portion of the pattern; and

a processor unit coupled to the camera, wherein the processor is configured to identify at least a portion of one or more elements in the image and determine a position of the camera relative to the surface from the information encoded in the one or more pattern elements in the image, and wherein the processor unit is configured to use the position of the camera as a control input to a video game program.

12. A method for receiving a control input to a video game program, comprising:

obtaining an image of a least a portion of a structurally encoded pattern formed on a surface with a camera, wherein the structurally encoded pattern includes a plurality of elements, wherein each element is encoded with information identifying a location and orientation of the element with respect to the surface;

with a processor unit coupled to the camera, identifying at least a portion of one or more pattern elements in the image; and

with the processor unit, determining a position of the camera relative to the surface from the information encoded in the one or more pattern elements in the image, and wherein the position of the camera is the control input to the video game program.

13. The method of claim 12 wherein the structurally encoded pattern includes a plurality of cells, wherein each cell contains a unique element that is different from every other element in every other cell in the pattern.

14. The method of claim 12 wherein each element is not rotationally symmetric with respect to an axis normal to the plane of the surface.

15. The method of claim 12 wherein the structurally encoded pattern is formed on a mat.

16. The method of claim 12 wherein the processor is configured to determine coordinates of a location of the camera relative to the pattern.

17. The method of claim 12 wherein the processor is configured to determine an angular orientation of the camera relative to the pattern.

18. The method of claim 12 wherein the processor is configured to compare the image to reference data representing the pattern.

19. The method of claim 12 wherein the camera is adapted to be mounted to a user'"'"'s foot.

20. The method of claim 12 wherein the camera is adapted to be mounted to a user'"'"'s hand.

21. A non-transitory computer readable storage medium having embodied therein computer executable instructions that, upon execution, cause a computer to implement a method for receiving a control input to a video game program, the method comprising:

obtaining an image of a least a portion of a structurally encoded pattern formed on a surface with a camera, wherein the structurally encoded pattern includes a plurality of elements, wherein each element is encoded with information identifying a location and orientation of the element with respect to the surface;

identifying at least a portion of one or more pattern elements in the image; and

determining from the one or more portions a position of the camera relative to the surface from the information encoded in the one or more pattern elements in the image, and wherein the position of the camera is the control input to the video game program.