COLOR CALIBRATION FOR OBJECT TRACKING

• US 20100303297A1
• Filed: 05/27/2010
• Published: 12/02/2010
• Est. Priority Date: 05/30/2009
• Status: Abandoned Application

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Mikhailov, Anton, Marks, Richard Lee
• Application Number:
  US12/789,389
• Publication Number:
  US 20100303297A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/843 involving concurrently two or more players on the same game device, e.g. requiring the use of a plurality of controllers or of a specific view of game data for each player
• CPC Class Codes: A63F 2300/1018 Calibration; Key and button assignment
• CPC Class Codes: A63F 2300/1043 being characterized by constructional details
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: H04N 9/75 Chroma key

Abstract

To calibrate a tracking system a computing device locates an object in one or more images taken by an optical sensor. The computing device determines environment colors included in the image, the environment colors being colors in the one or more images that are not emitted by the object. The computing device determines one or more trackable colors that, if assumed by the object, will enable the computing device to track the object.

Claims

1. A method of calibrating a tracking system, comprising:

locating, by a computing device executing a calibrating logic, an object in one or more images taken by an optical sensor;

determining environment colors included in the one or more images by a computing device, the environment colors being colors in the one or more images that are not emitted by the object; and

determining one or more trackable colors that, if assumed by the object, will enable the computing device to track the object.

2. The method of claim 1, wherein a color is a trackable color if there is less than a threshold amount of the color included in the environment colors.

3. The method of claim 1, wherein locating the object in the one or more images comprises:

causing the object to assume a plurality of predefined colors, in sequence, wherein the one or more images include a separate image for each of the plurality of predefined colors; and

identifying a group of pixels that have different colors in each of the one or more images.

4. The method of claim 3, further comprising:

for each predefined color, determining an associated set of colors of the object that are observed by the optical sensor; and

predicting additional sets of colors that will be observed by the optical sensor if the object assumes other colors.

5. The method of claim 4, further comprising:

causing the object to assume a specific color; and

tracking the object by identifying where in subsequent images a specific set of colors associated with the specific color are observed.

6. The method of claim 5, further comprising:

failing to successfully track the object; and

causing the object to assume a new specific color.

7. The method of claim 3, wherein the plurality of predefined colors includes three primary colors.

8. The method of claim 3, wherein the optical sensor has a frame rate, the method further comprising:

causing the object to assume each of the plurality of predefined colors for at least three frames of the optical sensor.

9. The method of claim 3, wherein the optical sensor has a frame rate, and wherein frames of the optical sensor are time synchronous with a clock of the computing device, the method further comprising:

causing the object to assume each of the plurality of predetermined colors for fewer than three frames.

10. The method of claim 1, wherein determining the environment colors comprises:

causing the object to emit no light, wherein the one or more images include an image taken while the objected emitted no light; and

identifying colors of the object from the image taken while the object emitted no light, which represent environment colors reflected by the object.

11. The method of claim 10, further comprising:

subtracting the colors of the object in the image taken while the object emitted no light from the colors of the object in the one or more images taken while the object emitted light of a predefined color to identify a true color emitted by the object.

12. The method of claim 1, further comprising:

periodically updating the environment colors based on momentarily causing the object to emit no light and taking an image while the object emits no light; and

re-determining the one or more trackable colors.

13. The method of claim 1, further comprising:

detecting a change in an exposure setting of the optical sensor; and

adjusting a brightness of the object.

14. The method of claim 1, further comprising:

determining that tracking quality of the tracking system will be sub-optimal;

if the environment light exceeds a threshold, notifying a user of the computing device to turn off lights or close blinds; and

if there are fewer than a threshold number of trackable colors, identifying to the user additional objects included in the one or more images that are causing color conflicts.

15. A computing apparatus, comprising:

an optical sensor to take one or more images; and

a processing device, to execute instructions for a calibrating logic, wherein the instructions cause the processing device to locate an object in the one or more images, determine environment colors included in the one or more images, the environment colors being colors in the one or more images that are not emitted by the object, and determine one or more trackable colors that, if assumed by the object, will enable the processing device to track the object.

16. The computing apparatus of claim 15, wherein a color is a trackable color if there is less than a threshold amount of the color included in the environment colors.

17. The computing apparatus of claim 15, wherein locating the object in the one or more images comprises:

causing the object to assume a plurality of predefined colors, in sequence, wherein the one or more images include a separate image for each of the plurality of predefined colors; and

identifying a group of pixels that have different colors in each of the one or more images.

18. The computing apparatus of claim 17, wherein the instructions cause the processing device to:

for each predefined color, determine an associated set of colors of the object that are observed by the optical sensor; and

predict additional sets of colors that will be observed by the optical sensor if the object assumes other colors.

19. The computing apparatus of claim 18, wherein the instructions cause the processing device to:

cause the object to assume a specific color; and

track the object by identifying where in subsequent images a specific set of colors associated with the specific color are observed.

20. The computing apparatus of claim 19, wherein the instructions cause the processing device to:

instruct the object to assume a new specific color when the processing device fails to successfully track the object.

21. The computing apparatus of claim 17, wherein the plurality of predefined colors includes three primary colors.

22. The computing apparatus of claim 17, wherein the optical sensor has a frame rate, the instructions cause the processing device to:

instruct the object to assume each of the plurality of predefined colors for at least three frames of the optical sensor.

23. The computing apparatus of claim 17, wherein the optical sensor has a frame rate, and wherein frames of the optical sensor are time synchronous with a clock of the computing device, the instructions cause the processing device to:

instruct the object to assume each of the plurality of predetermined colors for fewer than three frames.

24. The computing apparatus of claim 15, wherein determining the environment colors comprises:

causing the object to emit no light, wherein the one or more images include an image taken while the objected emitted no light; and

identifying colors of the object from the image taken while the object emitted no light, which represent environment colors reflected by the object.

25. The computing apparatus of claim 24, wherein the instructions cause the processing device to:

subtract the colors of the object in the image taken while the object emitted no light from the colors of the object in the one or more images taken while the object emitted light of a predefined color to identify a true color emitted by the object.

26. The computing apparatus of claim 15, wherein the instructions cause the processing device to:

periodically update the environment colors based on momentarily causing the object to emit no light and taking an image while the object emits no light; and

re-determine the one or more trackable colors.

27. The computing apparatus of claim 15, wherein the instructions cause the processing device to:

detect a change in an exposure setting of the optical sensor; and

adjust a brightness of the object.

28. The computing apparatus of claim 15, wherein the instructions cause the processing device to:

determine that tracking quality of the tracking system will be sub-optimal;

notify a user of the computing device to turn off lights or close blinds if the environment light exceeds a threshold; and

identify to the user additional objects included in the one or more images that are causing color conflicts if there are fewer than a threshold number of trackable colors.

29. A computer readable storage medium including instructions that, when executed by a processing device, cause the processing device to perform method comprising:

locating, by a computing device executing a calibrating logic, an object in one or more images taken by an optical sensor;

determining environment colors included in the one or more images by a computing device, the environment colors being colors in the one or more images that are not emitted by the object; and

determining one or more trackable colors that, if assumed by the object, will enable the computing device to track the object.

30. The computer readable storage medium of claim 29, wherein locating the object in the one or more images comprises:

causing the object to assume a plurality of predefined colors, in sequence, wherein the one or more images include a separate image for each of the plurality of predefined colors; and

identifying a group of pixels that have different colors in each of the one or more images.

31. The computer readable storage medium of claim 29, wherein determining the environment colors comprises:

causing the object to emit no light, wherein the one or more images include an image taken while the objected emitted no light; and

identifying colors of the object from the image taken while the object emitted no light, which represent environment colors reflected by the object.