OBJECT TRACKING WITH PROJECTED REFERENCE PATTERNS

• US 20120262365A1
• Filed: 04/12/2011
• Published: 10/18/2012
• Est. Priority Date: 04/12/2011
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Mallinson, Dominic S.
• Granted Patent: US 8,791,901 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G01S 1/725 Marker, boundary, call-sign or like beacons transmitting signals not carrying directional information
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

Systems and methods for tracking an object'"'"'s position and orientation within a room using patterns projected onto an interior surface of the room, such as the ceiling. Systems include at least one optical position sensitive device embedded in the object to detect relative changes in the projected pattern as the object'"'"'s position and/or orientation is changed. In particular systems, the pattern includes a plurality of beacons projected by one or more steerable lasers. Projected beacons may be steered automatically to accommodate a variety of room topologies. Additional optical position sensitive devices may be disposed in known physical positions relative to the projected pattern to view either or both the projected pattern and the object. A subset of object positional data may be derived from a video camera viewing the object while another subset of object positional data is derived from the projected pattern.

Claims

1. A system for controlling a graphic on a display screen based on positional data determined for a tracked object, the system comprising:

a projector to project a pattern of identifiable radiation onto at least one interior surface of a room in which the display screen is to be disposed;

a first optical position sensing device disposed in the object to collect the identifiable radiation as reflected from the at least one interior surface;

a processor to determine positional data for the object based on the reflected radiation collected by the first optical position sensing device; and

a graphics engine to render the graphic based on the object positional data.

2. The system of claim 1, wherein the at least one interior surface includes the room ceiling.

3. The system of claim 1, wherein the projector comprises a steerable laser having an emission spectrum comprising the near IR band, wherein the pattern of identifiable radiation comprises a plurality of beacons sequentially projected to mapped locations on the at least one interior surface, and wherein the first optical position sensing device comprises a photodiode to determine a centroid of the reflected radiation.

4. The system of claim 1, wherein the at least one interior surface includes the room ceiling, wherein the projector comprises a fixed projection beam comprising the near IR band, the pattern comprises a plurality of interconnected repeating units, and wherein the first optical position sensing device comprises a video camera to collect images of the projected pattern.

5. The system of claim 1, further comprising a non-projected beacon emitting radiation at a known location relative to the projected pattern, and wherein the first optical position sensing device is to further determine beacon positional data from radiation emitted from the non-projected beacon.

6. The system of claim 3, further comprising a processor to determine the plurality of mapped beacon locations, the processor to:

cause the projector to steer the projected radiation for one or more of the plurality of beacons from a first location on the interior surface to a second location on the interior surface,

analyze beacon positional data associated with the first and second locations; and

to select one of the first and second locations as one of the plurality of mapped beacon locations.

7. The system of claim 6, wherein the processor is to analyze the beacon positional data by comparing the intensity the reflected radiation for the first and second locations.

8. The system of claim 6, wherein the processor is to analyze the beacon positional data to evaluate movement of a centroid of an intensity distribution associated with one or more of the plurality of beacon locations as a function of steering angles corresponding to the first location to the second location.

9. The system of claim 6, wherein the processor is to analyze the beacon positional data to compare the distance between the projector and the first location with the distance between the projector and the second location.

10. The system of claim 9, wherein the processor is to analyze the beacon positional data by comparing the change in distance between the projector and first and second locations with a change in projection direction caused by the beam steering.

11. The system of claim 6, wherein the processor is to analyze the beacon positional data to maximize the number in the plurality of beacons mapped to locations which lie on a same plane.

12. They system of claim 6, wherein the processor is to analyze the beacon positional data by executing an algorithm to maximize uniformity in the beacon spacing.

13. The system of claim 6, wherein the processor is communicatively coupled to the first optical position sensing device, and wherein the processor is to receive from the first optical position sensing device a separate centroid of an intensity distribution associated with one or more of the plurality of beacon locations when the one or more of the plurality of beacons are at the first location and at the second location.

14. The system of claim 6, further comprising a second optical position sensing device disposed in known fixed position and orientation relative to the beacon projector and communicatively coupled to the processor, and wherein the processor is to receive the beacon positional data associated with the first and second locations from the second optical position sensing device.

15. The system of claim 14, wherein the second optical position sensing device is to determine a centroid of an intensity distribution associated with one or more of the plurality of beacon locations when the one or more of the plurality of beacons are steered from the first location and the second location.

16. The system of claim 1, further comprising:

a video camera to image the object concurrently with the pattern projection; and

a processor to reconcile object positional data derived from image data generated by the video camera with object positional data derived from the projected pattern.

17. The system of claim 16, wherein the first optical position sensing device is disposed in a top side of the object, and wherein the processor is to determine yaw of the object based solely on object positional data derived from the projected pattern.

18. The system of claim 17, wherein the processor is to determine an x and y position of the object based at least in part on object positional data derived from image data generated by the video camera and is to determine a z position of the object based at least in part on object positional data derived from the projected pattern.

19. The system of claim 17, wherein the projector comprises a fixed IR projector, and wherein the first optical position sensing device comprises a second video camera to collect images of the projected pattern.

20. The system of claim 1, wherein the first optical position sensing device is one of a plurality of optical position sensing device embedded in the object, each of the plurality having know relative positions and orientations, and wherein the processor is to determine positional data for the object based on the reflected radiation collected by the plurality of optical position sensing devices.

21. A method of controlling a graphic on a display screen based on positional data determined for a tracked object, the method comprising:

projecting a pattern of identifiable radiation onto at least one interior surface of a room in which the display screen is to be disposed;

collecting the identifiable radiation as reflected from the at least one interior surface with a first optical position sensing device disposed in the object;

determining positional data for the object based on the reflected radiation collected by the first optical position sensing device; and

controlling rendition of the graphic based on the object positional data.

22. The method of claim 21, wherein the at least one interior surface includes the room ceiling.

23. The method of claim 21, further comprising activating an LED as a fixed beacon emitting radiation at a known location relative to the projected pattern, and wherein collecting the identifiable radiation further includes collecting the radiation emitted by the LED.

24. The system of claim 21, wherein the at least one interior surface includes the room ceiling, wherein projecting the pattern of identifiable radiation further comprises projecting a fixed pattern comprising a plurality of interconnected repeating units.

25. The system of claim 24, wherein the first optical position sensing device comprises a video camera to collect images of the projected pattern.

26. The method of claim 21, wherein projecting the pattern of identifiable radiation further comprises sequentially projecting a plurality of beacons to mapped locations on the at least one interior surface using at least one laser having an emission spectrum comprising the IR band.

27. The method of claim 26, wherein collecting the identifiable radiation further comprises determining a centroid of an intensity distribution associated with one or more of the plurality of beacon locations.

28. The method of claim 26, further comprising determining the plurality of mapped beacon locations, the determining further comprising:

steering the projected radiation for one or more of the plurality of beacons from a first location on the interior surface to a second location on the interior surface,

analyzing beacon positional data associated with the first and second locations; and

selecting one of the first and second locations as one of the plurality of mapped beacon locations.

29. The method of claim 28, wherein the analyzing further comprises comparing the intensity the reflected radiation for the first and second locations.

30. The method of claim 28, wherein the analyzing further comprises evaluating movement of a centroid of an intensity distribution associated with one or more of the plurality of beacon locations when the one or more of the plurality of beacons are steered from the first location to the second location.

31. The method of claim 28, wherein the analyzing further comprises comparing the distance between the projector and the first location with the distance between the projector and the second location.

32. The method of claim 31, wherein the analyzing further comprises comparing the change in distance between the projector and the first and second locations with a change in projection direction caused by the beam steering.

33. The method of claim 28, wherein the selecting one of the first and second locations is to maximize the number in the plurality of beacons mapped to locations which lie on a same plane.

34. The method of claim 28, wherein the selecting one of the first and second locations is to maximize uniformity in the beacon spacing across the same plane.

35. The method of claim 28, further comprising generating a separate centroid of an intensity distribution associated with one or more of the plurality of beacon locations when the one or more of the plurality of beacons are at each of the first second locations.

36. The method of claim 35, wherein the separate centroids are provided by a second optical position sensing device disposed in known fixed position and orientation relative to the beacon projector.

37. The method of claim 21, further comprising:

determining positional data of the object from image data generated by a video camera having the object in the camera field of view; and

reconciling object positional data derived from the collected image data with object positional data derived from the projected pattern.

38. The method of claim 37, wherein reconciling the object positional data further comprises determining yaw of the object based solely on object positional data derived from the projected pattern.

39. The method of claim 38, wherein reconciling the object positional data further comprises an x and y position of the object based at least in part on object positional data derived from image data generated by the video camera and determining a z position of the object based on object positional data derived from the projected pattern.

40. A computer readable storage medium including instructions that, when executed by a processing system, cause the processing system to perform the method of claim 21.