Calibration of sensors and projector

• US 10,241,616 B2
• Filed: 02/28/2014
• Issued: 03/26/2019
• Est. Priority Date: 02/28/2014
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Lyons, Nicholas P, Kang, Jinman
• Primary Examiner(s): Sarma, Abhishek
• Application Number:
  US15/121,690
• Publication Number:
  US 20160378258A1
• Time in Patent Office: 1,852 Days
• CPC Class Codes: G03B 17/54 with projector
• CPC Class Codes: G03B 21/28 Reflectors in projection beam in illumination beam G03B21/2066
• CPC Class Codes: G03B 43/00 Testing correct operation of photographic apparatus or parts thereof
• CPC Class Codes: G06F 1/1639 the display being based on projection
• CPC Class Codes: G06F 2200/1631 Panel PC, e.g. single housing hosting PC and display panel
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

An example system, including a projector unit, an all-in-one computer comprising a calibration module and attachable to the projector unit, and a plurality of sensors communicatively coupled to the all-in-one computer is provided. In addition, the all-in-one computer stores mapping information relating to mappings between the plurality of sensors and the projector unit in a common coordinate system. Further, the calibration module calibrates the plurality of sensors and the projector unit using the mapping information.

Claims

1. A system, comprising:

a projector;

a computer comprising instructions and attachable to the projector,

a plurality of sensors communicatively coupled to the computer; and

the computer to store mapping information relating to mappings between the plurality of sensors and the projector in a common coordinate system, and

wherein the instructions are executable by the computer to;

calibrate the plurality of sensors and the projector using the mapping information, and

align the projector and a touch sensitive surface, the projector to project an image onto the touch sensitive surface, and the touch sensitive surface within a field of view of the plurality of sensors to capture an interaction between an object and the touch sensitive surface, the aligning of the projector and the touch sensitive surface comprising aligning a point of a projector display space of the projector and a corresponding point of the touch sensitive surface.

2. The system of claim 1, wherein the instructions are executable by the computer to calculate perspective transformation between the plurality of sensors to derive the mappings between the plurality of sensors.

3. The system of claim 1, wherein the instructions are executable by the computer to map the plurality of sensors and the projector to a reference sensor, wherein the reference sensor is a depth sensor.

4. The system of claim 1, wherein the mappings comprise a three dimension (3D)-to-two dimensional (2D) mapping that maps between 3D coordinates of a first sensor and 2D coordinates of a second sensor in the plurality of sensors.

5. The system of claim 1, wherein the plurality of sensors comprises a depth sensor and a gesture sensor, and the mappings comprise a mapping of data of the depth sensor and the gesture sensor.

6. The system of claim 1, wherein the plurality of sensors comprises a depth sensor, and the mappings comprise a projection mapping between 3D coordinates of the depth sensor and 2D coordinates of the projector.

7. The system of claim 1, wherein the mappings between the plurality of sensors include data related to scale, rotation, translation, and depth.

8. The system of claim 1, wherein one of the plurality of sensors is a depth sensor, and the depth sensor detects a calibration pattern projected by the projector onto the touch-sensitive surface.

9. The system of claim 8, wherein the calibration pattern comprises a checkerboard pattern.

10. The system of claim 8, wherein the calibration pattern is used in relation to the mappings between the plurality of sensors and the projector in the common coordinate system.

11. The system of claim 1, wherein the plurality of sensors comprise a plurality of cameras, some of which are used for depth detection, gesture sensing, and stereoscopic stylus tracking.

16. The system of claim 1, wherein the aligning of the point of the projector display space of the projector and the corresponding point of the touch sensitive surface comprises aligning a corner of the projector display space of the projector and a corresponding corner of the touch sensitive surface.

17. The system of claim 1, wherein the instructions are executable by the computer to:

determine an offset between the point of the projector display space of the projector and the corresponding point of the touch sensitive surface, and

map, based on the offset, the corresponding point of the touch sensitive surface to the point of the projector display space of the projector to perform the aligning of the projector and the touch sensitive surface.

18. The system of claim 1, wherein the plurality of sensors are to capture a touch input made by a user'"'"'s hand or a stylus with the image projected by the projector onto the touch sensitive surface.

12. A method performed by a system comprising a hardware processor for providing a calibration between a plurality of sensors and a projector, comprising:

receiving a calibration pattern, wherein the plurality of sensors comprises a first sensor and a second sensor;

detecting features of the calibration pattern, the features associated with coordinates in a coordinate space of the first sensor;

mapping the coordinates of the detected features to a common coordinate space;

deriving a mapping between coordinates in the coordinate space of a second sensor and the common coordinate space based on the mapped coordinates and the coordinates in the coordinate space of the first sensor; and

aligning a point of a projector display space of the protector and a corresponding point of a touch sensitive surface, the projector to project an image onto the touch sensitive surface, and the touch sensitive surface within a field of view of the plurality of sensors to capture an interaction between an object and the touch sensitive surface.

13. The method of claim 12, further comprising calculating a perspective geometry between the first sensor and the second sensor.

14. The method of claim 12, further comprising deriving a projective mapping between the plurality of sensors and the projector.

15. The method of claim 14, wherein the plurality of sensors comprises a depth sensor, and the projective mapping is between 3D coordinates of the depth sensor and 2D coordinates of the projector.

19. The method of claim 12, further comprising:

determining an offset between the point of the projector display space of the projector and the corresponding point of the touch sensitive surface, and

mapping, based on the offset, the corresponding point of the touch sensitive surface to the point of the projector display space of the projector to perform aligning of the projector and the touch sensitive surface.

20. A non-transitory machine-readable storage medium storing instructions that upon execution cause a system to:

receive a calibration pattern;

detect features of the calibration pattern, the features associated with coordinates in a coordinate space of a first sensor of a plurality of sensors;

map coordinates of the detected features to a common coordinate space;

derive a mapping between coordinates in the coordinate space of a second sensor of the plurality of sensors and the common coordinate space based on the mapped coordinates and the coordinates in the coordinate space of the first sensor; and

align a point of a projector display space of the projector and a corresponding point of a touch sensitive surface, the projector to project an image onto the touch sensitive surface, and the touch sensitive surface within a field of view of the plurality of sensors to capture a touch interaction between an object and the touch sensitive surface.