Systems and methods for time-multiplexing temporal pixel-location data and regular image projection for interactive projection

• US 9,462,239 B2
• Filed: 07/15/2014
• Issued: 10/04/2016
• Est. Priority Date: 07/15/2014
• Status: Active Grant

• Current Assignee: Fujifilm Business Innovation Corp. (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Inventors: Liu, Qiong, Tang, Hao, Chiu, Patrick, Ma, Shang
• Primary Examiner(s): YENKE, BRIAN P
• Application Number:
  US14/332,370
• Publication Number:
  US 20160021346A1
• Time in Patent Office: 812 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03 Arrangements for converting the position or the displacement of a member into a coded form
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: H04N 9/3105 for displaying all colours simultaneously, e.g. by using two or more electronic spatial light modulators simultaneous projection in colour photography G03B33/10; beam splitting or combining systems per se G02B27/10
• CPC Class Codes: H04N 9/3155 for controlling the light source light source control per se H05B35/00 – H05B47/00; control of an illumination source for displays in general G09G3/3406

Abstract

A system and method for generating a projected image signal encoded with position information. The temporal pixel-location data is hidden within visible light signal used for regular image projection. This enables the user to utilize one or multiple ring-shaped light receivers to implement touch or multi-touch functions on a regular image content generated using light from the same projector. The temporal light signal generated by the projector is sliced into multiple temporal segments. The segments include segments of two different types. A segment of the first type is used for carrying a temporal position signal of each pixel in the projector signal. On the other hand, a segment of the second type is used to adjust the color of each pixel. With the color adjustment segment projected to each image pixel, a meaningful visible content of a regular image can be displayed despite the presence of the position-encoded signal segments.

Claims

1. A computer-implemented method performed in connection with a computerized system comprising a processing unit, a memory and a projector, the computer-implemented method comprising:

a. receiving an image data, the image data comprising at least a color value for each of a plurality of pixels of an image, the color value representing a color of said each of the plurality of image pixels, wherein each of the plurality of image pixels is characterized by pixel coordinates;

b. causing the projector to project a temporal light signal to create a projection of the image on a projection surface;

c. encoding the temporal projector light signal, for each pixel of the projected image, with an information segment comprising the pixel coordinates of the each image pixel of the plurality of image pixels;

d. additionally encoding the temporal projector light signal with a color compensation segment configured to reproduce substantially the color of said each image pixel of the plurality of image pixels on the projection surface, wherein the information segment and the color compensation segment are time-multiplexed within the temporal projector light signal; and

e. detecting the temporal light signal using a mobile device positioned within an illumination field of the projector and determining a position of the mobile device based on the detected temporal light signal, wherein the temporal projector light signal is a visible light signal.

2. The computer-implemented method of claim 1, wherein the pixel coordinates are descriptive of a location of the pixel within the image.

3. The computer-implemented method of claim 1, wherein the pixel coordinates are descriptive of a location of the pixel within a coordinate system of the projector.

4. The computer-implemented method of claim 1, wherein the temporal projector light signal is encoded with the pixel coordinates using a sequential binary code comprising a sequential plurality of light pulses.

5. The computer-implemented method of claim 4, wherein in the sequential binary code, a first binary digit is represented by a first light pulse followed by a first pause and a second binary digit is represented by a second pause followed by a second light pulse.

6. The computer-implemented method of claim 1, wherein the sequential plurality of light pulses of the temporal projector light signal are generated using a white color light.

7. The computer-implemented method of claim 1, wherein the color compensation segment comprises a sequential plurality of light pulses of a plurality of light colors.

8. The computer-implemented method of claim 7, wherein the light pulses in the sequential plurality of light pulses have widths determined based on the color value of the corresponding image pixel.

9. The computer-implemented method of claim 7, wherein the sequential plurality of light pulses comprises a red light pulse, a green light pulse and a blue light pulse, wherein the widths of the red light pulse, the green light pulse and the blue light pulse are determined based on the color value of the corresponding image pixel in an RGB color space.

10. The computer-implemented method of claim 1, further comprising using the determined position of the mobile device to cause the computerized system or the mobile device to perform a predetermined action.

11. A non-transitory computer-readable medium embodying a set of computer-executable instructions, which, when executed in connection with a computerized system comprising a processing unit, a memory and a projector, cause the computerized system to perform a method comprising:

a. receiving an image data, the image data comprising at least a color value for each of a plurality of pixels of an image, the color value representing a color of said each of the plurality of image pixels, wherein each of the plurality of image pixels is characterized by pixel coordinates;

b. causing the projector to project a temporal light signal to create a projection of the image on a projection surface;

c. encoding the temporal projector light signal, for each pixel of the projected image, with an information segment comprising the pixel coordinates of the each image pixel of the plurality of image pixels;

d. additionally encoding the temporal projector light signal with a color compensation segment configured to reproduce substantially the color of said each image pixel of the plurality of image pixels on the projection surface, wherein the information segment and the color compensation segment are time-multiplexed within the temporal projector light signal; and

e. detecting the temporal light signal using a mobile device positioned within an illumination field of the projector and determining a position of the mobile device based on the detected temporal light signal, wherein the temporal projector light signal is a visible light signal.

12. The non-transitory computer-readable medium of claim 11, wherein the temporal projector light signal is encoded with the pixel coordinates using a sequential binary code comprising a sequential plurality of light pulses.

13. The non-transitory computer-readable medium of claim 12, wherein in the sequential binary code, a first binary digit is represented by a first light pulse followed by a first pause and a second binary digit is represented by a second pause followed by a second light pulse.

14. The non-transitory computer-readable medium of claim 11, wherein the sequential plurality of light pulses of the temporal projector light signal are generated using a white color light.

15. The non-transitory computer-readable medium of claim 11, wherein the color compensation segment comprises a sequential plurality of light pulses of a plurality of light colors.

16. The non-transitory computer-readable medium of claim 15, wherein the light pulses in the sequential plurality of light pulses have widths determined based on the color value of the corresponding image pixel.

17. The non-transitory computer-readable medium of claim 15, wherein the sequential plurality of light pulses comprises a red light pulse, a green light pulse and a blue light pulse, wherein the widths of the red light pulse, the green light pulse and the blue light pulse are determined based on the color value of the corresponding image pixel in an RGB color space.

18. A computerized system comprising a processing unit, a projector and a memory storing a set of instructions, which, when executed by the processing unit, causes the computerized system to perform a method comprising:

a. receiving an image data, the image data comprising at least a color value for each of a plurality of pixels of an image, the color value representing a color of said each of the plurality of image pixels, wherein each of the plurality of image pixels is characterized by pixel coordinates;

b. causing the projector to project a temporal light signal to create a projection of the image on a projection surface;

c. encoding the temporal projector light signal, for each pixel of the projected image, with an information segment comprising the pixel coordinates of the each image pixel of the plurality of image pixels; and

d. additionally encoding the temporal projector light signal with a color compensation segment configured to reproduce substantially the color of said each image pixel of the plurality of image pixels on the projection surface, wherein the information segment and the color compensation segment are time-multiplexed within the temporal projector light signal; and

e. detecting the temporal light signal using a mobile device positioned within an illumination field of the projector and determining a position of the mobile device based on the detected temporal light signal, wherein the temporal projector light signal is a visible light signal.

19. The computerized system of claim 18, wherein the temporal projector light signal is encoded with the pixel coordinates using a sequential binary code comprising a sequential plurality of light pulses.