Image display device, image display method, program, and projection system

• US 7,039,253 B2
• Filed: 07/23/2002
• Issued: 05/02/2006
• Est. Priority Date: 07/24/2001
• Status: Expired due to Fees

• Current Assignee: Casio Computer Company Limited
• Original Assignee: Casio Computer Company Limited
• Inventors: Matsuda, Takashi, Matsui, Shinichi, Kasahara, Tsuyoshi, Nakagomi, Kouichi, Matsuoka, Takeshi, Iizuka, Nobuo
• Primary Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US10/201,371
• Publication Number:
  US 20030021492A1
• Time in Patent Office: 1,379 Days
• Field of Search: 382/154, 382/266, 382/295, 382/293, 382/312, 382/276, 348/183, 348/761, 349/2, 349/25, 358/471, 359/250, 359/252, 651/02, 651/06, 653/92, 65/301.3, 345/2.2, 345/156, 345/157, 345/661, 345/175, 345180-183
• US Class Current: 382/295
• CPC Class Codes: G06F 3/0386 for light pen
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886

Abstract

A data processing apparatus includes a central control section, an image analysis section, a RAM (Random Access Memory), an external memory section and a user operation input section. The central control section is connected to the image analysis section, the RAM and the external memory section via a bus. Based on image data prepared by an image input section, the image analysis section performs a process of specifying where on the display screen of a display section a portion specified by the image input section lies.

Claims

1. An image display device comprising:

an image pickup section which picks up a spot and an area including said spot from an arbitrary position;

a generation section which generates image data of said area picked up by said image pickup section;

a display section which displays an image based on said image data generated by said generation section;

a processing section which processes said image data in such a way that said image data is adequately displayed on a display screen of said display section;

a measuring section which measures positional coordinates of said spot in said area; and

a coordinate conversion section which converts said positional coordinates of said spot measured by said measuring section to positional coordinates that are reflected on image data after the processing acquired as a result of a process performed by said processing section;

wherein the area before processing, represented by the image data before processing by the processing section, is a quadrangle, and the area after processing, represented by the image data after the processing by the processing section, is a rectangle; and

wherein said processing section;

(i) acquires a first pair of lines, each of which passes through the spot in said area after processing and perpendicularly intersects two opposite sides of said rectangle that defines said area after processing, (ii) acquires a second pair of lines that internally divide two opposite sides of said quadrangle that defines said area before processing, by an interior division ratio by which said first pair of lines internally divide said two opposite sides of said rectangle, and (iii) processes said image data in such a way that the spot at a point of intersection of said second pair of lines is positioned at a point of intersection of said first pair of lines after processing.

2. The image display device according to claim 1, wherein said picked-up area is part of a projected image, said spot is a dot of light on said projected image, and said light has a specific wavelength and a luminance that is set equal to or higher than a luminance of another portion of said projected image.

3. The image display device according to claim 1, wherein said image pickup section picks up said display screen from an arbitrary position.

4. The image display device according to claim 1, wherein said area is an area surrounded by at least tour points whose positional coordinates have been set beforehand.

5. The image display device according to claim 4, wherein said processing section comprises an area setting section which inputs information designating said at least four points surrounding said area and sets a region surrounded by said points indicated by said input information as said area.

6. The image display device according to claim 4, wherein said processing section comprises an area setting section which specifies at least four spots included in an image represented by image data before processing and sets said specified spots as said points surrounding said area.

7. The image display device according to claim 4, wherein said display section comprises a section which displays a figure for extracting an area surrounded by at least four points in an area to be picked up by said image pickup section, and

wherein said processing section comprises an area setting section which extracts at least four points from said figure for extracting said area represented by the image data before processing and sets said extracted points as said points surrounding said area.

8. The image display device according to claim 1, wherein said area comprises a plurality of regions, and

wherein said measuring section specifies a region in which said spot is included, and measures said positional coordinates of said spot in said area by measuring positional coordinates of said spot in said specified region.

9. The image display device according to claim 1, wherein said display section comprises a section which displays a curve indicating a track along which the spot has moved, over an image based on the image data after processing.

10. The image display device according to claim 1, wherein said processing section comprises a section which specifies one of a plurality of levels to which a luminance of the spot represented by the image data before processing corresponds, and

wherein said display section displays the spot to have said luminance level specified by said processing section.

11. The image display device according to claim 1, wherein said measuring section detects a spot identification pattern comprising a plurality of spots from the image data before processing and measures positional coordinates of said detected spot identification pattern as the positional coordinates of the spot.

12. The image display device according to claim 1, wherein said measuring section comprises a section which specifies positions and velocity of the spot at two times based on a plurality of pieces of the image data before processing, and

wherein said processing section determines an estimated position of said spot between said two times based on said positions and velocity of said spot at said two times specified by said measuring section, and processes the image data before processing in such a way that an image having said spot positioned at said determined estimated position.

13. The image display device according to claim 1, wherein said measuring section specifies positions of the spot at a plurality of times based on a plurality of pieces of image data before processing, determines a representative position representing a position of said spot in a time zone including said plurality of times based on said specified positions, and treats positional coordinates of said determined representative position as results of measuring the positional coordinates of said spot.

14. The image display device according to claim 1, further comprising a color change detection section which detects a change in color of the spot represented by a plurality of pieces of the image data before processing based on said plurality of pieces of the image data.

15. The image display device according to claim 1, wherein said measuring section comprises a section which specifies a velocity of the spot based on a plurality of pieces of the image data before processing, and

wherein said processing section determines a position in an image indicated by the image data after processing which said spot is to occupy next, based on said velocity specified by said measuring section and a latest position in said image indicated by the image data after processing which said spot has occupied, and processes the image data in such a way that said spot is positioned at said determined position.

16. The image display device according to claim 1, further comprising a spot-shape determining section which determines a shape of the spot to be represented by the image data after processing, based on said positional coordinates of said spot measured by said measuring section.

17. The image display device according to claim 1, wherein said processing section comprises a section which externally receives a signal indicating that a light source of said spot is emitting light and processes image data in response to said received signal.

18. An image display device comprising:

an image pickup section which picks up a spot and an area including said spot from an arbitrary position;

a generation section which generates image data of said area picked up by said image pickup section;

a display section which displays an image based on said image data generated by said generation section;

a processing section which processes said image data in such a way that said image data is adequately displayed on a display screen of said display section;

a measuring section which measures positional coordinates of said spot in said area; and

a coordinate conversion section which converts said positional coordinates of said spot measured by said measuring section to positional coordinates that are reflected on image data after the processing acquired as a result of a process performed by said processing section;

wherein the area before processing, represented by the image data before processing by the processing section, is a quadrangle, and the area after processing, represented by image data after the processing performed by said processing section, is a rectangle;

wherein said processing section performs a process of;

(i) determining in which one of four quadrangles said spot is included, said four quadrangles being obtained by quadrisecting said quadrangle that defines said area before processing by two lines, each of which connects a point of intersection of one of two sets of two opposite sides of said quadrangle to a point of intersection of diagonal lines of said quadrangle, and then (ii) moving by a predetermined amount a point lying in said rectangle that defines said area after processing in a direction of a vertex equivalent to a vertex of the one of the four quadrangles determined as including said spot that also serves as a vertex of said quadrangle defining said area before processing;

wherein the processing section repeats said process, by regarding the one of the four quadrangles determined as including said spot as a quadrangle defining said area before processing for a next repetition of the process, until said point lying in said rectangle that defines said area after processing substantially converges to a given position, and a position of said point which reaches convergence becomes a position of the spot in the area after processing; and

wherein when said process is repeated, said predetermined amount by which the point in the rectangle is moved becomes smaller as a number of times the quadrisecting of said quadrangle is performed becomes larger.

19. An image display method comprising:

picking up a spot and an area including said spot from an arbitrary position;

generating image data of said picked-up area;

displaying an image based on said generated image data on a display screen;

processing said image data in such a way that said image data is adequately displayed on said display screen;

measuring positional coordinates of said spot in said area; and

converting said measured positional coordinates of said spot to positional coordinates that are reflected on image data after the processing acquired as a result of the processing;

wherein the area before the processing represented by image data before processing, is a quadrangle, and the area after processing, represented by the image data after the processing, is a rectangle; and

wherein said processing comprises;

(i) acquiring a first pair of lines, each of which passes through the spot in said area after processing and perpendicularly intersect two opposite sides of said rectangle that defines said area after processing, (ii) acquiring a second pair of lines that internally divide two opposite sides of said quadrangle that defines said area before processing, by an interior division ratio by which said first pair of lines internally divide said opposite two sides of said rectangle, and (iii) processing said image data in such a way that the spot at a point of intersection of said second pair of lines is positioned at a point of intersection of said first pair of lines after processing.

20. The image display method according to claim 19, wherein said picked-up area is part of a projected image, said spot is a dot of light on said projected image, and said light has a specific wavelength and a luminance that is set equal to or higher than a luminance of another portion of said projected image.

21. The image display method according to claim 19, wherein said image pickup step picks up said display screen from an arbitrary position.

22. The image display method according to claim 19, wherein said area is an area surrounded by at least four points whose positional coordinates have been set beforehand.

23. The image display method according to claim 22, further comprising setting the area by inputting information designating said at least four points surrounding said area and setting a region surrounded by said points indicated by said input information as said area.

24. The image display method according to claim 22, further comprising setting the area by setting at least four spots included in an image represented by the image data before processing and setting said specified spots as said points surrounding said area.

25. The image display method according to claim 22, further comprising displaying a figure for extracting an area surrounded by at least four points in an area to be picked up in said image pickup step, and

wherein said image display method further comprises setting the area by extracting at least four points from said figure for extracting said area represented by the image data before processing and setting said extracted points as said points surrounding said area.

26. The image display method according to claim 19, wherein said area comprises a plurality of regions, and

wherein said measuring the positional coordinates comprises specifying a region in which said spot is included, and measuring said positional coordinates of said spot in said area by measuring positional coordinates of said spot in said specified region.

27. The image display method according to claim 19, further comprising displaying a curve indicating a track along which the spot has moved, over an image based on the image data after processing.

28. The image display method according to claim 19, further comprising specifying one of a plurality of levels to which a luminance of the spot represented by the image data before processing corresponds, and

wherein the spot is displayed to have said specified luminance level.

29. The image display method according to claim 19, wherein said measuring the positional coordinates comprises detecting a spot identification pattern comprising a plurality of spots from the image data before processing, and measuring positional coordinates of said detected spot identification pattern as the positional coordinates of the spot.

30. The image display method according to claim 19, further comprising specifying positions and velocity of the spot at two times based on a plurality of pieces of the image data before processing, and

wherein said processing comprises determining an estimated position of said spot between said two times based on said positions and velocity of said spot at said two times, and processing the image data before processing in such a way that an image having said spot positioned at said determined estimated position is shown.

31. The image display method according to claim 19, said measuring the positional coordinates comprises specifying positions of the spot at a plurality of times based on a plurality of pieces of the image data before processing, determining a representative position representing a position of said spot in a time zone including said plurality of times based on said specified positions, and treating positional coordinates of said determined representative position as results of measuring the positional coordinates of said spot.

32. The image display method according to claim 19, further comprising detecting a change in color of the spot represented by a plurality of pieces of the image data before processing based on said plurality of pieces of the image data.

33. The image display method according to claim 19, further comprising specifying a velocity of the spot based on a plurality of pieces of the image data before processing, and

wherein said processing comprises determining a position in an image indicated by the image data after processing which said spot is to occupy next, based on said specified velocity and a latest position in said image indicated by the image data after processing which said spot has occupied, and processing the image data in such a way that said spot is positioned at said determined position.

34. The image display method according to claim 19, further comprising determining a shape of the spot to be represented by the image data after processing, based on said measured positional coordinates of said spot.

35. The image display method according to claim 19, further comprising receiving a signal indicating that a light source of said spot is emitting light, wherein said processing is performed in response to said received signal.

36. An image display method comprising:

picking up a spot and an area including said spot from an arbitrary position;

generating image data of said pickedup area;

displaying an image based on said generated image data on a display screen;

processing said image data in such a way that said image data is adequately displayed on said display screen;

measuring positional coordinates of said spot in said area; and

converting said measured positional coordinates of said spot to positional coordinates that are reflected on image data after the processing acquired as a result of the processing;

wherein the area before processing, represented by the image data before processing, is a quadrangle, and the area after processing, represented by the image data after the processing, is a rectangle;

wherein said processing comprises performing a process of;

(i) determining in which one of four quadrangles said spot is included, said four quadrangles being obtained by quadrisecting said quadrangle that defines said area before processing by two lines, each of which connects a point of intersection of one of two sets of two opposite sides of said quadrangle to a point of intersection of diagonal lines of said quadrangle, and then (ii) moving by a predetermined amount a point lying in said rectangle that defines said area after processing in a direction of a vertex equivalent to a vertex of the one of the four quadrangles determined as including said spot that also serves as a vertex of said quadrangle;

wherein said processing comprises repeating said process, by regarding the one of the four quadrangles determined as including said spot as a quadrangle defining said area before processing for a next iteration of the process, until said point lying in said rectangle that defines said area after processing substantially converges to a given position, and a position of said point which reaches convergence becomes a position of the spot represented in the area after processing; and

wherein when said process is repeated, said predetermined amount by which the point in the rectangle is moved becomes smaller as a number of times the quadrisecting of said quadrangle is performed becomes larger.

37. A computer readable storage medium having program stored thereon that is executable by a computer connected to an image display device and an image pickup section to cause the computer to function as:

an image pickup control section which causes said image pickup section to pick up a spot and an area including said spot from an arbitrary position;

a generation section which generates image data of said area picked up by said image pickup control section;

a display control section which displays an image based on said image data generated by said generation section on a display screen of said image display device;

a processing section which processes said image data in such a way that said image data is adequately displayed on said display screen;

a measuring section which measures positional coordinates of said spot in said area; and

a coordinate conversion section which converts said positional coordinates of said spot measured by said measuring section to positional coordinates that are reflected on image data after the processing acquired as a result of a process performed by said processing section;

wherein the area before processing, represented by the image data before processing by the processing section, is a quadrangle, and the area after processing, represented by the image data after the processing by the processing section, is a rectangle; and

wherein said processing section;

(i) acquires a first pair of lines, each of which passes through the spot in said area after processing and perpendicularly intersects two opposite sides of said rectangle that defines said area after processing, (ii) acquires a second pair of lines that internally divide two opposite sides of said quadrangle that defines said area before processing, by an interior division ratio by which said first pair of lines internally divide said two opposite sides of said rectangle, and (iii) processes said image data in such a way that the spot at a point of intersection of said second pair of lines is positioned at a point of intersection of said first pair of lines after processing.

38. A computer readable storage medium having program stored thereon that is executable by a computer connected to an image display device and a plurality of image pickup sections to cause the computer to function as;

an image pickup control section which causes said plurality of image pickup sections to pick up a spot and an area including said spot from a plurality of arbitrary positions;

a generation section which generates image data of a plane in space picked up by said image pickup control section;

a display control section which displays an image based on said image data generated by said generation section on said image display device;

a processing section which processes said image data generated by said generation section in such a way that said image data is adequately displayed on a display screen of said image display device;

a measuring section which measures positional coordinates of said spot in said space;

a spot-shape determining section which determines a shape of a spot to be represented by image data after processing performed by said processing section, based on said positional coordinates of said spot measured by said measuring section; and

a coordinate conversion section which converts said positional coordinates of said spot measured by said measuring section to positional coordinates that are reflected on the image data after the processing by the processing section;

wherein the area before processing, represented by the image data before processing by the processing section, is a quadrangle, and the area after processing, represented by the image data after the processing by the processing section, is a rectangle; and

wherein said processing section;

(i) acquires a first pair of lines, each of which passes through the spot in said area after processing and perpendicularly intersects two opposite sides of said rectangle that defines said area after processing, (ii) acquires a second pair of lines that internally divide two opposite sides of said quadrangle that defines said area before processing, by an interior division ratio by which said first pair of lines internally divide said two opposite sides of said rectangle, and (iii) processes said image data in such a way that the spot at a point of intersection of said second pair of lines is positioned at a point of intersection of said first pair of lines after processing.

39. A projection system including a projector, said system comprising:

an image pickup section which picks up an area of an image projected by said projector from an arbitrary position;

a generation section which generates image data of said area picked up by said image pickup section;

a processing section which processes said image data generated by said generation section in such a way that the image data is adequate for display on a display screen;

a detection section which detects a spot area which emits light having a specific wavelength from said picked-up area;

a measuring section which measures positional coordinates of said spot area detected by said detection section; and

a coordinate conversion section which converts said positional coordinates measured by said measuring section to positional coordinates that are reflected on image data processed by said processing section;

wherein the area before processing, represented by the image data before processing by the processing section, is a quadrangle, and the area after processing, represented by the image data after the processing by the processing section, is a rectangle; and

wherein said processing section;

(i) acquires a first pair of lines, each of which passes through the spot area in said area after processing and perpendicularly intersects two opposite sides of said rectangle that defines said area after processing, (ii) acquires a second pair of lines that internally divide two opposite sides of said quadrangle that defines said area before processing, by an interior division ratio by which said first pair of lines internally divide said two opposite sides of said rectangle, and (iii) processes said image data in such a way that the spot area at a point of intersection of said second pair of lines is positioned at a point of intersection of said first pair of lines after processing.

40. A computer readable storage medium having program stored thereon that is executable by a computer connected to an image display device and an image pickup section to cause the computer to function as:

an image pickup control section which causes said image pickup section to pick up a spot and an area including said spot from an arbitrary position;

a generation section which generates image data of said area picked up by said image pickup control section;

a display control section which displays an image based on said image data generated by said generation section on a display screen of said image display device;

a processing section which processes said image data in such a way that said image data is adequately displayed on said display screen;

a measuring section which measures positional coordinates of said spot in said area; and

a coordinate conversion section which converts said positional coordinates of said spot measured by said measuring section to positional coordinates that are reflected on image data after the processing acquired as a result of a process performed by said processing section;

wherein the area before processing, represented by the image data before processing by the processing section, is a quadrangle, and the area after processing, represented by image data after the processing performed by said processing section, is a rectangle;

wherein said processing section performs a process of;

(i) determining in which one of four quadrangles said spot is included, said four quadrangles being obtained by quadrisecting said quadrangle that defines said area before processing by two lines, each of which connects a point of intersection of one of two sets of two opposite sides of said quadrangle to a point of intersection of diagonal lines of said quadrangle, and then (ii) moving by a predetermined amount a point lying in said rectangle that defines said area after processing in a direction of a vertex equivalent to a vertex of the one of the four quadrangles determined as including said spot that also serves as a vertex of said quadrangle defining said area before processing;

wherein the processing section repeats said process, by regarding the one of the four quadrangles determined as including said spot as a quadrangle defining said area before processing for a next repetition of the process, until said point lying in said rectangle that defines said area after processing substantially converges to a given position, and a position of said point which reaches convergence becomes a position of the spot in the area after processing; and

wherein when said process is repeated, said predetermined amount by which the point in the rectangle is moved becomes smaller as a number of times the quadrisecting of said quadrangle is performed becomes larger.

41. A projection system including a projector, said system comprising:

an image pickup section which picks up an area of an image projected by said projector from an arbitrary position;

a generation section which generates image data of said area picked up by said image pickup section;

a processing section which processes said image data generated by said generation section in such a way that the image data is adequate for display on a display screen;

a detection section which detects a spot area which emits light having a specific wavelength from said pickedup area;

a measuring section which measures positional coordinates of said spot area detected by said detection section; and

a coordinate conversion section which converts said positional coordinates measured by said measuring section to positional coordinates that are reflected on image data processed by said processing section;

wherein the area before processing, represented by the image data before processing by the processing section, is a quadrangle, and the area after processing, represented by image data after the processing performed by said processing section, is a rectangle;

wherein said processing section performs a process of;

(i) determining in which one of tour quadrangles said spot is included, said four quadrangles being obtained by quadrisecting said quadrangle that defines said area before processing by two lines, each of which connects a point of intersection of one of two sets of two opposite sides of said quadrangle to a point of intersection of diagonal lines of said quadrangle, and then (ii) moving by a predetermined amount a point lying in said rectangle that defines said area after processing in a direction of a vertex equivalent to a vertex of the one of the four quadrangles determined as including said spot that also serves as a vertex of said quadrangle defining said area before processing;

wherein the processing section repeats said process, by regarding the one of the four quadrangles determined as including said spot as a quadrangle defining said area before processing for a next repetition of the process, until said point lying in said rectangle that defines said area after processing substantially converges to a given position, and a position of said point which reaches convergence becomes a position of the spot in the area after processing; and

wherein when said process is repeated, said predetermined amount by which the point in the rectangle is moved becomes smaller as a number of times the quadrisecting of said quadrangle is performed becomes larger.