Image processing apparatus, image processing method, and storage medium

• US 20020001029A1
• Filed: 06/28/2001
• Published: 01/03/2002
• Est. Priority Date: 06/29/2000
• Status: Active Grant

• Current Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Original Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Inventors: Abe, Tsutomu
• Granted Patent: US 6,970,600 B2
• Time in Patent Office: Days
• US Class Current: 348/49
• CPC Class Codes: G01B 11/2509 Color coding
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1434 Special illumination such as grating, reflections or deflections, e.g. for characters with relief
• CPC Class Codes: G06V 30/228 of three-dimensional handwriting, e.g. writing in the air

Abstract

The present invention provides an image processing apparatus that efficiently identifies input characters and the like by an intensity image analysis based on range information. The image processing apparatus performs distance measurement by performing three-dimensional measurement by pattern projection to characters written to a manuscript, whiteboard, and the like, gets an intensity image used as a so-called actual image and an image for distance measurement in parallel, and identifies input characters by an intensity image analysis based on range information. The image processing apparatus picks up characters, patterns, and the like written onto paper or the like with a pen, performs geometric transformation for picked-up images, and performs natural input character analysis and reading processing. Furthermore, by comparing images picked up in a time-series, noise elimination and manuscript position modifications become possible.

Claims

1. An image processing apparatus, comprising:

a three-dimensional image pickup part that includes a projecting part that projects a pattern, a first image pickup part that picks up an intensity image and a projection pattern image from the direction of an optical axis of the projecting part, and a second image pickup part that picks up the projection pattern image from a direction different from the optical axis of the projecting part, the three-dimensional image pickup part creating first range information based on a pattern picked up by the second image pickup part; and

a geometric transformation part that performs geometric transformation for the intensity image picked up by the first image pickup part, based on the first range information.

2. The image processing apparatus according to claim 1, wherein, for an area where the amount of change of the pattern picked up by the first image pickup part with respect to the projection pattern is equal to or greater than a predetermined value, new code corresponding to the pattern picked up by the first image pickup part is assigned, and the first range information is created from the pattern picked up by the second image pickup part based on the new code.

3. The image processing apparatus according to claim 1, further comprising:

a frame data comparison part that makes comparison between frame data images picked up in a time-series by the three-dimensional image pickup part; and

an image processing part that eliminates noise data from the frame data image based on a result of the comparison between the frame data images in the frame data comparison part.

4. The image processing apparatus according to claim 1, further comprising:

a frame data comparison part that makes comparison between frame data images picked up in a time-series by the three-dimensional image pickup part; and

an image processing part that modifies a position of the frame data image based on a result of the comparison between the frame data images in the frame data comparison part.

5. The image processing apparatus according to claim 1, further comprising:

a storage part that stores, as initial frame data, an initial image of frame data picked up in a time-series by the three-dimensional image pickup part;

a frame data comparison part that makes comparison between the frame data images picked up in a time-series by the three-dimensional image pickup part; and

an image processing part that extracts only differential data as storage data based on a result of the comparison in the frame data comparison part between the initial frame data and frame data subsequently picked up.

6. The image processing apparatus according to claim 1, wherein:

the projecting part has a light source to emit light of an invisible region; and

the first and second image pickup parts have a filter for transmitting light of invisible region and a filter for cutting off light of an invisible region, and pick up the projection pattern image and intensity image in parallel.

7. The image processing apparatus according to claim 1, wherein, for an area where the amount of change of the pattern picked up by the first image pickup part with respect to the projection pattern by the projecting part is less than a predetermined value, second range information is created by bringing the area into correspondence with intensity information obtained by the first and second image pickup parts.

8. The image processing apparatus according to claim 1, wherein the second image pickup part includes plural image pickup parts that pick up the measurement target at different angles, and range information is created based on projection patterns respectively picked up by the plural image pickup parts of the second image pickup part.

9. An image processing method, comprising:

projecting a pattern by a projecting part;

picking up an intensity image and a projection pattern image by a first image pickup part from an optical axis direction of the projecting part, and picking up the projection pattern image by a second image pickup part from a direction different from the optical axis direction of the projecting part;

creating first range information based on the pattern picked up by the second image pickup part; and

performing geometric transformation for the intensity image produced by the first image pickup part based on the range information.

10. The image processing method according to claim 9, wherein the range information creation step includes the step of:

for an area where the amount of change of the pattern picked up by the first image pickup part with respect to the projection pattern is equal to or greater than a predetermined value, assigning new code corresponding to the pattern picked up by the first image pickup part, and creating the first range information from the pattern picked up by the second image pickup part based on the new code.

11. The image processing method according to claim 9, further comprising:

making comparison between frame data images picked up in a time-series in the image pickup step; and

eliminating noise data from the frame data image based on a result of the comparison between the frame data images in the frame data comparison step.

12. The image processing method according to claim 9, further comprising:

making comparison between frame data images picked up in a time-series in the image pickup step; and

modifying a position of the frame data image based on a result of the comparison between the frame data images in the frame data comparison step.

13. The image processing method according to claim 9, further comprising:

storing in a storage part an initial image of frame data picked up in a time-series in the image pickup step as initial frame data;

making comparison between frame data images picked up in a time-series in the image pickup step; and

extracting only differential data as storage data based on a result of the comparison in the frame data comparison step between the initial frame data and frame data subsequently picked up.

14. The image processing method according to claim 9, wherein:

a pattern light is formed by invisible-region light by using an invisible-region light source using infrared or ultraviolet light as the light source; and

the pattern projection image and intensity image are picked up in parallel.

15. The image processing method according to claim 9, further comprising the step of:

for an area where the amount of change of the pattern picked up by the first image pickup part with respect to the projection pattern by the projecting part is less than a predetermined value, creating second range information by bringing the area into correspondence with intensity information obtained by the first and second image pickup parts.

16. The image processing method according to claim 9, wherein:

the second image pickup part includes plural image pickup parts that pick up the measurement target at different angles, and includes the step of creating range information based on projection patterns respectively picked up by the plural image pickup parts of the second image pickup part.

17. An image processing apparatus, comprising:

a projecting part that projects light to an image holding medium to form an image thereon;

an image pickup part that picks up the image on the image holding medium projected by the projecting part;

an intensity image acquisition part that acquires an intensity image based on the image picked up by the image pickup part;

a range information acquisition part that acquires range information from the picked-up image;

a geometric transformation part that performs geometric transformation for the intensity image based on the range information acquired in the range information acquisition part; and

an image extracting part that extracts difference between a geometric-transformed intensity image and the intensity image acquired in advance.

18. The image processing apparatus according to claim 17, wherein the image holding medium is one of a manuscript sheet, whiteboard, blackboard, screen, wall, and screen projection sheet.

19. The image processing apparatus according to claim 17, wherein the intensity image acquired in advance as a processing target in the image extracting part is a preceding frame image inputted precedent to the geometric transformation part.

20. The image processing apparatus according to claim 17, further comprising:

a storage part that stores image data,

wherein the intensity image acquired in advance as a processing target in the image extracting part is the image data stored in advance in the storage part.

21. The image processing apparatus according to claim 17, comprising:

a document database in which plural pieces of document format data are stored; and

a document identifying part that performs matching between the geometric-transformed intensity image and the pieces of document format data stored in the document database,

wherein the image extracting part extracts differences between the geometric-transformed intensity image and the pieces of document format data stored in the document database.

22. The image processing apparatus according to claim 17, further comprising a character transformation processing part that reads character data extracted by the image extracting part and transforms it to character data replaceable as a code value.

23. The image processing apparatus according to claim 17, further comprising:

an authentication information database in which handwriting history data of registered users is stored; and

an authentication processing part that inputs the geometric-transformed intensity image and performs matching between the input image and handwriting history data stored in the authentication information database.

24. The image processing apparatus according to claim 23, wherein:

the authentication information database stores handwriting history data and signature shape data of registered users; and

the authentication processing part inputs the geometric-transformed intensity image and performs matching between the input image and the handwriting history data stored in the authentication information database, and between the input image and the signature shape data.

25. The image processing method according to claim 17, further comprising a display part that displays an image produced as a result of performing geometric transformation for the intensity image, based on the range information in the geometric transformation part.

26. The image processing apparatus claim 17, further comprising:

a storage part that stores range information acquired by the range information acquisition part,

wherein a distance between the image holding medium and the image pickup part is fixed, and the geometric transformation part performs geometric transformation for the intensity image based on the range information stored in the storage part.

27. An image processing method, comprising:

projecting light to an image holding medium to form an image thereon;

picking up the image projected on the image holding medium;

acquiring an intensity image based on the image picked up in the image pickup step;

acquiring range information from the picked-up image;

performing geometric transformation for the intensity image based on the range information acquired in the range information acquisition step; and

extracting difference between the geometric-transformed intensity image and the intensity image acquired in advance.

28. The image processing method according to claim 27, wherein the image holding medium is one of a manuscript sheet, whiteboard, blackboard, screen, wall and screen projection sheet.

29. The image processing method according to claim 27, wherein the intensity image acquired in advance as a processing target in the image extracting step is a preceding frame image inputted precedent to the geometric transformation step.

30. The image processing method according to claim 27, wherein the intensity image acquired in advance as a processing target in the image extracting step is image data stored in advance in a storage part.

31. The image processing method according to claim 27, further comprising:

storing plural pieces of document format data in a document database; and

performing matching between a geometric-transformed intensity image and the pieces of document format data stored in the document database,

wherein the image extracting step extracts difference between the geometric-transformed intensity image and the pieces of document format data stored in the document database.

32. The image processing method according to claim 27, further comprising:

reading character data extracted in the image extracting step and transforming it to character data replaceable as a code value.

33. The image processing method according to claim 27, further comprising:

storing handwriting history data of registered users in a authentication information database; and

inputting the geometric-transformed intensity image and performing matching between the input image and the handwriting history data stored in the authentication information database.

34. The image processing apparatus according to claim 33, further comprising:

in addition to the handwriting history data, storing signature shape data of registered users in the authentication information database; and

inputting the geometric-transformed intensity image and performing matching between the input image and the handwriting history data stored in the authentication information database, and between the input image and the signature shape data.

35. The image processing method according to claim 27, further comprising:

displaying an image produced as a result of performing geometric transformation for the intensity image based on the range information.

36. The image processing method according to claim 27, further comprising:

storing range information acquired in the range information acquiring step,

wherein a distance between an image holding medium and the image pickup part is fixed and the geometric transformation for the intensity image is performed based on the range information stored in the storage step.

37. A storage medium readable by a computer, the storage medium storing a program of instructions executable by the computer to perform method steps for performing image processing, the method comprising the steps of:

projecting light to an image holding medium to form an image thereon;

picking up the image formed on the image holding medium;

acquiring an intensity image based on the image picked up in the image pickup step;

acquiring range information from the picked-up image;

performing geometric transformation for the intensity image based on the range information acquired in the range information acquisition step; and

extracting difference between the geometric-transformed intensity image and the intensity image acquired in advance.

38. An image processing apparatus, comprising:

a projecting part that projects light;

an image pickup part that picks up the projected light;

an intensity image acquisition part that acquires an intensity image from the picked-up light;

a range information acquisition part that acquires range information from the picked-up light;

a geometric transformation part that performs geometric transformation for the intensity image based on the range information; and

an image extracting part that extracts difference between the geometric-transformed intensity image and the intensity image acquired in advance.

39. An image processing method, comprising:

projecting light;

picking up the projected light;

acquiring an intensity image from the picked-up light;

acquiring range information from the picked-up light;

performing geometric transformation for the intensity image based on the range information; and

extracting difference between the geometric-transformed intensity image and the intensity image acquired in advance.

40. A storage medium readable by a computer, the storage medium storing a program of instructions executable by the computer to perform method steps for performing image processing, the method comprising the steps of:

projecting light;

picking up the projected light;

acquiring an intensity image from the picked-up light;

acquiring range information from the picked-up light;

performing geometric transformation for the intensity image based on the range information; and

extracting difference between the geometric-transformed intensity image and the intensity image acquired in advance.