Image input-and-output apparatus

• US 20070025612A1
• Filed: 09/28/2006
• Published: 02/01/2007
• Est. Priority Date: 03/31/2004
• Status: Active Grant

• Current Assignee: Brother Kogyo Kabushiki Kaisha
• Original Assignee: Brother Kogyo Kabushiki Kaisha
• Inventors: SASAKI, Hiroyuki, IWASAKI, Takeo
• Granted Patent: US 7,809,193 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light

Abstract

There is provided an image input-and-output apparatus comprising an imaging unit that takes an image of an object to be photographed to obtain imaging data, a recognizing unit that recognizes object information from the imaging data obtained by the imaging unit, and a projecting unit that projects related information related to the object information recognized by the recognizing unit.

Claims

1. An image input-and-output apparatus, comprising:

an imaging unit that takes an image of an object to be photographed to obtain imaging data;

a recognizing unit that recognizes object information from the imaging data obtained by the imaging unit; and

a projecting unit that projects related information related to the object information recognized by the recognizing unit.

2. The image input-and-output apparatus according to claim 1, further comprising a three-dimensional-shape detection unit that projects pattern light to the object, takes an image of the object by the imaging unit, and detects the three-dimensional shape of the object.

3. The image input-and-output apparatus according to claim 2, wherein the recognizing unit recognizes, as the object information, written information on the object, detected by the three-dimensional-shape detection unit.

4. The image input-and-output apparatus according to claim 3, further comprising:

a light source that emits light;

a spatial modulation unit that performs spatial modulation of light emitted from the light source and outputs image signal light;

a projection optical system that projects the image signal light output from the spatial modulation unit in a projection direction; and

a reference-information acquisition unit that obtains reference information of the object to be photographed by the imaging unit,

wherein the projecting unit comprises;

a reference-information projecting unit that outputs image signal light based on the reference information obtained by the reference-information acquisition unit, through the spatial modulation unit, and projects the output image signal light onto the object or onto an image-projectable plane in the projection direction through the projection optical system.

5. The image input-and-output apparatus according to claim 4, further comprising:

a comparison-result-information acquisition unit that compares the reference information obtained by the reference-information acquisition unit and the written information recognized by the recognizing unit, and obtains the result of comparison as comparison result information,

wherein the projecting unit comprises;

a comparison-result-information projecting unit that outputs image signal light based on the comparison result information obtained by the comparison-result-information acquisition unit, through the spatial modulation unit and projects the output image signal light onto the object or onto an image-projectable plane in the projection direction through the projection optical system.

6. The image input-and-output apparatus according to claim 3, further comprising:

a light source that emits light;

a spatial modulation unit that performs spatial modulation of light emitted from the light source and outputs image signal light;

a projection optical system for projecting the image signal light output from the spatial modulation unit toward a projection plane;

a reference-information acquisition unit that obtains reference information related to the object to be photographed by the imaging unit; and

a comparison-result-information acquisition unit that compares the reference information obtained by the reference-information acquisition unit and the written information recognized by the recognizing unit, and obtains the comparison result as comparison result information,

wherein the projecting unit comprises;

a comparison-result-information projecting unit that outputs image signal light based on the comparison result information obtained by the comparison-result-information acquisition unit, through the spatial modulation unit, and projects the output image signal light onto the object or onto an image-projectable plane in the projection direction through the projection optical system.

7. The image input-and-output apparatus according to claim 6,

wherein the three-dimensional-shape detection unit comprises:

a three-dimensional-information generation unit that generates three-dimensional information of the object or an image-projectable plane in the projection direction, on the basis of the imaging data of the image signal light taken by the imaging unit when the image signal light has a predetermined pattern shape and the image signal light is projected onto the object or onto the image-projectable plane in the projection direction by the projection optical system,

wherein the image input-and-output apparatus comprises;

a plane-image correction unit that corrects the imaging data of the object obtained by the imaging unit to be equal to imaging data of the object positioned in a space containing the projection direction, taken by the imaging unit in a direction almost perpendicular to a surface of the object, in accordance with the three-dimensional information of the object or the image-projectable plane in the projection direction, generated by the three-dimensional-information generation unit, and

wherein the comparison-result-information acquisition unit obtains the comparison result information by using the written information recognized by the recognizing unit in accordance with the imaging data corrected by the plane-image correction unit.

8. The image input-and-output apparatus according to claim 7, further comprising:

a spatial modulation control unit that controls the spatial modulation unit in accordance with the three-dimensional information generated by the three-dimensional-information generation unit such that the reference-information projecting unit or the comparison-result-information projecting unit projects image signal light without deformation onto the object or onto an image-projectable plane in the projection direction.

9. The image input-and-output apparatus according to claim 6, wherein the reference information is information of a model of the object.

10. The image input-and-output apparatus according to claim 6, wherein the reference information is model information of a process of work formed of one or more stages;

the comparison-result-information acquisition unit obtains the comparison result information by comparing imaging data of the object in at least one stage of the work and the model information corresponding to the stage after the imaging data is taken by the imaging unit; and

the comparison-result-information projecting unit projects the comparison result information onto the object or the image-projectable plane in the projection direction through the comparison-result-information acquisition unit.

11. The image input-and-output apparatus according to claim 10, wherein the projecting unit comprises:

a reference-information projecting unit that projects the model information of each stage onto the object or an image-projectable plane in the projection direction, together with the comparison result information of the corresponding stage projected onto the object by the comparison-result-information projecting unit.

12. The image input-and-output apparatus according to claim 6, wherein the comparison result information is correction information of the object with respect to the reference information.

13. The image input-and-output apparatus according to claim 6, wherein the comparison result information is evaluation information of the object with respect to the reference information.

14. The image input-and-output apparatus according to claim 6, further comprising a movable unit capable of moving a projection section which includes the light source, the spatial modulation unit, and the projection optical system, and an imaging section which includes the imaging unit and which is integrally provided with the projection section, as the object is moved.

15. The image input-and-output apparatus according to claim 14, wherein the movable unit can move the projection section and the imaging section, such that the projection of the image signal light onto the object by the projecting unit and the imaging of the object by the imaging unit can be made in the direction opposite to the direction of the work performed on the object, in the top view.

16. The image input-and-output apparatus according to claim 14, wherein the movable unit can move the projection section and the imaging section such that the projecting unit projects the image signal light onto the object and the imaging unit takes an image of the object from the left in the front with respect to a person working on the object.

17. The image input-and-output apparatus according to claim 3, wherein the imaging unit obtains an image of the object, on which written information, including text and graphics, is written, to obtain first imaging data;

the recognizing unit comprises written-information recognizing unit that recognizes the written information from the first imaging data obtained by the imaging unit;

the image input-and-output apparatus further comprises;

a first storage unit that stores reference information which is set in advance; and

a first determining unit that determines whether the reference information stored in the first storage unit and the written information recognized by the written-information recognizing unit match; and

the projecting unit projects assessment result information indicating the result of the determination carried out by the first determining unit.

18. The image input-and-output apparatus according to claim 17, wherein the first determining unit determines whether or not the reference information stored in the first storage unit and the written information recognized by the written-information recognizing unit match;

and

when a match is determined by the first determining unit, the projecting unit projects, as the assessment result information, match information indicating that the written information and the reference information match;

when no match is determined by the first determining unit, the projecting unit projects, as the assessment result information, no-match information indicating that the written information and the reference information do not match.

19. The image input-and-output apparatus according to claim 18, wherein, if it is determined by the first determining unit that the reference information stored in the first storage unit and the written information recognized by the written-information recognizing unit do not match, the projecting unit projects the reference information corresponding to that written information as a part of the no-match information.

20. The image input-and-output apparatus according to claim 17, further comprising a second determining unit that determines whether or not it is possible to determine, using the first determining unit, whether or not the reference information stored in the first storage unit and the written information recognized by the written-information recognizing unit match,

and wherein, when it is determined by the second determining unit that the determination is impossible, the projecting unit projects determination-impossible information indicating that the determination is impossible.

21. The image input-and-output apparatus according to claim 20, wherein, if it is determined by the second determining unit that the determination is impossible, the projecting unit projects, as part of the determination-impossible information, the reference information corresponding to the written information.

22. The image input-and-output apparatus according to claim 18, further comprising a calculating unit that calculates an evaluation parameter indicating an evaluation related to the match information projected by the projecting unit,

and wherein the projection projects the evaluation parameter calculated by the calculating unit.

23. The image input-and-output apparatus according to claim 22, further comprising a counting unit that counts the number of match information items projected by the projecting unit,

and wherein the calculating unit calculates the evaluation parameter, which is related to the match information, based on the number of match information items counted by counting unit.

24. The image input-and-output apparatus according to claim 23, wherein the imaging unit acquires an image of the object when corresponding information is written which corresponds to the match information written on the object by the user based on the assessment result information projected by the projecting unit, and obtains the image as second imaging data;

the written-information recognizing unit recognizes the corresponding information from that second imaging data; and

the counting unit counts the number of corresponding information items recognized by the written-information recognizing unit as the number of match information items projected by the projecting unit.

25. The image input-and-output apparatus according to claim 17, further comprising a detection unit that detects a designated area designated by the user from the first imaging data obtained by the imaging unit,

and wherein the written-information recognizing unit recognize written information identified based on the designated area detected by the detection unit.

26. The image input-and-output apparatus according to claim 17, further comprising:

a second storage unit that stores information projected from the projecting unit in accordance with the assessment result of the first determining unit or the second determining unit; and

a third determining unit that determines whether or not a predetermined number of determinations by the first determining unit have been completed,

and wherein the projecting unit projects information stored in the second storage unit all together after it is determined by the third determining unit that the determinations by the first determining unit have been completed.

27. The image input-and-output apparatus according to claim 26, wherein the second storage unit also stores the evaluation parameter calculated by the calculating unit;

and

the projecting unit projects the information stored in the second storage unit all together when it is determined by the third determining unit that the determinations by the first determining unit have been completed and after the evaluation parameter calculated by the calculating unit has been stored in the second storage unit.

28. The image input-and-output apparatus according to claim 17, wherein the projecting unit projects the information projected from the projecting unit one by one in accordance with the assessment results, each time it is determined that one determination has been completed by the first determining unit or each time it is determined by the second determining unit that determination is impossible.

29. The image input-and-output apparatus according to claim 17, further comprising a projection-position control unit that controls the projecting unit such that the assessment result information projected by the projecting unit is projected on the object, close to the written information corresponding to that assessment result.

30. The image input-and-output apparatus according to claim 17, wherein the projecting unit projects pattern light on the object;

an image of the object is acquired by the imaging unit while the pattern light is being projected from the projecting unit onto the object to obtain the image as third imaging data; and

the image input-and-output apparatus comprises a three-dimensional information detecting unit that detects three-dimensional information related to the three-dimensional shape of the object based on the third imaging data obtained by the imaging unit.

31. The image input-and-output apparatus according to claim 30, further comprising a projection-form control unit that controls the form of the assessment result information projected onto the object by the projecting unit, based on the three-dimensional information detected by three-dimensional information detecting unit.

32. The image input-and-output apparatus according to claim 30, further comprising a planar-imaging-data generating unit that generates planar imaging data in which the surface of the object is developed onto a plane, from the first imaging data obtained by the imaging unit, based on the three-dimensional information detected by the three-dimensional information detecting unit,

and wherein the written-information recognizing unit recognizes the written information from the planar imaging data generated by the planar-imaging-data generating unit.

33. The image input-and-output apparatus according to claim 17, wherein the projecting unit comprises a plurality of semiconductor light-emitting elements for emitting light, a spatial modulator device for outputting image-signal light formed by subjecting the light output from the plurality of semiconductor light-emitting elements to spatial modulation, and a projection optical system for projecting the image-signal light output from the spatial modulator device towards a projection plane;

and

the plurality of semiconductor light-emitting elements is disposed in a staggered arrangement on a substrate supporting the plurality of semiconductor light-emitting elements.

34. The image input-and-output apparatus according to claim 33, wherein the plurality of semiconductor light-emitting elements is disposed such that the distance between one of the semiconductor light-emitting elements and any semiconductor light-emitting element around the one of the semiconductor light-emitting elements is defined so as to be equal to or less than the full-width at half-maximum of the illuminance distribution formed at the spatial modulator device by the light emitted from one of the semiconductor light-emitting elements.

35. The image input-and-output apparatus according to claim 33, wherein the plurality of semiconductor light-emitting elements is configured to all emit light of the same color.

36. The image input-and-output apparatus according to claim 35, wherein the plurality of semiconductor light-emitting elements is light-emitting diodes, and the emission colors emitted therefrom is all amber.

37. The image input-and-output apparatus according to claim 3, further comprising:

a related-information storage unit in which related information corresponding to the written information is stored in advance; and

a related-information extracting unit that extracts the related information from the t related-information storage unit in accordance with the written information recognized by the recognizing unit.

38. The image input-and-output apparatus according to claim 3, further comprising a projection-position setting unit that sets the projection position of the related information projected by the projecting unit so as to be close to the written information corresponding to that related information.

39. The image input-and-output apparatus according to claim 38, wherein the projection-position setting unit sets the projection position of an item of related information corresponding to each different item of written information so as to be close to the different item of written information corresponding to the item of related information.

40. The image input-and-output apparatus according to claim 3, further comprising a first specifying unit that sets the written information to be recognized by the recognizing unit, to all of the written information included in the imaging data,

wherein the projecting unit projects related information corresponding to the written information specified by the first specifying unit.

41. The image input-and-output apparatus according to claim 3, further comprising a second specifying unit that sets the written information to be recognized by the recognizing unit, to a part of the written information included in the imaging data,

wherein the projecting unit projects related information related to the written information specified by the second specifying unit.

42. The image input-and-output apparatus according to claim 41, further comprising a designated-area detecting unit that detects a designated area designated on the object,

wherein the second specifying unit specifies the written information to be recognized by the recognizing unit, based on the designated area detected by the designated-area detecting unit.

43. The image input-and-output apparatus according to claim 42, wherein the projecting unit projects an image which allows the designated area detected by the designated-area detecting unit to be distinguished from the other parts on the object.

44. The image input-and-output apparatus according to claim 43, wherein the projecting unit projects an area-designating image which allows the designated area detected by the designated-area detecting unit to be distinguished from the other parts on the object, towards that designated area.

45. The image input-and-output apparatus according to claim 3, further comprising a transformation-related-information setting unit that transforms the related information projected by the projecting unit according to the shape of the object, based on the three-dimensional information detected by the three-dimensional-shape detection unit.

46. The image input-and-output apparatus according to claim 45, further comprising a planar-imaging-data generating unit that generates planar imaging data which is formed by expanding onto a plane the three-dimensional shape formed by the surface of the object in the imaging data related to the object, based on the three-dimensional information detected by the three-dimensional-shape detection unit,

and wherein the recognizing unit recognizes the written information from the imaging data generated by the planar-imaging-data generating unit.

47. The image input-and-output apparatus according to claim 3, wherein the related information is any of dictionary information that includes meaning information, translation information, pronunciation information, and graphics information;

correction information related to a typographical error or a clerical error in the written information;

postal-number information; and

address information.

48. The image input-and-output apparatus according to claim 3, wherein the projecting unit comprises a plurality of semiconductor light-emitting elements for emitting light, a spatial modulator device for outputting image-signal light formed by subjecting the light emitted from the plurality of semiconductor light-emitting elements to spatial modulation, and a projecting optical system for projecting the image-signal light output from the spatial modulating device towards a projection plane;

and

the plurality of semiconductor light-emitting elements is disposed in a staggered arrangement on a substrate supporting the plurality of semiconductor light-emitting elements.

49. The image input-and-output apparatus according to claim 48, wherein the distance between one of the semiconductor light-emitting elements and any semiconductor light-emitting element around the one of the semiconductor light-emitting elements is defined so as to be equal to or less than the full-width at half-maximum of the luminance distribution formed at the spatial modulator device by the light emitted from one of the semiconductor light-emitting elements.

50. The image input-and-output apparatus according to claim 48, wherein the plurality of semiconductor light-emitting elements all emits light of the same color.

51. The image input-and-output apparatus according to claim 50, wherein the plurality of semiconductor light-emitting elements is light-emitting diodes;

and

the emission colors emitted therefrom are all amber.

52. The image input-and-output apparatus according to claim 2, wherein the recognizing unit recognizes, as the object information, three-dimensional information of the entire object, detected by the three-dimensional-shape detection unit.

53. The image input-and-output apparatus according to claim 52, further comprising:

a light-source that emits light;

a spatial modulating unit that outputs image-signal light formed by subjecting the light emitted from the light source to spatial modulation;

a three-dimensional-information generating unit that, when image-signal light having a predetermined pattern is projected by the projecting unit onto the object or onto an image-projectable plane in the projection direction, generates three-dimensional information of the object or the image-projectable plane in the projection direction in accordance with the object information recognized by the recognizing unit; and

a measurement unit that measures the object based on the three-dimensional information generated by the three-dimensional-information generating unit and designated information generated from imaging data obtained when the imaging unit takes an image of a designated condition applied to the object,

and wherein image-signal light corresponding to measurement information obtained from the measurement by the measurement unit is output by the spatial modulating unit; and

the output image-signal light is projected by the projecting unit onto the object or onto the image-projectable plane in the projection direction.

54. The image input-and-output apparatus according to claim 53, further comprising a designated-point inputting unit that inputs a designated point on the object,

and wherein the measurement unit carries out measurement of the object at a point, line segment, or area formed by at least one designated point which is input by the designated-point inputting unit.

55. The image input-and-output apparatus according to claim 54, wherein the imaging unit takes images of a pointing member for indicating the designated point on the object at different points in time, and

the designated-point inputting unit inputs the designated point based on the imaging data thereof.

56. The image input-and-output apparatus according to claim 53, wherein the measurement unit measures volume information of the object based on the three-dimensional information generated by the three-dimensional-information generating unit, and

the image-signal light projected by the projecting unit onto the object or onto the image-projectable plane in the projection direction is instruction information corresponding to the volume information.

57. The image input-and-output apparatus according to claim 53, further comprising a density-information acquisition unit that obtains density information of the object,

wherein:

the measurement unit measures weight-distribution information of the object based on the three-dimensional information generated by the three-dimensional-information generating unit and the density information obtained by the density-information acquisition unit; and

the image-signal light projected by the projecting unit onto the object or onto the image-projectable plane in the projection direction is instruction information corresponding to the weight-distribution information.

58. The image input-and-output apparatus according to claim 53, wherein the image-signal light projected by the projecting unit onto the object or onto the image-projectable plane in the projection direction is text information related to the measurement information.

59. The image input-and-output apparatus according to claim 58, wherein the image-signal light projected by the projecting unit onto the object or onto the image-projectable plane in the projection direction is graphical information related to the measurement information.

60. The image input-and-output apparatus according to claim 58, wherein the image-signal light projected by the projecting unit onto the object or onto the image-projectable plane in the projection direction is a work instruction line based on the measurement information.

61. The image input-and-output apparatus according to claim 53, wherein the image-signal light projected by the projecting unit onto the object or onto the image-projectable plane in the projection direction is a work instruction line for dividing the object into substantially equal volumes or substantially equal weights in accordance with the volume information or the weight distribution information of the object, measured by the measurement unit.

62. The image input-and-output apparatus according to claim 61, wherein the work instruction line is a cutting instruction line.

63. The image input-and-output apparatus according to claim 62, further comprising a cutting unit that cuts the object in accordance with the cutting instruction line.

64. The image input-and-output apparatus according to claim 53, further comprising a movable unit capable of moving a projection section which includes the light source, the spatial modulation unit, and the projecting unit;

an imaging section which includes the imaging unit and which is integrally provided with the projection section; and

the object in terms of their mutual relative positions.

65. The image input-and-output apparatus according to claim 53, wherein the projecting unit turns off the image signal light having the predetermined pattern shape, and then, projects the image signal light based on the measurement information.

66. The image input-and-output apparatus according to claim 53, wherein the imaging unit takes an image of the object or the image-projectable plane in the projection direction to which the projecting unit projects the image signal light based on the measurement information, and obtains the imaging data thereof.

67. The image input-and-output apparatus according to claim 52, further comprising:

a light-source that emits light;

a spatial modulating unit that outputs image-signal light formed by subjecting the light output from the light source to spatial modulation;

a three-dimensional-information generating unit that, when image-signal light having a predetermined pattern is projected by the projecting unit onto the object or onto an image-projectable plane in the projection direction, generates three-dimensional information of the object or the image-projectable plane in the projection direction based on the object information recognized by the recognizing unit; and

an instruction-information generating unit that generates instruction information for the object based on the three-dimensional information generated by the three-dimensional-information generating unit,

and wherein image-signal light based on the instruction information generated by the instruction-information generating unit is output by the spatial modulating unit; and

the output image-signal light is projected by the projecting unit onto the object or onto the image-projectable plane in the projection direction.

68. The image input-and-output apparatus according to claim 67, further comprising a spatial-light-modulation control unit that controls the spatial modulation unit in accordance with the three-dimensional information generated by the three-dimensional-information generating unit such that no-deformation image-signal light is projected by the projecting unit onto the object or onto the image-projectable plane in the projection direction.

69. The image input-and-output apparatus according to claim 67, wherein the image-signal light projected by the projecting unit onto the object or onto the image-projectable plane in the projection direction is text information related to the instruction information.

70. The image input-and-output apparatus according to claim 67, wherein the image-signal light projected by the projecting unit onto the object or onto the image-projectable plane in the projection direction is graphical information based on the instruction information.

71. The image input-and-output apparatus according to claim 67, wherein the image-signal light projected by the projecting unit onto the object or onto the image-projectable plane in the projection direction is a work-instruction line based on the instruction information.

72. The image input-and-output apparatus according to claim 67, wherein the image-signal light projected by the projecting unit onto the object or onto the image-projectable plane in the projection direction is work-instruction information for dividing the object into substantially equal volumes or substantially equal weights, the work-instruction information being generated as instruction information by the instruction-information generating unit.

73. The image input-and-output apparatus according to claim 72, wherein the work-instruction information is a cutting-instruction line.

74. The image input-and-output apparatus according to claim 73, further comprising a cutting unit that cuts the object along the cutting-instruction line.

75. The image input-and-output apparatus according to claim 67, further comprising a movable unit capable of moving a projection section which includes the light source, the spatial modulation unit, and the projecting unit;

an imaging section which includes the imaging unit and which is integrally provided with the projection section; and

the object in terms of their mutual relative positions.

76. The image input-and-output apparatus according to claim 67 wherein the projecting unit turns off the image signal light having the predetermined pattern shape, and then, projects the image signal light based on the instruction information.

77. The image input-and-output apparatus according to claim 67, wherein the imaging unit takes an image of the object or the image-projectable plane in the projection direction to which the projecting unit projects the image signal light based on the instruction information, and obtains the imaging data thereof.

78. The image input-and-output apparatus according to claim 52, further comprising:

a light source that emits light;

a space modulation unit that applies space modulation to the light emitted from the light source and outputting image signal light;

a related-information search unit that searches an external data accumulation unit communicably connected to the image input-and-output apparatus via a communication unit or a data accumulation unit built in the image input-and-output apparatus, for related information about the object based on the three-dimensional information detected by the three-dimensional-shape detection unit; and

a related-information output unit that outputs the related information found by the related-information search unit.

79. The image input-and-output apparatus according to claim 78, wherein the related-information search unit searches for the related information about the object based on information indicating the three-dimensional shape of the object from among the three-dimensional information detected by the three-dimensional-shape detection unit.

80. The image input-and-output apparatus according to claim 78, wherein the related-information search unit searches for the related information about the object based on information indicating the three-dimensional shape of the object from among the three-dimensional information detected by the three-dimensional-shape detection unit and the imaging data acquired by the imaging unit.

81. The image input-and-output apparatus according to claim 78, wherein, when imaging data is acquired by taking images of a plurality of objects at the same time by the imaging unit, the related-information search unit searches for related information of each of at least particular objects from among the plurality of objects whose images have been taken by the imaging unit.

82. The image input-and-output apparatus according to claim 78, further comprising an object-image extraction unit that extracts two-dimensional-image information of an object part having a solid shape separable from the background from among imaging data acquired by the imaging unit in accordance with the three-dimensional information detected by the three-dimensional-shape detection unit,

wherein the related-information search unit searches for the related information about the object in accordance with the two-dimensional-image information extracted by the object-image extraction unit.

83. The image input-and-output apparatus according to claim 78, further comprising an object-image extraction unit that extracts two-dimensional-image information of an object part having a solid shape separable from the background from among imaging data acquired by the imaging unit in accordance with the three-dimensional information detected by the three-dimensional-shape detection unit,

wherein the related-information search unit searches for related information about the object, having the solid shape, based on the three-dimensional information detected by the three-dimensional-shape detection unit and the two-dimensional-image information extracted by the object-image extraction unit.

84. The image input-and-output apparatus according to claim 83, wherein the related-information search unit searches for related information in accordance with information indicating the three-dimensional shape of the object from among the three-dimensional information detected by the three-dimensional-shape detection unit and the two-dimensional-image information extracted by the object-image extraction unit.

85. The image input-and-output apparatus according to claim 82, wherein, when imaging data is acquired by taking images of a plurality of objects at the same time by the imaging unit, the object-image extraction unit extracts two-dimensional-image information of the object part for each of at least particular objects from among the plurality of objects whose images have been captured by the imaging unit, and the related-information search unit searches for related information about each object in accordance with the two-dimensional-image information extracted by the object-image extraction unit.

86. The image input-and-output apparatus according to claim 78, wherein the related information output by the related-information output unit comprises at least one of the name and an attribute of the object.

87. The image input-and-output apparatus according to claim 78, wherein the related-information output unit outputs image signal light based on the related information found by the related-information search unit, through the space modulation unit, and projects the output image signal light through the projecting unit upon the object or an image-projectable plane in the projection direction.

88. The image input-and-output apparatus according to claim 78, further comprising:

a display-content search unit that searches for a display content via a network in accordance with the related information found by the related-information search unit; and

a display-content output unit that outputs information about the display content found by the display-content search unit.

89. The image input-and-output apparatus according to claim 88, wherein the display-content output unit outputs image signal light based on information about the display content found by the display-content search unit, through the space modulation unit, and projects the output image signal light through the projecting unit upon the object or an image-projectable plane in the projection direction.