Method and apparatus for generating texture in a three-dimensional scene

• US 8,947,505 B2
• Filed: 03/30/2011
• Issued: 02/03/2015
• Est. Priority Date: 03/31/2010
• Status: Expired due to Fees

• Current Assignee: Omron Scientific Technologies Incorporated (Omron Corporation)
• Original Assignee: Omron Scientific Technologies Incorporated (Omron Corporation)
• Inventors: Tyagi, Ambrish, Drinkard, John
• Primary Examiner(s): Senfi, Behrooz
• Assistant Examiner(s): Jebari, Mohammed
• Application Number:
  US13/075,214
• Publication Number:
  US 20110242283A1
• Time in Patent Office: 1,406 Days
• US Class Current: 348/46
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

In one aspect of the teachings herein, a 3D imaging apparatus uses a “texture tool” or facilitates the use of such a tool by an operator, to add artificial texture to a scene being imaged, for 3D imaging of surfaces or background regions in the scene that otherwise lack sufficient texture for determining sufficiently dense 3D range data. In at least one embodiment, the 3D imaging apparatus provides for iterative or interactive imaging, such as by dynamically indicating that one or more regions of the scene require artificial texturing via the texture tool for accurate 3D imaging, and then indicating whether or not the addition of artificial texture cured the texture deficiency and/or whether any other regions within the image require additional texture.

Claims

1. A method of 3D imaging implemented in an apparatus that includes a stereoscopic or other type of multi-view imaging system, said method comprising:

capturing multi-view image data for a field of view of the multi-view imaging system;

evaluating the multi-view image data to identify any regions in the field of view lacking sufficient texture for determining corresponding 3D range data, which regions are regarded as low-texture regions;

in the case that one or more low-texture regions are identified, generating output data or signals to control a texture tool or to guide a user controlling the texture tool, to project synthetic texture into the field of view for the one or more low-texture regions;

capturing multi-view image data at least for the one or more low-texture regions while illuminated with the synthetic texture; and

determining whether sufficient texture has been provided for 3D ranging of the low-texture regions and, if so, updating the corresponding 3D range data for the low-texture regions;

wherein the user rather than the apparatus controls the texture tool, and wherein, in the case that one or more low-texture regions are identified, said generating output data or signals to guide the user controlling the texture tool comprises generating output data or signals causing a representation of a scene corresponding to the field of view to be displayed, and further causing the one or more identified low-texture regions to be indicated within said representation; and

further wherein generating the output data or signals to guide the user controlling the texture tool further comprises indicating to the user when sufficient texture has been obtained, for individual ones of said identified low-texture regions.

2. The method of claim 1, further comprising configuring monitoring boundaries in the field of view, using the 3D range data, including using the updated 3D range data corresponding to the low-texture regions.

3. The method of claim 2, further comprising verifying the configured monitoring boundaries to ensure that they are within allowable tolerances with respect to the 3D range data, including the updated 3D range data corresponding to the low-texture regions.

4. The method of claim 1, further comprising, if sufficient texture has not been provided for any low-texture region, generating further output data or signals to indicate that case to the user, or generating further output or data signals to again guide the user in controlling the texture tool, to project synthetic texture into the low-texture region or regions where sufficient texture has not been provided.

5. The method of claim 1, wherein generating the output data or signals to guide the user controlling the texture tool further comprises indicating to the user via indicators in said representation where within the field of view of the apparatus that synthetic texture is currently detected.

6. An apparatus that includes a stereoscopic or other type of multi-view imaging system and is configured to perform 3D imaging, said apparatus comprising one or more processing circuits configured to:

capture multi-view image data for a field of view of the multi-view imaging system;

evaluate the multi-view image data to identify any regions in the field of view lacking sufficient texture for determining corresponding 3D range data, which regions are regarded as low-texture regions;

in the case that one or more low-texture regions are identified, generate output data or signals to control a texture tool or to guide a user controlling the texture tool, for projecting synthetic texture into the field of view for the one or more low-texture regions;

capture multi-view image data at least for the one or more low-texture regions while illuminated with the synthetic texture; and

determine whether sufficient texture has been provided for 3D ranging of the low-texture regions and, if so, update the corresponding 3D range data for the low-texture regions;

wherein the apparatus includes at least one of;

a computer interface for transferring guidance information to an external computer, for guiding a user in projecting the synthetic texture where needed; and

a user interface for providing the guidance information.

7. The apparatus of claim 6, wherein said one or more processing circuits are configured to configure monitoring boundaries in the field of view, using the 3D range data, including using the updated 3D range data corresponding to the low-texture regions.

8. The apparatus of claim 7, wherein said one or more processing circuits are configured to verify the configured monitoring boundaries to ensure that they are within allowable tolerances with respect to the 3D range data, including the updated 3D range data corresponding to the low-texture regions.

9. The apparatus of claim 6, wherein, if sufficient texture has not been provided for any low-texture region, the apparatus is configured to generate further output data or signals to indicate that case to the user or to generate further output or data signals to again guide the user controlling the texture tool, for projecting synthetic texture into the low-texture region or regions where sufficient texture has not been provided.

10. The apparatus of claim 6, wherein the one or more processing circuits comprise at least one computer-based circuit that is configured according to its execution of stored computer program instructions.

11. The apparatus of claim 6, wherein the one or more processing circuits comprise a capture controller configured to control capturing of image data by the apparatus, a 3D image evaluator to evaluate the multi-view image data captured by the apparatus and to thereby identify the presence of any low-texture regions within the field of view, and a guidance data generator to generate the output data or signals, for guiding the user controlling the texture tool.

12. The apparatus of claim 6, wherein the apparatus comprises a machine guarding or 3D volume monitoring apparatus that includes a stereoscopic camera system for multi-view imaging of a volume defined by the field of view of the multi-view imaging system.

13. The apparatus of claim 6, wherein the apparatus is configured to determine whether the projection of synthetic texture did not provide sufficient texture for determining the corresponding 3D range data for any of the identified low-texture regions, and, if so, to generate further output data or signals to indicate that case to the user.

14. The apparatus of claim 6, wherein the apparatus is configured to determine whether the projection of synthetic texture did not provide sufficient texture for determining the corresponding 3D range data for any of the identified low-texture regions, and, if so, to generate further output data or signals to again guide the user in controlling the texture tool, to project synthetic texture into the low-texture region or regions where sufficient texture was not provided.

15. The apparatus of claim 6, wherein, in the case that one or more low-texture regions are identified, the apparatus is configured to generate the output data or signals to guide the user controlling the texture tool based on generating output data or signals causing a representation of the field of view to be displayed, and further causing the one or more identified low-texture regions to be indicated within said representation.

16. The apparatus of claim 15, wherein the apparatus is configured to indicate to the user via indicators in said representation where within the field of view of the apparatus that synthetic texture is currently detected.

17. An apparatus that includes a stereoscopic or other type of multi-view imaging system and is configured to perform 3D imaging, said apparatus comprising one or more processing circuits configured to:

capture multi-view image data for a field of view of the multi-view imaging system;

evaluate the multi-view image data to identify any regions in the field of view lacking sufficient texture for determining corresponding 3D range data, which regions are regarded as low-texture regions;

in the case that one or more low-texture regions are identified, generate output data or signals to control a texture tool or to guide a user controlling the texture tool, for projecting synthetic texture into the field of view for the one or more low-texture regions;

capture multi-view image data at least for the one or more low-texture regions while illuminated with the synthetic texture; and

determine whether sufficient texture has been provided for 3D ranging of the low-texture regions and, if so, update the corresponding 3D range data for the low-texture regions;

wherein the user rather than the apparatus controls the texture tool, and wherein, in the case that one or more low-texture regions are identified, the apparatus is configured to generate the output data or signals to guide the user controlling the texture tool based on generating output data or signals causing a representation of the field of view to be displayed, and further causing the one or more identified low-texture regions to be indicated within said representation; and

further wherein the apparatus is configured to indicate to the user when sufficient texture has been obtained, for individual ones of said identified low-texture regions.