Inspection processing system, inspection processing method, and inspection processing program

• US 11,094,120 B2
• Filed: 07/25/2018
• Issued: 08/17/2021
• Est. Priority Date: 07/27/2017
• Status: Active Grant

• Current Assignee: Obayashi Corporation
• Original Assignee: Obayashi Corporation
• Inventors: Ikeda, Yuichi, Horii, Tamaki, Goto, Kazuyuki, Takenaka, Hidefumi, Yamamoto, Takayuki
• Primary Examiner(s): Tswei, YuJang
• Application Number:
  US16/045,577
• Publication Number:
  US 20190035155A1
• Time in Patent Office: 1,119 Days
• Field of Search: None
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30132 Masonry; Concrete
• CPC Class Codes: G06T 2207/30136 Metal
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes
• CPC Class Codes: G06T 7/0004 Industrial image inspection

Abstract

An inspection processing system includes a control unit connected to a photographing unit and an output unit. The control unit includes circuitry. The inspection processing system identifies a current position, displays a virtual image of a three-dimensional model of a structure corresponding to the current position. The virtual image is superimposed on a structure image obtained from the photographing unit. The inspection processing system further obtains an evaluation result of an evaluation item of an inspection subject using the structure image on which the displayed virtual image is superimposed, and records the evaluation result in an inspection information memory in association with the position of the inspection subject in the three-dimensional model.

Claims

1. An inspection processing system comprising a user terminal, wherein the user terminal includes circuitry, the circuitry is configured to:

identify a current position at which the user terminal is currently positioned m a construction or building site that includes a building structure;

generate, in response to the user terminal currently being positioned at the current position, a virtual image of the building structure that corresponds to the current position, the generation of the virtual image being from data of a designed three-dimensional model of the building structure;

obtain, with the user terminal, an actual structure image of the building structure at the current position;

superimpose the virtual image on the actual structure image;

obtain a measurement value for an evaluation item of an inspection subject of the building structure by measuring the actual structure image in the superimposition of the virtual image on the actual structure image, the measurement value including at least one of a size of the inspection subject or a number of the inspection subject;

obtain an evaluation result of the evaluation item by comparing at least one of the obtained size of the inspection subject or number of the inspection subject with a design value for the evaluation item obtained from the three-dimensional model; and

record the evaluation result in an inspection information memory in association with the position of the inspection subject in the three-dimensional model.

2. The inspection processing system according to claim 1, wherein the circuitry is further configured to:

generate the virtual image to have a predetermined transmittance, and

display the virtual image as being superimposed on the actual structure image according to the predetermined transmittance.

3. The inspection processing system according to claim 1, wherein the circuitry is further configured to:

generate the virtual image according to a contour of the building structure that corresponds to the current position as indicated by the three-dimensional model.

4. The inspection processing system according to claim 1, wherein the circuitry is further configured to:

change a manner of displaying the virtual image based on the evaluation result.

5. The inspection processing system according to claim 4, wherein the circuitry is further configured to:

change a color scheme of the virtual image based on a fitting status between the virtual image and the actual structure image in the evaluation result.

6. The inspection processing system according to claim 1, wherein the circuitry is further configured to:

calculate the evaluation result of the actual structure image based on attribute information of the three-dimensional model.

7. The inspection processing system according to claim 1, wherein the evaluation result of the evaluation item indicates whether an inspection result of the inspection subject is one of pass or fail for the evaluation item.

8. A method for assisting inspection using an inspection processing system comprising:

identifying a current position at which a user terminal is currently positioned in a construction or building site that includes a building structure;

generating, in response to the user terminal currently being positioned at the current position, a virtual image of the building structure that corresponds to the current position, the generation of the virtual image being from data of a designed three-dimensional model of the building structure;

obtaining, with the user terminal, an actual structure image of the building structure at the current position;

superimposing the virtual image on the actual structure image;

obtaining a measurement value for an evaluation item of an inspection subject of the building structure by measuring the actual structure image in the superimposition of the virtual

image on the actual structure image, the measurement value including at least one of a size of the inspection subject or a number of the inspection subject;

obtaining an evaluation result of the evaluation item by comparing at least one of the obtained size of the inspection subject or number of the inspection subject with a design value for the evaluation item obtained from the three-dimensional model; and

recording the evaluation result in an inspection information memory in association with the position of the inspection subject in the three-dimensional model.

9. The method according to claim 8, the method further comprising:

generating the virtual image to have a predetermined transmittance, and

displaying the virtual image as being superimposed on the actual structure image according to the predetermined transmittance.

10. The method according to claim 8, the method further comprising:

generating the virtual image according to a contour of the building structure that corresponds to the current position as indicated by the three-dimensional model.

11. The method according to claim 8, the method further comprising:

changing a manner of displaying the virtual image based on the evaluation result.

12. The method according to claim 11, the method further comprising:

changing a color scheme of the virtual image based on a fitting status between the virtual image and the actual structure image in the evaluation result.

13. The method according to claim 8, the method further comprising:

calculating the evaluation result of the actual structure image based on attribute information of the three-dimensional model.

14. The method according to claim 8, wherein the evaluation result of the evaluation item indicates whether an inspection result of the inspection subject is one of pass or fail for the evaluation item.

15. A non-transitory computer-readable medium that

stores a program thereon, wherein the program, when executed by circuitry of a control unit of an inspection processing system connected to a photographing unit and an output unit, causes the circuitry to:

identify a current position at which a user terminal is currently positioned in a construction or building site that includes a building structure;

generate, in response to the user terminal currently being positioned at the current position, a virtual image of the building structure that corresponds to the current position, the generation of the virtual image being from data of a designed three-dimensional model of the building structure;

obtain, with the user terminal, an actual structure image of the building structure at the current position;

superimpose the virtual image on the actual structure image;

obtain a measurement value for an evaluation item of an inspection subject of the building structure by measuring the actual structure image in the superimposition of the virtual image on the actual structure image, the measurement value including at least one of a size of the inspection subject or a number of the inspection subject;

obtain an evaluation result of the evaluation item by comparing at least one of the obtained size of the inspection subject or number of the inspection subject with a design value for the evaluation item obtained from the three-dimensional model; and

record the evaluation result in an inspection information memory in association with the position of the inspection subject in the three-dimensional model.

16. The non-transitory computer-readable medium according to claim 15, wherein the program, when executed by the circuitry, further causes the circuitry to:

generate the virtual image to have a predetermined transmittance, and

display the virtual image as being superimposed on the actual structure image according to the predetermined transmittance.

17. The non-transitory computer-readable medium according to claim 15, wherein the program, when executed by the circuitry, further causes the circuitry to:

generate the virtual image according to a contour of the building structure that corresponds to the current position as indicated by the three-dimensional model.

18. The non-transitory computer-readable medium according to claim 15, wherein the program, when executed by the circuitry, further causes the circuitry to:

change a manner of displaying the virtual image based on the evaluation result.

19. The non-transitory computer-readable medium according to claim 18, wherein the program, when executed by the circuitry, further causes the circuitry to:

change a color scheme of the virtual image based on a fitting status between the virtual image and the actual structure image in the evaluation result.

20. The non-transitory computer-readable medium according to claim 15, wherein the program, when executed by the circuitry, further causes the circuitry to:

calculate the evaluation result of the actual structure image based on attribute information of the three-dimensional model.

21. The non-transitory computer-readable medium according to claim 15, wherein the evaluation result of the evaluation item indicates whether an inspection result of the inspection subject is one of pass or fail for the evaluation item.