Object oriented image processing and rendering in a multi-dimensional space

• US 9,652,887 B2
• Filed: 01/21/2015
• Issued: 05/16/2017
• Est. Priority Date: 01/22/2014
• Status: Active Grant

• Current Assignee: Hankookin Inc.
• Original Assignee: Hankookin Inc.
• Inventors: Chun, James Jiwen, Chun, Andrew Youngho, Chun, Angela Soyoung, Chun, Jennifer Miseong
• Primary Examiner(s): Nguyen, Phu K
• Application Number:
  US14/601,290
• Publication Number:
  US 20150206339A1
• Time in Patent Office: 846 Days
• Field of Search: 345419
• CPC Class Codes: G06T 15/205 Image-based rendering

Abstract

A computer implemented method and an image processing system (IPS) for constructing and rendering an object oriented image (OOI) of a target object in an original spatial position and orientation of the target object are provided. While recording an image of a target object, the IPS records extended image data (EID), for example, spatial coordinates of an image recording device (IMR), multi-dimensional position and orientation of a line of vision of the IMR and the target object, etc. The IPS determines the original spatial position and orientation of the target object from the EID, constructs the OOI of the target object by repositioning and reorienting the recorded image based on the original spatial position and orientation of the target object, and renders the constructed OOI on an image display device. For constructing a three-dimensional OOI, the IPS records multiple images and the EID of the target object at multiple orientations.

Claims

1. A computer implemented method for constructing and rendering an object oriented image of a target object in an original spatial position and an original orientation of said target object, said method employing an image processing system comprising at least one processor configured to execute computer program instructions for performing said method, said method comprising:

receiving an image of said target object being recorded, from an image recording device by said image processing system;

recording extended image data by said image processing system, in communication with said image recording device and one or more auxiliary devices, during recording of said image of said target object;

determining said original spatial position and said original orientation of said target object in a multi-dimensional space from said recorded extended image data by said image processing system;

constructing said object oriented image of said target object by said image processing system by repositioning and reorienting said received image of said target object being recorded, based on said determined original spatial position and said original orientation of said target object; and

rendering said constructed object oriented image of said target object in said determined original spatial position and said original orientation on an image display device by said image processing system.

2. The computer implemented method of claim 1, wherein said extended image data comprises time of said recording of said image of said target object, spatial coordinates of said image recording device, multi-dimensional position and orientation of a line of vision of said image recording device, at least one of magnification and minimization of said image of said target object, and multi-dimensional spatial coordinates and spatial orientation of said target object being recorded by said image recording device.

3. The computer implemented method of claim 2, wherein said line of vision of said image recording device is a vector comprising spatial coordinates of an intersection of an imaging plane of said target object with a focal axis of said image recording device, and multi-dimensional spatial coordinates of a direction of said focal axis of said image recording device.

4. The computer implemented method of claim 1, further comprising obtaining supplementary information comprising physical dimensions of said target object, a three-dimensional image of said target object, time dependent spatial motion of said target object, and relative spatial relations between said image recording device and said target object by said image processing system, in communication with said image recording device and said one or more auxiliary devices, using said recorded extended image data, and enhancing said recorded extended image data with said supplementary information by said image processing system.

5. The computer implemented method of claim 1, further comprising configuring access elements on one or more regions of said rendered object oriented image of said target object by said image processing system, wherein said access elements, upon activation, are configured to render said one or more regions of said rendered object oriented image of said target object in one or more viewing windows for enhanced access to said one or more regions of said rendered object oriented image of said target object.

6. The computer implemented method of claim 1, wherein said constructed object oriented image of said target object is rendered as a two-dimensional image by said image processing system.

7. A computer implemented method for constructing and rendering a three-dimensional object oriented image of a target object in an original spatial position and an original orientation of said target object, said method employing an image processing system comprising at least one processor configured to execute computer program instructions for performing said method, said method comprising:

receiving a plurality of images of said target object being recorded at a plurality of orientations, from an image recording device by said image processing system;

recording extended image data at each of said orientations of said image recording device by said image processing system, in communication with said image recording device and one or more auxiliary devices, during recording of said images of said target object;

determining said original spatial position and said original orientation of said target object in a multi-dimensional space from said recorded extended image data by said image processing system;

constructing said three-dimensional object oriented image of said target object by said image processing system by repositioning and reorienting said received images of said target object being recorded, based on said determined original spatial position and said original orientation of said target object; and

rendering said constructed three-dimensional object oriented image of said target object in said determined original spatial position and said original orientation on an image display device by said image processing system.

8. The computer implemented method of claim 7, wherein said extended image data comprises time of said recording of said images of said target object, two-dimensional image data of said target object, spatial coordinates of said image recording device at said each of said orientations, multi-dimensional position and orientation of a line of vision of said image recording device at said each of said orientations, at least one of magnification and minimization of each of said images of said target object, and multi-dimensional spatial coordinates and spatial orientation of said target object being recorded by said image recording device at said each of said orientations.

9. The computer implemented method of claim 8, wherein said line of vision of said image recording device is a vector comprising spatial coordinates of an intersection of an imaging plane of said target object with a focal axis of said image recording device, and multi-dimensional spatial coordinates of a direction of said focal axis of said image recording device.

10. The computer implemented method of claim 7, further comprising obtaining supplementary information comprising physical dimensions of said target object, a three-dimensional image of said target object, time dependent spatial motion of said target object, and relative spatial relations between said image recording device and said target object by said image processing system, in communication with said image recording device and said one or more auxiliary devices, using said recorded extended image data, and enhancing said recorded extended image data with said supplementary information by said image processing system.

11. The computer implemented method of claim 7, further comprising rendering said constructed three-dimensional object oriented image of said target object in said determined original spatial position and said original orientation by said image processing system using spatial coordinates of said image display device.

12. The computer implemented method of claim 7, further comprising reconstructing said constructed three-dimensional object oriented image of said target object in said determined original spatial position and said original orientation by said image processing system using one or more of a spatial relation and an orientation relation of said target object with respect to one or more surrounding objects proximal to said target object in said multi-dimensional space.

13. The computer implemented method of claim 12, further comprising integrating one or more supplementary three-dimensional images of said one or more surrounding objects with said reconstructed three-dimensional object oriented image of said target object by said image processing system for generating an integrated three-dimensional object oriented image of a three-dimensional target scene by said image processing system.

14. The computer implemented method of claim 7, further comprising generating and rendering views of said constructed three-dimensional object oriented image of said target object by said image processing system, wherein said views comprise an internal view and an external view.

15. The computer implemented method of claim 7, further comprising configuring access elements on one or more regions of said rendered three-dimensional object oriented image of said target object by said image processing system, wherein said access elements, upon activation, are configured to render said one or more regions of said rendered three-dimensional object oriented image of said target object in one or more viewing windows for enhanced access to said one or more regions of said rendered three-dimensional object oriented image of said target object.

16. An image processing system for constructing and rendering an object oriented image of a target object in an original spatial position and an original orientation of said target object, said image processing system comprising:

a non-transitory computer readable storage medium configured to store computer program instructions defined by modules of said image processing system;

at least one processor communicatively coupled to said non-transitory computer readable storage medium, said at least one processor configured to execute said defined computer program instructions;

a data reception module configured to receive an image of said target object being recorded, from an image recording device;

an extended image data processing module configured to record extended image data, in communication with said image recording device and one or more auxiliary devices, during said recording of said image of said target object;

a positioning and orientation module configured to determine said original spatial position and said original orientation of said target object in a multi-dimensional space from said recorded extended image data;

an image configuration module configured to construct said object oriented image of said target object by repositioning and reorienting said received image of said target object being recorded, based on said determined original spatial position and said original orientation of said target object; and

an image rendering module configured to render said constructed object oriented image of said target object in said determined original spatial position and said original orientation on an image display device.

17. The image processing system of claim 16, wherein said extended image data comprises time of said recording of said image of said target object, spatial coordinates of said image recording device, multi-dimensional position and orientation of a line of vision of said image recording device, at least one of magnification and minimization of said image of said target object, and multi-dimensional spatial coordinates and spatial orientation of said target object being recorded by said image recording device.

18. The image processing system of claim 17, wherein said line of vision of said image recording device is a vector comprising spatial coordinates of an intersection of an imaging plane of said target object with a focal axis of said image recording device, and multi-dimensional spatial coordinates of a direction of said focal axis of said image recording device.

19. The image processing system of claim 16, wherein said extended image data processing module, in communication with said image recording device and said one or more auxiliary devices, is further configured to obtain supplementary information comprising physical dimensions of said target object, a three-dimensional image of said target object, time dependent spatial motion of said target object, and relative spatial relations between said image recording device and said target object using said recorded extended image data, and wherein said extended image data processing module is further configured to enhance said recorded extended image data with said supplementary information.

20. The image processing system of claim 16, wherein said image configuration module is further configured to construct a three-dimensional object oriented image of said target object by repositioning and reorienting a plurality of images of said target object being recorded at a plurality of orientations, based on an original spatial position and an original orientation of said target object, wherein said positioning and orientation module is further configured to determine said original spatial position and said original orientation of said target object in said multi-dimensional space from said extended image data of said target object being recorded at each of said orientations, and wherein said image rendering module is further configured to render said constructed three-dimensional object oriented image of said target object in said determined original spatial position and said original orientation on said image display device.

21. The image processing system of claim 20, wherein said image configuration module is further configured to reconstruct said constructed three-dimensional object oriented image of said target object in said determined original spatial position and said original orientation using one or more of a spatial relation and an orientation relation of said target object with respect to one or more surrounding objects proximal to said target object in said multi-dimensional space.

22. The image processing system of claim 21, wherein said image configuration module is further configured to integrate one or more supplementary three-dimensional images of said one or more surrounding objects with said reconstructed three-dimensional object oriented image of said target object for generating an integrated three-dimensional object oriented image of a three-dimensional target scene.

23. The image processing system of claim 20, wherein said image rendering module is further configured to render said constructed three-dimensional object oriented image of said target object in said determined original spatial position and said original orientation using spatial coordinates of said image display device.

24. The image processing system of claim 20, wherein said image configuration module is further configured to generate views of said constructed three-dimensional object oriented image of said target object, wherein said views comprise an internal view and an external view.

25. The image processing system of claim 16, wherein said image configuration module is further configured to configure access elements on one or more regions of said rendered object oriented image of said target object, wherein said access elements, upon activation, are configured to render said one or more regions of said rendered object oriented image of said target object in one or more viewing windows for enhanced access to said one or more regions of said rendered object oriented image of said target object.

26. A computer program product comprising a non-transitory computer readable storage medium, said non-transitory computer readable storage medium storing computer program codes that comprise instructions executable by at least one processor, said computer program codes comprising:

a first computer program code for receiving an image of a target object being recorded, from an image recording device;

a second computer program code for recording extended image data during recording of said image of said target object, wherein said extended image data comprises time of said recording of said image of said target object, spatial coordinates of said image recording device, multi-dimensional position and orientation of a line of vision of said image recording device, at least one of magnification and minimization of said image of said target object, and multi-dimensional spatial coordinates and spatial orientation of said target object being recorded by said image recording device;

a third computer program code for determining an original spatial position and an original orientation of said target object in a multi-dimensional space from said recorded extended image data;

a fourth computer program code for constructing an object oriented image of said target object by repositioning and reorienting said received image of said target object being recorded, based on said determined original spatial position and said original orientation of said target object; and

a fifth computer program code for rendering said constructed object oriented image of said target object in said determined original spatial position and said original orientation on an image display device.

27. The computer program product of claim 26, further comprising:

a sixth computer program code for obtaining supplementary information comprising physical dimensions of said target object, a three-dimensional image of said target object, time dependent spatial motion of said target object, and relative spatial relations between said image recording device and said target object using said recorded extended image data; and

a seventh computer program code for enhancing said recorded extended image data with said supplementary information.

28. The computer program product of claim 26, further comprising an eighth computer program code for constructing a three-dimensional object oriented image of said target object by repositioning and reorienting a plurality of images of said target object being recorded at a plurality of orientations, based on an original spatial position and an original orientation of said target object determined from said extended image data of said target object being recorded at each of said orientations.

29. The computer program product of claim 28, further comprising a ninth computer program code for reconstructing said constructed three-dimensional object oriented image of said target object in said determined original spatial position and said original orientation using one or more of a spatial relation and an orientation relation of said target object with respect to one or more surrounding objects proximal to said target object in said multi-dimensional space.

30. The computer program product of claim 29, further comprising a tenth computer program code for integrating one or more supplementary three-dimensional images of said one or more surrounding objects with said reconstructed three-dimensional object oriented image of said target object for generating an integrated three-dimensional object oriented image of a three-dimensional target scene.