Apparatus and method to reduce an amount of coordinate data representing an object taken by an imaging device in a three dimensional space

• US 10,341,683 B1
• Filed: 11/07/2018
• Issued: 07/02/2019
• Est. Priority Date: 12/26/2017
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Kubota, Tomonori, Murata, Yasuyuki
• Primary Examiner(s): Park, Edward
• Application Number:
  US16/182,663
• Publication Number:
  US 20190200041A1
• Time in Patent Office: 237 Days
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/564 from contours
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding

Abstract

An apparatus obtains a second straight-line by mapping a first straight-line passing a projection center of a target-image taken by a first imaging-device in a three-dimensional space and a point representing an object in a projection-plane of the first imaging-device, onto each of reference-images respectively taken by second imaging-devices, and generates a line-segment representing an existing-range of the object on the second straight-line for each reference-image. The apparatus stores the coordinate-value of the second-endpoint and a difference between coordinate-values of the first-endpoint and a second-endpoint of the line-segment in a memory. The apparatus restores the coordinate-value of the first-endpoint from the coordinate-value of the second-endpoint and the difference stored in the memory, maps the coordinate-values of the endpoints, onto a depth-direction line of the projection plane of the first imaging-device, and determines overlap of the line-segments on the depth-direction line, based on the mapped coordinate-values.

Claims

1. An information processing apparatus comprising:

a memory; and

a processor coupled to the memory and configured to;

obtain a second straight line by mapping a first straight line that passes a projection center of a target image taken by a first imaging device in a three dimensional space and a point representing an object in a projection plane of the first imaging device, onto each of a plurality of reference images respectively taken by a plurality of second imaging devices, and generate a reference line-segment representing an existing range of the object on the second straight line for each of the plurality of reference images;

transform, for each of the plurality of reference line-segments respectively generated on the plurality of reference images, a coordinate value of a first endpoint of the reference line-segment into a difference between a coordinate value of the first end point and a coordinate value of a second endpoint of the reference line-segment;

store the coordinate value of the second endpoint and the difference in the memory;

restore the coordinate value of the first endpoint from the coordinate value of the second endpoint and the difference stored in the memory; and

map the coordinate value of the second endpoint and the restored coordinate value of the first endpoint, onto a depth-direction line that is perpendicular to the projection plane of the first imaging device, and determine overlap of a plurality of line-segments on the depth-direction line whose endpoints are mapped from each of the plurality of reference line-segments on the plurality of reference images.

2. The information processing apparatus of claim 1, wherein:

the processor obtains an equation of the second straight line, which is described based on a horizontal coordinate and a vertical coordinate of each of the plurality of reference images, by using mapping information for mapping a point in the three dimensional space onto each of the plurality of reference images;

the coordinate value of the second endpoint and the difference stored in the memory are each described as a first-coordinate value based on a first coordinate that is one of the horizontal coordinate and the vertical coordinate;

the processor;

restores a first-coordinate value of the first endpoint, from a first-coordinate value of the second endpoint stored in the memory and a difference between a first-coordinate value of the first end point and the first-coordinate value of the second endpoint,

obtains, using the equation of the second straight line, a second-coordinate value of the second endpoint from the first-coordinate value of the second endpoint, the second coordinate value being a value described based on a second coordinate that is one of the horizontal coordinate and the vertical coordinate different from the first coordinate,

obtains, using the equation of the second straight line, a second-coordinate value of the first endpoint, from the first-coordinate value of the first endpoint, and

maps the first-coordinate value and the second-coordinate value of the first end point, and the first-coordinate value and the second-coordinate value of the second end point, onto the depth-direction line of the projection plane of the first imaging device.

3. The information processing apparatus of claim 1, wherein:

a difference between the first-coordinate values of the first and second endpoints is larger than a difference between the second-coordinate values of the first and second endpoints; and

the first-coordinate value of the second endpoint and the difference between the first-coordinate values of the first and second endpoints are each described as an integer variable.

4. The information processing apparatus of claim 3, wherein:

the processor determines a bit width of the integer variable for representing the first-coordinate value of the second endpoint, based on the first-coordinate value of the second endpoint; and

the processor determines a bit width of the integer variable for representing a difference between the first-coordinate values of the first and second endpoints, based on a value of the difference between the first-coordinate values of the first and second endpoints.

5. The information processing apparatus of claim 1, wherein:

the processor generates a first line-segment representing the existing range of the object on the second straight line, and generates a plurality of second line-segments representing existing ranges of one or more objects including the object;

the processor transforms a coordinate value of a first endpoint of each of the plurality of second line-segments into a difference between a coordinate value of the first endpoint and one of a coordinate value of a second endpoint of the second line-segment, a coordinate value of an endpoint of another second line-segment, and a coordinate value of one endpoint of the first line-segment;

the processor further stores the differences generated from coordinate values of endpoints of the plurality of second line-segments;

the processor reconstructs the coordinate values of the endpoints of the plurality of second line-segments from the differences generated from the coordinate values of the endpoints of the plurality of second line segments, and maps the reconstructed coordinate values of the endpoints of the plurality of second line-segments onto the depth-direction line of the projection plane of the first imaging device; and

the processor determines overlap of the plurality of second line-segments generated from the plurality of reference images on the depth-direction line of the projection plane of the first imaging device.

6. A non-transitory, computer-readable recording medium having stored therein a program for causing a computer to execute a process comprising:

obtaining a second straight line by mapping a first straight line that passes a projection center of a target image taken by a first imaging device in a three dimensional space and a point representing an object in a projection plane of the first imaging device, onto each of a plurality of reference images respectively taken by a plurality of second imaging devices, and generating a reference line-segment representing an existing range of the object on the second straight line for each of the plurality of reference images;

transforming, for each of the plurality of reference line-segments respectively generated on the plurality of reference images, a coordinate value of a first endpoint of the reference line-segment into a difference between a coordinate value of the first end point and a coordinate value of a second endpoint of the reference line-segment;

storing the coordinate value of the second endpoint and the difference in a memory;

restoring the coordinate value of the first endpoint from the coordinate value of the second endpoint and the difference stored in the memory; and

mapping the coordinate value of the second endpoint and the restored coordinate value of the first endpoint, onto a depth-direction line that is perpendicular to the projection plane of the first imaging device, and determining overlap of a plurality of line-segments on the depth-direction line whose endpoints are mapped from each of the plurality of reference line-segments on the plurality of reference images.

7. The non-transitory, computer-readable recording medium of claim 6, the process further comprising:

obtaining an equation of the second straight line, which is described based on a horizontal coordinate and a vertical coordinate of each of the plurality of reference images, by using mapping information for mapping a point in the three dimensional space onto each of the plurality of reference images, wherein the coordinate value of the second endpoint and the difference stored in the memory are each described as a first-coordinate value based on a first coordinate that is one of the horizontal coordinate and the vertical coordinate;

restoring a first-coordinate value of the first endpoint, from a first-coordinate value of the second endpoint stored in the memory and a difference between a first-coordinate value of the first end point and the first-coordinate value of the second endpoint;

obtaining, by using the equation of the second straight line, a second-coordinate value of the second endpoint from the first-coordinate value of the second endpoint, the second coordinate value being a value described based on a second coordinate that is one of the horizontal coordinate and the vertical coordinate different from the first coordinate;

obtaining, by using the equation of the second straight line, a second-coordinate value of the first endpoint, from the first-coordinate value of the first endpoint; and

mapping the first-coordinate value and the second-coordinate value of the first end point, and the first-coordinate value and the second-coordinate value of the second end point, onto the depth-direction line of the projection plane of the first imaging device.

8. The non-transitory, computer-readable recording medium of claim 6, wherein:

a difference between the first-coordinate values of the first and second endpoints is larger than a difference between the second-coordinate values of the first and second endpoints; and

the first-coordinate value of the second endpoint and the difference between the first-coordinate values of the first and second endpoints are each described as an integer variable.

9. The non-transitory, computer-readable recording medium of claim 8, the process further comprising:

determining a bit width of the integer variable for representing the first-coordinate value of the second endpoint, based on the first-coordinate value of the second endpoint; and

determining a bit width of the integer variable for representing a difference between the first-coordinate values of the first and second endpoints, based on a value of the difference between the first-coordinate values of the first and second endpoints.

10. The non-transitory, computer-readable recording medium of claim 6, the process further comprising:

generating a first line-segment representing the existing range of the object on the second straight line, and generating a plurality of second line-segments representing existing ranges of one or more objects including the object;

transforming a coordinate value of a first endpoint of each of the plurality of second line-segments into a difference between a coordinate value of the first endpoint and one of a coordinate value of a second endpoint of the second line-segment, a coordinate value of an endpoint of another second line-segment, and a coordinate value of one endpoint of the first line-segment;

storing the differences generated from coordinate values of endpoints of the plurality of second line-segments;

reconstructing the coordinate values of the endpoints of the plurality of second line-segments from the differences generated from the coordinate values of the endpoints of the plurality of second line segments, and maps the reconstructed coordinate values of the endpoints of the plurality of second line-segments onto the depth-direction line of the projection plane of the first imaging device; and

determining overlap of the plurality of second line-segments generated from the plurality of reference images on the depth-direction line of the projection plane of the first imaging device.

11. A method comprising:

obtaining a second straight line by mapping a first straight line that passes a projection center of a target image taken by a first imaging device in a three dimensional space and a point representing an object in a projection plane of the first imaging device, onto each of a plurality of reference images respectively taken by a plurality of second imaging devices, and generating a reference line-segment representing an existing range of the object on the second straight line for each of the plurality of reference images;

transforming, for each of the plurality of reference line-segments respectively generated on the plurality of reference images, a coordinate value of a first endpoint of the reference line-segment into a difference between a coordinate value of the first end point and a coordinate value of a second endpoint of the reference line-segment;

storing the coordinate value of the second endpoint and the difference in a memory;

restoring the coordinate value of the first endpoint from the coordinate value of the second endpoint and the difference stored in the memory; and

mapping the coordinate value of the second endpoint and the restored coordinate value of the first endpoint, onto a depth-direction line that is perpendicular to the projection plane of the first imaging device, and determining overlap of a plurality of line-segments on the depth-direction line whose endpoints are mapped from each of the plurality of reference line-segments on the plurality of reference images.

12. The method of claim 11, further comprising:

obtaining an equation of the second straight line, which is described based on a horizontal coordinate and a vertical coordinate of each of the plurality of reference images, by using mapping information for mapping a point in the three dimensional space onto each of the plurality of reference images, wherein the coordinate value of the second endpoint and the difference stored in the memory are each described as a first-coordinate value based on a first coordinate that is one of the horizontal coordinate and the vertical coordinate;

restoring a first-coordinate value of the first endpoint, from a first-coordinate value of the second endpoint stored in the memory and a difference between a first-coordinate value of the first end point and the first-coordinate value of the second endpoint;

obtaining, by using the equation of the second straight line, a second-coordinate value of the second endpoint from the first-coordinate value of the second endpoint, the second coordinate value being a value described based on a second coordinate that is one of the horizontal coordinate and the vertical coordinate different from the first coordinate;

obtaining, by using the equation of the second straight line, a second-coordinate value of the first endpoint, from the first-coordinate value of the first endpoint; and

mapping the first-coordinate value and the second-coordinate value of the first end point, and the first-coordinate value and the second-coordinate value of the second end point, onto the depth-direction line of the projection plane of the first imaging device.

13. The method of claim 11, wherein:

a difference between the first-coordinate values of the first and second endpoints is larger than a difference between the second-coordinate values of the first and second endpoints; and

the first-coordinate value of the second endpoint and the difference between the first-coordinate values of the first and second endpoints are each described as an integer variable.

14. The method of claim 13, further comprising:

determining a bit width of the integer variable for representing the first-coordinate value of the second endpoint, based on the first-coordinate value of the second endpoint; and

determining a bit width of the integer variable for representing a difference between the first-coordinate values of the first and second endpoints, based on a value of the difference between the first-coordinate values of the first and second endpoints.

15. The method of claim 11, further comprising:

generating a first line-segment representing the existing range of the object on the second straight line, and generating a plurality of second line-segments representing existing ranges of one or more objects including the object;

transforming a coordinate value of a first endpoint of each of the plurality of second line-segments into a difference between a coordinate value of the first endpoint and one of a coordinate value of a second endpoint of the second line-segment, a coordinate value of an endpoint of another second line-segment, and a coordinate value of one endpoint of the first line-segment;

storing the differences generated from coordinate values of endpoints of the plurality of second line-segments;

reconstructing the coordinate values of the endpoints of the plurality of second line-segments from the differences generated from the coordinate values of the endpoints of the plurality of second line segments, and maps the reconstructed coordinate values of the endpoints of the plurality of second line-segments onto the depth-direction line of the projection plane of the first imaging device; and

determining overlap of the plurality of second line-segments generated from the plurality of reference images on the depth-direction line of the projection plane of the first imaging device.