Coordinate calculating apparatus and coordinate calculating program

• US 7,834,848 B2
• Filed: 04/18/2006
• Issued: 11/16/2010
• Est. Priority Date: 03/09/2006
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Ohta, Keizo
• Primary Examiner(s): Nguyen; Kevin M
• Application Number:
  US11/405,664
• Publication Number:
  US 20070211026A1
• Time in Patent Office: 1,673 Days
• Field of Search: 345156-158, 463 36- 39
• US Class Current: 345/157
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A coordinate calculating apparatus detects a coordinate point (detected coordinate point) of an area satisfying a predetermined condition in a captured image for each area. Next, a vector (assumed vector) indicating a relative positional relationship between two position coordinate points in the captured image is set. The coordinate calculating apparatus determines one of the position coordinate points detected by coordinate detecting means, as one of object position coordinate points, and determines the other object position coordinate point using the one position coordinate point and the vector.

Claims

1. A coordinate calculating apparatus for calculating two object position coordinate points from an image captured by an image capturing apparatus, the two object position coordinate points indicating positions of two predetermined objects whose images are captured, such positions being indicated on a plane corresponding to a plane of the captured image, the apparatus comprising:

coordinate detecting programmed logic circuitry configured to detect a position coordinate point for each area satisfying a predetermined condition in the captured image;

vector setting programmed logic circuitry configured to set a vector indicating a relative positional relationship between two position coordinate points in the captured image; and

coordinate determining programmed logic circuitry configured to determine one of the detected position coordinate points detected by the coordinate detecting programmed logic circuitry as one of the object position coordinate points, and determining the other object position coordinate point using the one of the detected position coordinate points and the vector.

2. The coordinate calculating apparatus according to claim 1, wherein the vector setting programmed logic circuitry includes:

direction setting programmed logic circuitry configured to set a direction of the vector based on a tilt determined by a tilt determining means for determining a tilt of the image capturing apparatus; and

length setting programmed logic circuitry configured to set a length of the vector to be a predetermined length.

3. The coordinate calculating apparatus according to claim 2, further comprising:

object position storing means of storing object position coordinate data indicating at least one of a set of the two object position coordinate points determined by the coordinate determining programmed logic circuitry, and a vector connecting the two object position coordinate points, every time a determination is performed by the coordinate determining programmed logic circuitry,

wherein the length setting programmed logic circuitry sets the length of the vector to be a length connecting between the two object position coordinate points indicated by object position coordinate data most recently stored.

4. The coordinate calculating apparatus according to claim 2, wherein the direction setting programmed logic circuitry sets the direction of the vector based on a correspondence relationship between a tilt determined by the tilt determining means in a situation that the image capturing, apparatus is at a predetermined tilt, and a direction connecting the two object position coordinate points detected from an image captured by the image capturing apparatus in the situation.

5. The coordinate calculating apparatus according to claim 1, wherein the tilt determining means includes an acceleration sensor which is fixedly provided to a housing used in common with the image capturing apparatus and detects accelerations in two axial directions perpendicular to an image capturing direction of the image capturing apparatus, and

the vector setting programmed logic circuitry calculates the vector based on the acceleration in the axial directions detected by the acceleration sensor.

6. The coordinate calculating apparatus according to claim 1, further comprising:

object position storing means of storing object position coordinate data indicating at least one of a set of the two object position coordinate points determined by the coordinate determining programmed logic circuitry, and a vector connecting the two object position coordinate points, every time a determination is performed by the coordinate determining means,

wherein the vector setting programmed logic circuitry calculates or obtains a vector connecting between the two object position coordinate points indicated by object position coordinate data most recently stored.

7. The coordinate calculating apparatus according to claim 1, wherein the coordinate determining programmed logic circuitry includes:

first determination means of determining which of a first object position coordinate point and a second object position coordinate point of the two object position coordinate points one of the position coordinate points detected by the coordinate detecting means is; and

second determination means of determining the second object position coordinate point by adding the vector to the first object position coordinate point when the first object position coordinate point is determined by the first determination means, and the first object position coordinate point by subtracting the vector from the second object position coordinate point when the second object position coordinate point is determined by the first determination means.

8. The coordinate calculating apparatus according to claim 7, further comprising:

object position storing means of storing object position coordinate data including the two object position coordinate points determined by the coordinate determining means, every time determination is performed by the coordinate determining programmed logic circuitry;

wherein the first determination means calculates a distance between each of the position coordinate points detected by the coordinate detecting programmed logic circuitry and each of a first object position coordinate point and a second object position coordinate point indicated by object position coordinate data most recently stored, and determines one of the position coordinate points detected by the coordinate detecting programmed logic circuitry, as a first object position coordinate point, when a distance between the one detected position coordinate point and the first object position coordinate point indicated by the object position coordinate data most recently stored is shortest, and one of the position coordinate points detected by the coordinate detecting programmed logic circuitry, as a second object position coordinate point, when a distance between the one detected position coordinate point and the second object position coordinate point indicated by the object position coordinate data most recently stored is shortest.

9. The coordinate calculating apparatus according to claim 1, wherein the coordinate determining programmed logic circuitry includes:

temporary coordinate calculating means of determining one of the position coordinate points detected by the coordinate detecting programmed logic circuitry, as a determined coordinate point, adding the vector to the determined coordinate point to obtain a first temporary coordinate point, and subtracting the vector from the determined coordinate point to obtain a second temporary coordinate point; and

determination executing means of determining the first temporary coordinate point and the determined coordinate point as the two object position coordinate points when only the first temporary coordinate point of the two temporary coordinate points is outside an area of a captured image, and the second temporary coordinate point and the determined coordinate point as the two object position coordinate points when only the second temporary coordinate point of the two temporary coordinate points is outside an area of a captured image.

10. The coordinate calculating apparatus according to claim 1, further comprising:

object position coordinate calculating means of, when the number of position coordinate points detected by the coordinate detecting programmed logic circuitry is two or more, calculating the two object position coordinate points using the position coordinate points,

wherein the coordinate determining programmed logic circuitry determines the two object position coordinate points when calculation of the two object position coordinate points by the object position coordinate calculating means is not possible.

11. A coordinate calculating apparatus for calculating two object position coordinate points from an image captured by an image capturing apparatus, the two object position coordinate points indicating positions of two predetermined objects whose images are captured, such positions being indicated on a plane corresponding to a plane of the captured image, the apparatus comprising:

coordinate detecting means of detecting a position coordinate point of an area satisfying a predetermined condition in the captured image for each area;

reference setting means of setting a reference with respect to at least one of a direction connecting two position coordinate points in the captured image and a length between the two position coordinate points, when a plurality of position coordinate points is detected by the coordinate detecting means;

selection means of selecting a combination of two position coordinate points most close to the reference set by the reference setting means, from combinations of two position coordinate points among the plurality of position coordinate points;

determination means of determining whether or not two position coordinate points have been selected by the selection means;

output means of outputting the two position coordinate points selected by the selection means, as the two object position coordinate points, when a result of the determination by the determination means is positive;

vector setting means of setting a vector indicating a relative positional relationship between two position coordinate points in the captured image, when the result of the determination by the determination means is negative; and

coordinate determining means of determining one of the position coordinate points determined by the coordinate determining means as one of the object position coordinate points, and determining the other object position coordinate point using the one of the detected position coordinate points and the vector, when the result of the determination by the determination means is negative.

12. A computer readable non-transitory tangible computer readable storing medium storing a program which is executed by a computer for computing two object position coordinate points in an image captured by an image capturing apparatus, the two object position coordinate points indicating positions of two predetermined objects whose images are captured, such positions being indicated on a plane corresponding to a plane of the captured image, the program causing the computer to perform operations comprising:

setting a vector indicating a relative positional relationship between two position coordinate points in the captured image; and

a coordinate determining step of selecting one object position coordinate point from position coordinate points corresponding to an area satisfying a predetermined condition in the captured image and determining the other object position coordinate point using the selected one position coordinate point and the vector.

13. The non-transitory tangible computer readable storing medium according to claim 12, wherein the operation of setting a vector indicating a relative positional relationship between two position coordinate points in the captured image includes:

setting a direction of the vector based on a tilt of the image capturing apparatus with respect to gravity; and

setting a length of the vector based on the two object position coordinate points.

14. The non-transitory tangible computer readable storing medium according to claim 13, wherein the program causes the computer to perform further operations comprising:

storing object position coordinate data indicating at least one of a set of the two object position coordinate points and storing data describing a vector connecting the two object position coordinate points,

and wherein the length of the vector is set based on distance between the two object position coordinate points indicated by object position coordinate data most recently stored.

15. The non-transitory tangible computer readable storing medium according to claim 13, wherein a direction of the vector is set based on a correspondence relationship between a determined tilt whenever the image capturing apparatus is at a predetermined tilt and a direction connecting the two object position coordinate points is determined from an image captured by the image capturing apparatus.

16. The non-transitory tangible computer readable storing medium according to claim 13, wherein the tilt determining means includes an acceleration sensor which is fixedly provided to a housing used in common with the image capturing apparatus and detects accelerations in two axial directions perpendicular to an image capturing direction of the image capturing apparatus, and

the computer calculates the vector based on the acceleration in the axial directions detected by the acceleration sensor.

17. The non-transitory tangible computer readable storing medium according to claim 12, wherein the program causes the computer to further execute:

an object position storing step of storing object position coordinate data indicating at least one of a set of the two object position coordinate points determined by the coordinate determining step, and a vector connecting the two object position coordinate points, every time determination is performed by the coordinate determining step,

wherein the computer calculates or obtains a vector connecting between the two object position coordinate points indicated by object position coordinate data most recently stored.

18. The non-transitory tangible computer readable storing medium according to claim 12, wherein the coordinate determining step includes:

a first determination step of determining which of a first object position coordinate point and a second object position coordinate point of two object position coordinate points are the selected one of the position coordinate points; and

a second determination step of determining the second object position coordinate point by adding the vector to the first object position coordinate point when the first object position coordinate point is determined by the first determination step, and determining the first object position coordinate point by subtracting the vector from the second object position coordinate point when the second object position coordinate point is determined by the first determination step.

19. The non-transitory tangible computer readable storing medium according to claim 18, wherein the program causes the computer to further execute:

an object position storing step of storing object position coordinate data including the two object position coordinate points determined by the coordinate determining step, into a memory of the coordinate calculating apparatus, every time determination is performed in the coordinate determining step,

wherein the first determination step calculates a distance between each of the position coordinate points and each of a first object position coordinate point and a second object position coordinate point indicated by object position coordinate data most recently stored, and determines one of the detected position coordinate points, as a first object position coordinate point, when a distance between the one detected position coordinate point and the first object position coordinate point indicated by the object position coordinate data most recently stored is shortest, and one of the detected position coordinate points, as a second object position coordinate point, when a distance between the one detected position coordinate point and the second object position coordinate point indicated by the object position coordinate data most recently stored is shortest.

20. The non-transitory tangible computer readable storing medium according to claim 12, wherein the coordinate determining step includes:

a temporary coordinate calculating step of determining one of the detected position coordinate points, as a determined coordinate point, adding the vector to the determined coordinate point to obtain a first temporary coordinate point, and subtracting the vector from the determined coordinate point to obtain a second temporary coordinate point; and

a determination executing step of determining the first temporary coordinate point and the determined coordinate point as the two object position coordinate points when only the first temporary coordinate point of the two temporary coordinate points is outside an area of a captured image, and the second temporary coordinate point and the determined coordinate point as the two object position coordinate points when only the second temporary coordinate point of the two temporary coordinate points is outside an area of a captured image.

21. The non-transitory tangible computer readable storing medium according to claim 12, wherein the program causes the computer to further execute:

an object position coordinate calculating step of, when the number of position coordinate points detected by the coordinate detecting means is two or more, calculating the two object position coordinate points using the position coordinate points,

wherein the coordinate determining step is executed when calculation of the two object position coordinate points is not possible in the object position coordinate calculating step.

22. A non-transitory tangible computer readable storing medium storing a program which is executed by a computer computing two object position coordinate points from an image captured by an image capturing apparatus, the two object position coordinate points indicating positions of two predetermined objects whose images are captured, such positions being indicated on a plane corresponding to a plane of an image sensor on which the image is captured, the program causing the computer to perform operations comprising:

setting a reference with respect to at least one of a direction connecting two position coordinate points in the captured image and a length between the two position coordinate points;

identifying position coordinate points corresponding to one or more areas within the captured area which satisfy a predetermined condition and selecting a combination of two position coordinate points that are closest to the set reference, a selection being made from combinations of two position coordinate points among position coordinate points corresponding to one or more areas within the captured area which satisfy a predetermined condition;

determining whether a combination of two position coordinate points that are closest to the reference set can be identified;

outputting the two selected position coordinate points if it is determined that said combination of two position coordinate points can be identified; and

if it is determined that said combination of two position coordinate points can not be identified, setting a vector indicating a relative positional relationship between two position coordinate points in the captured image, and

then selecting one of said position coordinate points from at least one of said areas satisfying said predetermined condition in the captured image and designating it as one of the object position coordinate points, and then determining the other object position coordinate point using the one position coordinate point and the set vector.