Coordinate calculating apparatus and coordinate calculating program

• US 20070211026A1
• Filed: 04/18/2006
• Published: 09/13/2007
• Est. Priority Date: 03/09/2006
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Ohta, Keizo
• Granted Patent: US 7,834,848 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A coordinate calculating apparatus detects a coordinate point (detected coordinate point) of an area satisfying a predetermined condition in a captured image for each area. Next, a vector (assumed vector) indicating a relative positional relationship between two position coordinate points in the captured image is set. The coordinate calculating apparatus determines one of the position coordinate points detected by coordinate detecting means, as one of object position coordinate points, and determines the other object position coordinate point using the one position coordinate point and the vector.

Claims

1. A coordinate calculating apparatus for calculating two object position coordinate points from an image captured by an image capturing apparatus, the two object position coordinate points indicating positions of two predetermined objects whose images to be captured, the positions being on a plane corresponding to the captured image, the apparatus comprising:

coordinate detecting means of detecting a position coordinate point of an area satisfying a predetermined condition in the captured image for each area;

vector setting means of setting a vector indicating a relative positional relationship between two position coordinate points in the captured image; and

coordinate determining means of determining one of the L detected position coordinate points detected by the coordinate detecting means as one of the object position coordinate points, and determining the other object position coordinate point using the one of the detected position coordinate points and the vector.

2. The coordinate calculating apparatus according to claim 1, wherein the vector setting means includes:

direction setting means of setting a direction of the vector based on a tilt determined by a tilt determining means for a tilt of the image capturing apparatus; and

length setting means of setting a length of the vector to be a predetermined length.

3. The coordinate calculating apparatus according to claim 2, further comprising:

object position storing means of storing object position coordinate data indicating at least one of a set of the two object position coordinate points determined by the coordinate determining means, and a vector connecting the two object position coordinate points, every time determination is performed by the coordinate determining means,

wherein the length setting means sets the length of the vector to be a length connecting between the two object position coordinate points indicated by object position coordinate data most recently stored.

4. The coordinate calculating apparatus according to claim 2, wherein the direction setting means sets the direction of the vector based on a correspondence relationship between a tilt determined by the tilt determining means in a situation that the image capturing apparatus is at a predetermined tilt, and a direction connecting the two object position coordinate points detected from an image captured by the image capturing apparatus in the situation.

5. The coordinate calculating apparatus according to claim 1, wherein the tilt determining means includes an acceleration sensor which is fixedly provided to a housing used in common with the image capturing apparatus and detects accelerations in two axial directions perpendicular to an image capturing direction of the image capturing apparatus, and

the vector setting means calculates the vector based on the acceleration in the axial directions detected by the acceleration sensor.

6. The coordinate calculating apparatus according to claim 1, further comprising:

object position storing means of storing object position coordinate data indicating at least one of a set of the two object position coordinate points determined by the v coordinate determining means, and a vector connecting the two object position coordinate points, every time determination is performed by the coordinate determining means,

wherein the vector setting means calculates or obtains a vector connecting between the two object position coordinate points indicated by object position coordinate data most recently stored.

7. The coordinate calculating apparatus according to claim 1, wherein the coordinate determining means includes:

first determination means of determining which of a first object position coordinate point and a second object position coordinate point of the two object position coordinate points one of the position coordinate points detected by the coordinate detecting means is; and

second determination means of determining the second object position coordinate point by adding the vector to the first object position coordinate point when the first object position coordinate point is determined by the first determination means, and the first object position coordinate point by subtracting the vector from the second object position coordinate point when the second object position coordinate point is determined by the first determination means.

8. The coordinate calculating apparatus according to claim 7, further comprising:

object position storing means of storing object position coordinate data including the two object position coordinate points determined by the coordinate determining means, every time determination is performed in the coordinate determining means;

wherein the first determination means calculates a distance between each of the position coordinate points detected by the coordinate detecting means and each of a first object position coordinate point and a second object position coordinate point indicated by object position coordinate data most recently stored, and determines one of the position coordinate points detected by the coordinate detecting means, as a first object position coordinate point, when a distance between the one detected position coordinate point and the first object position coordinate point indicated by the object position coordinate data most recently stored is shortest, and one of the position coordinate points detected by the coordinate detecting means, as a second object position coordinate point, when a distance between the one detected position coordinate point and the second object position coordinate point indicated by the object position coordinate data most recently stored is shortest.

9. The coordinate calculating apparatus according to claim 1, wherein the coordinate determining means includes:

temporary coordinate calculating means of determining one of the position coordinate points detected by the coordinate detecting means, as a determined coordinate point, adding the vector to the determined coordinate point to obtain a first temporary coordinate point, and subtracting the vector from the determined coordinate point to obtain a second temporary coordinate point; and

determination executing means of determining the first temporary coordinate point and the determined coordinate point as the two object position coordinate points when only the first temporary coordinate point of the two temporary coordinate points is outside an area of a captured image, and the second temporary coordinate point and the determined coordinate point as the two object position coordinate points when only the second temporary coordinate point of the two temporary coordinate points is outside an area of a captured image.

10. The coordinate calculating apparatus according to claim 1, further comprising:

object position coordinate calculating means of, when the number of position coordinate points detected by the coordinate detecting means is two or more, calculating the two object position coordinate points using the position coordinate points,

wherein the coordinate determining means determines the two object position coordinate points when calculation of the two object position coordinate points by the object position coordinate calculating means is not possible.

11. A coordinate calculating apparatus for calculating two object position coordinate points from an image captured by an image capturing apparatus, the two object position coordinate points indicating positions of two predetermined objects whose images to be captured, the positions being on a plane corresponding to the captured image, the apparatus comprising:

coordinate detecting means of detecting a position coordinate point of an area satisfying a predetermined condition in the captured image for each area;

reference setting means of setting a reference with respect to at least one of a direction connecting two position coordinate points in the captured image and a length between the two position coordinate points, when a plurality of position coordinate points detected by the coordinate detecting means;

selection means of selecting a combination of two position coordinate points most close to the reference set by the reference setting means, from combinations of two position coordinate points among the plurality of position coordinate points;

determination means of determining whether or not two position coordinate points have been selected by the selection means;

output means of outputting the two position coordinate points selected by the selection means, as the two object position coordinate points, when a result of the determination by the determination means is positive;

vector setting means of setting a vector indicating a relative positional relationship between two position coordinate points in the captured image, when the result of the determination by the determination means is negative; and

coordinate determining means of determining one of the position coordinate points determined by the coordinate determining means as one of the object position coordinate points, and determining the other object position coordinate point using the one of the detected position coordinate points and the vector, when the result of the determination by the determination means is negative.

12. A computer readable storing medium storing a program which is executed by a computer of a coordinate calculating apparatus for calculating two object position coordinate points from an image captured by an image capturing apparatus, the two object position coordinate points indicating positions of two predetermined objects whose images to be captured, the positions being on a plane corresponding to the captured image, the program causes the computer to execute:

V′

a vector setting step of setting a vector indicating a relative positional relationship between two position coordinate points in the captured image; and

a coordinate determining step of determining one of position coordinate points in each area satisfying a predetermined condition in the captured image, as one of the object position coordinate points, and determining the other object position coordinate point using the one position coordinate point and the vector.

13. The storing medium according to claim 12, wherein the vector setting step includes:

a direction setting step of setting a direction of the vector based on a tilt determined by a tilt determining means for a tilt of the image capturing apparatus; and

a length setting step of setting a length of the vector to be a predetermined length.

14. The storing medium according to claim 13, wherein the program causes the computer to further execute:

an object position storing step of storing object position coordinate data indicating at least one of a set of the two object position coordinate points determined by the coordinate determining step, and a vector connecting the two object position coordinate points, every time determination is performed by the coordinate determining step,

wherein, in the length setting step, the computer sets the length of the vector to be a length connecting between the two object position coordinate points indicated by object position coordinate data most recently stored.

15. The storing medium according to claim 13, wherein, in the direction setting step, the computer sets the direction of the vector based on a correspondence relationship between a tilt determined by the tilt determining means in a situation that the image capturing apparatus is at a predetermined tilt, and a direction connecting the two object position coordinate points detected from an image captured by the image capturing apparatus in the situation.

16. The storing medium according to claim 13, wherein the tilt determining means includes an acceleration sensor which is fixedly provided to a housing used in common with the image capturing apparatus and detects accelerations in two axial directions perpendicular to an image capturing direction of the image capturing apparatus, and

in the vector setting step, the computer calculates the vector based on the acceleration in the axial directions detected by the acceleration sensor.

17. The storing medium according to claim 12, wherein the program causes the computer to further execute:

an object position storing step of storing object position coordinate data indicating at least one of a set of the two object position coordinate points determined by the coordinate determining step, and a vector connecting the two object position coordinate points, every time determination is performed by the coordinate determining step,

wherein, in the vector setting step, the computer calculates or obtains a vector connecting between the two object position coordinate points indicated by object position coordinate data most recently stored.

18. The storing medium according to claim 12, wherein the coordinate determining step includes:

a first determination step of determining which of a first object position coordinate point and a second object position coordinate point of the two object position coordinate points one of the detected position coordinate points; and

a second determination step of determining the second object position coordinate point by adding the vector to the first object position coordinate point when the first object position coordinate point is determined by the first determination step, and the first object position coordinate point by subtracting the vector from the second object position coordinate point when the second object position coordinate point is determined by the first determination step.

19. The storing medium according to claim 18, wherein the program causes the computer to further execute:

an object position storing step of storing object position coordinate data including the two object position coordinate points determined by the coordinate determining step, into a memory of the coordinate calculating apparatus, every time determination is performed in the coordinate determining step,

wherein the first determination step calculates a distance between each of the position coordinate points detected by the coordinate detecting means and each of a first object position coordinate point and a second object position coordinate point indicated by object position coordinate data most recently stored, and determines one of the detected position coordinate points, as a first object position coordinate point, when a distance between the one detected position coordinate point and the first object position coordinate point indicated by the object position coordinate data most recently stored is shortest, and one of the detected position coordinate points, as a second object position coordinate point, when a distance between the one detected position coordinate point and the second object position coordinate point indicated by the object position coordinate data most recently stored is shortest.

20. The storing medium according to claim 12, wherein the coordinate determining step includes:

a temporary coordinate calculating step of determining one of the detected position coordinate points, as a determined coordinate point, adding the vector to the determined coordinate point to obtain a first temporary coordinate point, and subtracting the vector from the determined coordinate point to obtain a second temporary coordinate point; and

a determination executing step of determining the first temporary coordinate point and the determined coordinate point as the two object position coordinate points when only the first temporary coordinate point of the two temporary coordinate points is outside an area of a captured image, and the second temporary coordinate point and the determined coordinate point as the two object position coordinate points when only the second temporary coordinate point of the two temporary coordinate points is outside an area of a captured image.

21. The storing medium according to claim 12, wherein the program causes the computer to further execute:

an object position coordinate calculating step of, when the number of position coordinate points detected by the coordinate detecting means is two or more, calculating the two object position coordinate points using the position coordinate points,

wherein the coordinate determining step is executed when calculation of the two object position coordinate points is not possible in the object position coordinate calculating step.

22. A storing medium storing a program which is executed by a computer of a coordinate calculating apparatus for calculating two object position coordinate points from an image captured by an image capturing apparatus, the two object position coordinate points indicating positions of two predetermined objects whose images to be captured, the positions being on a plane corresponding to the captured image, the program causes the computer to execute:

a reference setting step of setting a reference with respect to at least one of a direction connecting two position coordinate points in the captured image and a length between the two position coordinate points;

a selection step of selecting a combination of two position coordinate points most close to the reference set by the reference setting step, from combinations of two position coordinate points among position coordinate points in an area satisfying a predetermined condition in the captured area;

a determination step of determining whether or not two position coordinate points have been selected by the selection step;

an output step of outputting the two position coordinate points selected by the selection step, as the two object position coordinate points, when a result of the determination in the determination step is positive;

a vector setting step of setting a vector indicating a relative positional relationship between two position coordinate points in the captured image, when the result of the determination in the determination step is negative; and

a coordinate determining step of determining one of position coordinate points in an area satisfying a predetermined condition in the captured image, as one of the object position coordinate points, and determining the other object position coordinate point using the one position coordinate point and the vector, when the result of the determination by the determination means is negative.