Image processing apparatus and storage medium storing image processing program

• US 8,797,264 B2
• Filed: 09/19/2006
• Issued: 08/05/2014
• Est. Priority Date: 03/09/2006
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Ohta, Keizo
• Primary Examiner(s): McDowell, Jr., Maurice L
• Assistant Examiner(s): Ricks, Donna J
• Application Number:
  US11/522,997
• Publication Number:
  US 20070211027A1
• Time in Patent Office: 2,877 Days
• Field of Search: 345/619, 345/158
• US Class Current: 345/158
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G09G 5/373 for modifying the size of the graphic pattern

Abstract

Predetermined image processing is performed in accordance with an input operation performed by an input device having image pickup means for taking an image of one or a plurality of imaging targets. Target image data, which is obtained from one target image of the one imaging target or a plurality of target images of the plurality of imaging targets in the image taken by the image pickup means and which indicates a distance between the plurality of target images or a size of the one target image, is sequentially obtained. A display image is enlarged and reduced in accordance with a change in the target image data. Then, the display image processed in such a manner is displayed on a display device.

Claims

1. An image processing apparatus for performing predetermined image processing in accordance with an input operation performed by an input device including a drag button and an image pickup element for taking an image of a plurality of imaging targets, the image processing apparatus comprising:

target image data obtaining means for sequentially obtaining, from the plurality of target images of the plurality of imaging targets in the image taken by the image pickup element, target image data indicating a distance between the plurality of target images;

image processing means for performing enlargement and reduction of a display image of a user-operable object in a three-dimensional space displayed on a display device and controlled by a user in accordance with a change in the target image data;

display control means for displaying on the display device the display image of the enlarged and reduced object processed by the image processing means, wherein the display image of the object is translated and rotated on the display device in accordance with an operation performed by the user,

distance calculation means for, based on the target image data, sequentially calculating a distance between the image pickup element and the plurality of the imaging targets;

enlarging/reducing means for performing the enlargement and reduction of the display image in accordance with a change in the distance, by sequentially changing, in accordance with the change in the distance, a position of a virtual camera placed in a virtual space,

the distance calculation means for, when the drag button is depressed, calculating the distance between the image pickup element and the middle point between a first and a second imaging target, when the image pickup element is positioned diagonally with respect to the imaging targets, based on the target image data including the distance between the first and the second target images, a diameter diamL of the first target image, a diameter diamR of the second target image, and a predetermined diameter diamM of each of the plurality of imaging targets, including;

a) calculating a width, w1, which indicates the range for which the image pickup element takes an image of the plurality of imaging targets with respect to a setting position of a first imaging target, the width, w1, given by

w1=wi×

diamM/diamL, where,

wi is the width of the image taken by the image pickup element, diamM is the predetermined diameter of each of the plurality of imaging targets, and diamL is the diameter of the first imaging target;

b) calculating the distance, realDL, between the first imaging target and the image pickup element, the distance, realDL, given by

realDL=(w1/2)/{tan(θ

/2)}, where

θ

is a viewing angle of the image pickup element;

c) calculating a width, w2, which indicates the range for which the image pickup element takes an image of the plurality of imaging targets with respect to a setting position of a second imaging target, the width, w2, given by

w2=wi×

diamM/diamR, where,

diamR is the diameter of the second imaging target;

d) calculating the distance, realDR, between the second imaging target and the image pickup element, the distance, realDR, given by

realDR=(w2/2)/{tan(θ

/2)};

e) calculating an angle, δ

1, between a line connecting the first imaging target and the input device and a line connecting the first and second imaging targets, given by

cos δ

1=(realDL²×

m²−

realDR²)/(2×

realDL×

m), where

m is the distance between the first and the second imaging targets; and

f) calculating the distance between the image pickup element and the middle point between the first and the second imaging targets, realD, given by

realD=√

(realDL²+(m/2)²−

2×

realDL×

(m/2)×

cos δ

1); and

velocity calculating means for calculating a moving velocity velD of the input device based on successively-calculated values of said calculated distance realD.

2. The image processing apparatus according to claim 1, wherein

the image processing means enlarges the display image in accordance with an increase in the distance, and reduces the display image in accordance with a decrease in the distance.

3. The image processing apparatus according to claim 1, wherein

the image processing means reduces the display image in accordance with an increase in the distance, and enlarges the display image in accordance with a decrease in the distance.

4. The image processing apparatus according to claim 1, wherein

the image processing means performs enlargement and reduction of the display image in accordance with the change which occurs, during a predetermined time period, in the distance.

5. The image processing apparatus according to claim 4, wherein

the image processing means enlarges the display image in accordance with an increase in the distance, and reduces the display image in accordance with a decrease in the distance.

6. The image processing apparatus according to claim 4, wherein

the image processing means reduces the display image in accordance with an increase in the distance, and enlarges the display image in accordance with a decrease in the distance.

7. The image processing apparatus according to claim 4, wherein the input device includes at least one pressable operation key and outputs at least operation information corresponding to a state of the operation key being pressed, the apparatus further comprising:

operation information obtaining means for obtaining the operation information; and

storage locations for, when the operation information indicates that the operation key has just started being pressed, storing the distance calculated by the distance calculation means, wherein

the image processing means includes difference calculation means for obtaining the change in the distance, which occurs during the predetermined time period which is a time period during which the operation key is pressed, by sequentially calculating a difference between the distance stored in the storage locations and the distance which is calculated, while the operation information indicates that the operation key is currently pressed, by the distance calculation means, and

the image processing means performs the enlargement and reduction of the display image in accordance with the difference.

8. The image processing apparatus according to claim 7, wherein

the image processing means enlarges the display image in accordance with an increase in the distance, and reduces the display image in accordance with a decrease in the distance.

9. The image processing apparatus according to claim 7, wherein

the image processing means reduces the display image in accordance with an increase in the distance, and enlarges the display image in accordance with a decrease in the distance.

10. The image processing apparatus according to claim 1, further comprising

designated coordinates calculation means for, based on positions of the plurality of target images in the taken image, calculating designated coordinates associated with a display area of the display device, wherein

the image processing means moves the display image in accordance with a change in the designated coordinates.

11. The image processing apparatus according to claim 1, further comprising

tilt calculation means for, based on positions of the plurality of target images in the taken image, calculating a tilt of the input device, wherein

the image processing means rotates the display image in accordance with a change in the tilt.

12. The image processing apparatus according to claim 1, wherein

the image processing means performs enlargement and reduction of the display image by sequentially changing, in accordance with the change in the distance, a display size of a two-dimensional image.

13. The image processing apparatus according to claim 1,

wherein

the image processing means performs enlargement and reduction of the display image in accordance with the moving velocity.