Image processing apparatus and storage medium storing image processing program

• US 20070211027A1
• Filed: 09/19/2006
• Published: 09/13/2007
• Est. Priority Date: 03/09/2006
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Ohta, Keizo
• Granted Patent: US 8,797,264 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G09G 5/373 for modifying the size of the graphic pattern

Abstract

Predetermined image processing is performed in accordance with an input operation performed by an input device having image pickup means for taking an image of one or a plurality of imaging targets. Target image data, which is obtained from one target image of the one imaging target or a plurality of target images of the plurality of imaging targets in the image taken by the image pickup means and which indicates a distance between the plurality of target images or a size of the one target image, is sequentially obtained. A display image is enlarged and reduced in accordance with a change in the target image data. Then, the display image processed in such a manner is displayed on a display device.

Claims

1. An image processing apparatus for performing predetermined image processing in accordance with an input operation performed by an input device including image pickup means for taking an image of one or a plurality of imaging targets, the image processing apparatus comprising:

target image data obtaining means for sequentially obtaining, from one target image of the one imaging target or a plurality of target images of the plurality of imaging targets in the image taken by the image pickup means, target image data indicating a distance between the plurality of target images or a size of the one target image;

image processing means for performing at least either one of enlargement and reduction of a display image in accordance with a change in the target image data; and

display control means for displaying on a display device the display image processed by the image processing means.

2. The image processing apparatus according to claim 1, wherein

the image processing means includes:

distance calculation means for, based on the target image data, sequentially calculating a distance between the image pickup means and the one or the plurality of the imaging targets; and

enlarging/reducing means for performing at least either one of enlargement and reduction of the display image in accordance with a change in the distance.

3. The image processing apparatus according to claim 2, wherein the image processing means enlarges the display image in accordance with an increase in the distance, and reduces the display image in accordance with a decrease in the distance.

4. The image processing apparatus according to claim 2, wherein the image processing means reduces the display image in accordance with an increase in the distance, and enlarges the display image in accordance with a decrease in the distance.

5. The image processing apparatus according to claim 2, wherein the image processing means performs at least either one of enlargement and reduction of the display image in accordance with the change which occurs, during a predetermined time period, in the distance.

6. The image processing apparatus according to claim 5, wherein the image processing means enlarges the display image in accordance with an increase in the distance, and reduces the display image in accordance with a decrease in the distance.

7. The image processing apparatus according to claim 5, wherein the image processing means reduces the display image in accordance with an increase in the distance, and enlarges the display image in accordance with a decrease in the distance.

8. The image processing apparatus according to claim 5, wherein the input device includes at least one pressable operation key and outputs at least operation information corresponding to a state of the operation key being pressed, the apparatus further comprising:

operation information obtaining means for obtaining the operation information; and

storage means for, when the operation information indicates that the operation key has just started being pressed, storing the distance calculated by the distance calculation means, wherein

the image processing means includes difference calculation means for obtaining the change in the distance, which occurs during the predetermined time period which is a time period during which the operation key is pressed, by sequentially calculating a difference between the distance stored in the storage means and the distance which is calculated, while the operation information indicates that the operation key is currently pressed, by the distance calculation means, and

the image processing means performs at least either one of enlargement and reduction of the display image in accordance with the difference.

9. The image processing apparatus according to claim 8, wherein the image processing means enlarges the display image in accordance with an increase in the distance, and reduces the display image in accordance with a decrease in the distance.

10. The image processing apparatus according to claim 8, wherein the image processing means reduces the display image in accordance with an increase in the distance, and enlarges the display image in accordance with a decrease in the distance.

11. The image processing apparatus according to claim 2, further comprising designated coordinates calculation means for, based on a position of the one target image or positions of the plurality of target images in the taken image, calculating designated coordinates associated with a display area of the display device, wherein

the image processing means moves the display image in accordance with a change in the designated coordinates.

12. The image processing apparatus according to claim 2, further comprising tilt calculation means for, based on a position of the one target image or positions of the plurality of target images in the taken image, calculating a tilt of the input device, wherein

the image processing means rotates the display image in accordance with a change in the tilt.

13. The image processing apparatus according to claim 2, wherein the image processing means performs at least either one of enlargement and reduction of the display image by sequentially changing, in accordance with the change in the distance, a position of a virtual camera placed in a virtual space.

14. The image processing apparatus according to claim 2, wherein the image processing means performs at least either one of enlargement and reduction of the display image by sequentially changing, in accordance with the change in the distance, a display size of a two-dimensional image.

15. The image processing apparatus according to claim 1, further comprising velocity calculation means for, based on the target image data, sequentially calculating a moving velocity of the input device with respect to the one or the plurality of imaging targets, wherein

the image processing means performs at least either one of enlargement and reduction of the display image in accordance with the moving velocity.

16. A storage medium storing an image processing program to be executed by a computer which performs image processing in accordance with an input operation performed by an input device including image pickup means for taking an image of one or a plurality of imaging targets, the image processing program causing the computer to perform:

a target image data obtaining step of sequentially obtaining, from one target image of the one imaging target or a plurality of target images of the plurality of imaging targets in the image taken by the image pickup means, target image data indicating a distance between the plurality of target images or a size of the one target image;

an image processing step of performing at least either one of enlargement and reduction of a display image in accordance with a change in the target image data; and

a display control step of displaying on a display device the display image processed at the image processing step.

17. The storage medium storing the image processing program according to claim 16, wherein

the image processing step includes:

a distance calculation step of, based on the target image data, sequentially calculating a distance between the image pickup means and the one or the plurality of the imaging targets; and

an enlarging/reducing step of performing at least either one of enlargement and reduction of the display image in accordance with a change in the distance.

18. The storage medium storing the image processing program according to claim 17, wherein at the image processing step, the display image is enlarged in accordance with an increase in the distance, and reduced in accordance with a decrease in the distance.

19. The storage medium storing the image processing program according to claim 17, wherein at the image processing step, the display image is reduced in accordance with an increase in the distance, and enlarged in accordance with a decrease in the distance.

20. The storage medium storing the image processing program according to claim 17, wherein the image processing step performs at least either one of enlargement and reduction of the display image in accordance with the change which occurs, during a predetermined time period, in the distance.

21. The storage medium storing the image processing program according to claim 20, wherein at the image processing step, the display image is enlarged in accordance with an increase in the distance, and reduced in accordance with a decrease in the distance.

22. The storage medium storing the image processing program according to claim 20, wherein at the image processing step, the display image is reduced in accordance with an increase in the distance, and enlarged in accordance with a decrease in the distance.

23. The storage medium storing the image processing program according to claim 20, wherein the input device includes at least one pressable operation key and outputs at least operation information corresponding to a state of the operation key being pressed, the storage medium further causing the computer to perform:

an operation information obtaining step of obtaining the operation information; and

a storage control step of, when the operation information indicates that the operation key has just started being pressed, storing in a memory the distance calculated at the distance calculation step, wherein

the image processing step includes a difference calculation step of obtaining the change in the distance, which occurs during the predetermined time period which is a time period during which the operation key is pressed, by sequentially calculating a difference between the distance stored in the memory and the distance which is calculated, while the operation information indicates that the operation key is currently pressed, at the distance calculation step, and

the image processing step performs at least either one of enlargement and reduction of the display image in accordance with the difference.

24. The storage medium storing the image processing program according to claim 23, wherein at the image processing step, the display image is enlarged in accordance with an increase in the distance, and reduced in accordance with a decrease in the distance.

25. The storage medium storing the image processing program according to claim 23, wherein at the image processing step, the display image is reduced in accordance with an increase in the distance, and enlarged in accordance with a decrease in the distance.

26. The storage medium storing the image processing program according to claim 17, further causing the computer to perform a designated coordinates calculation step of, based on a position of the one target image or positions of the plurality of target images in the taken image, calculating designated coordinates associated with a display area of the display device, wherein

at the image processing step, the display image is moved in accordance with a change in the designated coordinates.

27. The storage medium storing the image processing program according to claim 17, further causing the computer to perform a tilt calculation step of, based on a position of the one target image or positions of the plurality of target images in the taken image, calculating a tilt of the input device, wherein

at the image processing step, the display image is rotated in accordance with a change in the tilt.

28. The storage medium storing the image processing program according to claim 17, wherein the image processing step performs at least either one of enlargement and reduction of the display image by sequentially changing, in accordance with the change in the distance, a position of a virtual camera placed in a virtual space.

29. The storage medium storing the image processing program according to claim 17, wherein the image processing step performs at least either one of enlargement and reduction of the display image by sequentially changing, in accordance with the change in the distance, a display size of a two-dimensional image.

30. The storage medium storing the image processing program according to claim 16, further causing the computer to perform a velocity calculation step of, based on the target image data, sequentially calculating a moving velocity of the input device with respect to the one or the plurality of imaging targets, wherein

the image processing step performs at least either one of enlargement and reduction of the display image in accordance with the moving velocity.