Storage medium storing image processing program and image processing apparatus

• US 9,720,556 B2
• Filed: 05/21/2012
• Issued: 08/01/2017
• Est. Priority Date: 10/01/2007
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Yamashita, Yoshikazu, Minato, Keiichi
• Primary Examiner(s): HARRISON, CHANTE E
• Application Number:
  US13/476,705
• Publication Number:
  US 20120236034A1
• Time in Patent Office: 1,898 Days
• Field of Search: 345660, 345156-158
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/6653 for altering the visibility of an object, e.g. preventing the occlusion of an object, partially hiding an object
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Abstract

An image is placed, based on a predetermined designation input, on a predetermined placement position in relation to a display area, and displayed. The image is enlarged and reduced in accordance with an enlargement ratio of the image, and an area within which the placement position can be set is changed in relation to the display area in accordance with the enlargement ratio. Then, based on an obtained designation input, a position within the area, which position corresponds to the designation input, is calculated as a placement position, and the image having been enlarged and reduced is placed on the placement position. The image is then displayed on a display device.

Claims

1. A storage medium storing an image processing program to be executed by a computer of an image processing apparatus for placing a displayed image on a predetermined placement position in a display area in accordance with a designation input obtained from an input device having an image-pickup configured to take an image of an imaging target, the image processing program comprising instructions causing the computer to:

obtain the designation input from the input device;

calculate, based on a position of the imaging target in the taken image, a distance between the imaging target and the image-pickup;

based on the distance, change a rate of change of the placement position in relation to a change of the position of the imaging target in the taken image, thereby calculating the placement position;

placing the image on the placement position; and

causing a display device to display the image at the image placement position,

wherein the placement position calculation instructions cause the computer to calculate the placement position in such a manner as to increase the rate of change of the placement position in accordance with a decrease in the distance calculated by the distance calculation.

2. The storage medium storing the image processing program according to claim 1, further causing the computer to function as an area setter for, in accordance with the distance calculated by the distance calculation, changing, in relation to the display area, an area within which the placement position is allowed to be set, wherein

the area setting sets the area so as to extend in accordance with a decrease in the distance calculated by the distance calculation.

3. The storage medium storing the image processing program according to claim 1, wherein a plurality of imaging targets are provided, and

the distance calculation calculates a distance between the image-pickup and the imaging targets, based on a distance between the imaging targets in the taken image.

4. An image processing apparatus for placing an image on a predetermined placement position in relation to a display area of a display device in accordance with a designation input obtained from an input device having an image-pickup for taking an image of an imaging target, the image processing apparatus comprising a processor configured to:

obtain the designation input from the input device;

calculate a distance, based on a position of the imaging target in the taken image, between the imaging target and the image-pickup;

based on the calculated distance, change a rate of change of the placement position in relation to a change of the position of the imaging target in the taken image, thereby calculating the placement position;

place the image on the placement position; and

causing a display device to display the placed image, wherein the placement position calculation calculates the placement position in such a manner as to increase the rate of change of the placement position in accordance with a decrease in the distance calculated by the distance calculation.

5. The image processing apparatus according to claim 4, wherein the processor is further configured to, in accordance with the distance calculated by the distance calculation, change, in relation to the display area, an area within which the placement position is allowed to be set, and set the area so as to extend in accordance with a decrease in the distance calculated by the distance calculation.

6. In an interactive display system for use with a display device and a handheld imaging device that images an imaging target, the display system comprising a processor that receives information from the handheld imaging device and is configured to enable a user to display and move an object on the display device in accordance with changes in where the input device is pointing, a method comprising:

(a) using the processor to automatically determine where to display the object based on information obtained by the handheld imaging device;

(b) using the processor to automatically determine, based on optical information obtained by the handheld imaging device, a change in distance of the handheld imaging device from the display device;

(c) using the processor to calculate an object image movement scaling value specifying the rate the placement position of a displayed object changes with change in the input device pointing direction so as to increase the rate of change of the placement position in accordance with a decrease in the distance of the handheld imaging device from the display device; and

(d) using the processor to automatically control how the displayed object moves on the display device in response to movement of the input device based at least in part on the calculated object image movement scaling value.

7. In an interactive display system for use with a display device and a handheld imaging device that images an imaging target, the display system comprising a processor that receives information from the handheld imaging device and is configured to enable a user to display and move an object on the display device in accordance with changes in where the input device is pointing, a method comprising:

(a) using the processor to automatically determine where to display the object based on information obtained from the handheld imaging device;

(b) using the processor to automatically determine, based on optical information, a change in distance between the handheld imaging device and the display device;

(c) using the processor to calculate an object image movement scaling value specifying the rate the placement position of a displayed object changes with change in the input device pointing direction so as to increase the rate of change of the placement position in accordance with a decrease in the distance; and

(d) using the processor to automatically control how the displayed object responds to movement of the input device based at least in part on the calculated object image movement scaling value.

8. A non-transitory computer readable storage medium having stored thereon an image processing program to be executed by a computer which displays a predetermined object image at a predetermined placement position in a display area on a display device in accordance with information obtained from an input device, the input device having an image-pickup device/sensor for taking images of an imaging target in a pointing direction of the input device, and for moving and displaying the predetermined object image at a determined placement position in the display area in accordance with changes in a position of the image of the imaging target in the image-pick up device corresponding to a movement of a pointing direction of the input device, the image processing program causing the computer to perform operations comprising:

determining a designation input based on information obtained from the input device and calculating the predetermined placement position based at least in part on the designation input;

calculating a placement position in accordance with changes in a position of the image of the imaging target in the image-pick up device, wherein the placement position is calculated using a user-settable image object scale value,

using the user-settable image object scale value to affect a rate of change of a placement position with respect to a movement in the position of the imaging target in the taken image in a manner that causes an increase in the rate of change of the placement position in accordance with a decrease in a distance between the imaging target and the image-pickup device;

displaying the image at the calculated placement position; and

causing a display device to display the object image at the calculated placement position on the display device.

9. An interactive display system for use with a display device and an associated imaging target, the display system comprising:

a handheld input device having an image-pickup for capturing views of the imaging target in a pointing direction of the input device;

a graphics processor configured to display an object at a placement position on the display device; and

a processor that receives information from the input device image-pickup and is operatively coupled to the graphics processor, the processor being configured to enable a user to perform object placement setting for displaying and moving the object on the display device in accordance with changes in the pointing direction of the input device, the processor being used to;

(a) automatically determine an object placement position based on information obtained from the input device image-pick up;

(b) based on information obtained from the input device image pickup, automatically determine a change in at least one aspect of distance of the input device from the imaging target as the user moves the input device toward or away from the display device;

(c) automatically controlling the graphics processor to generate a movable object to be displayed on the display device by using an object image size scale value based at least in part on the change in distance of the input device from the imaging target;

(d) using the object image size scale value to calculate an object image movement scaling value specifying the rate the placement position of the displayed movable object changes with change in the input device pointing direction; and

(e) using the object image size scale value and the object image movement scaling value to automatically control how the displayed object responds to movement of the input device.

10. The display system of claim 9 wherein the processor sets the object image size scale value based on user movement of the image-pickup device toward or away from the display device during an image object position setting process.

11. The system of claim 9 wherein the input device includes at least one user-depressible control, and the processor is connected to operatively receive an indication when the user depresses the user-depressible control and sets the object image size scale value scale value in response to said user depression of the user-depressible control.

12. The system of claim 9 wherein the processor uses the object image size scale value to calculate an object image movement scaling value thereby increasing the rate of change of the placement position in accordance with a decrease in distance between the image pickup and the imaging target.

13. A non-transitory computer readable storage medium having stored thereon an image processing program for performing an object image setting process to be executed by a computer operatively connected to a display device, the object image setting process enabling user control of movement, placement and display of a predetermined two-dimensional object image at a user designated placement position within a set display area in accordance with position designation information obtained from an input device, the input device having an image-pickup device for taking an image of an imaging target, computing an updated placement position in accordance with movements of an image of the imaging target acquired by the image-pickup device, and displaying the predetermined image at the updated placement position, the image processing program causing the computer to perform operations comprising:

initiating an object image setting process in response to a user input;

determining a designation input based on information obtained from the input device and determining a placement position of the object image in the display area based at least in part on the designation input, wherein the placement position is changed at a predetermined rate in relation to a movement/change in a position of the imaging target in the taken image;

determining a change in at least one aspect of a distance between the display device and the image-pickup device, the change based upon a user movement of the input device toward or away from the display device during the initiated object image setting process;

calculating an object image size scale value based at least in part on the determined change in the aspect of the distance;

using the object image size scale value to calculate an object image movement scaling value specifying a rate of change of a placement position with respect to a change of a position of an image of the imaging target acquired by the image-pickup device;

using the object image movement scaling value for determining an updated placement position of the object image in the display area in accordance with movements of an image of the imaging target acquired by the image-pickup device;

placing the object image at the updated placement position within the display area; and

causing the display device to display the object image at the updated placement position.

14. An interactive display system for use with a display device and a handheld imaging device that images an imaging target, the display system comprising a processor that receives information from the handheld imaging device and is configured to enable a user to display and move an object on the display device in accordance with changes in where the input device is pointing, the processor being used to:

(a) automatically determine where to display the object based on information obtained from the handheld imaging device;

(b) automatically determine a change in distance between the handheld imaging device and the display device;

(c) calculate an object image movement scaling value specifying the rate the placement position of a displayed object changes with change in the input device pointing direction so as to increase the rate of change of the placement position in accordance with an increase or decrease in the distance; and

(d) automatically control how the displayed object responds to movement of the input device based at least in part on the calculated object image movement scaling value.