STORAGE MEDIUM HAVING STORED THEREON IMAGE PROCESSING PROGRAM, IMAGE PROCESSING APPARATUS, IMAGE PROCESSING SYSTEM, AND IMAGE PROCESSING METHOD

• US 20120133676A1
• Filed: 08/03/2011
• Published: 05/31/2012
• Est. Priority Date: 11/30/2010
• Status: Abandoned Application

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: KITAHARA, Shinji
• Application Number:
  US13/197,231
• Publication Number:
  US 20120133676A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/5252 using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 9/74 for obtaining special effects H04N9/65 - H04N9/73 take precedence

Abstract

At least one virtual object for which a predetermined color is set is placed in a virtual world. In a captured image captured by a real camera, at least one pixel corresponding to the predetermined color is detected, using color information including at least one selected from the group including RGB values, a hue, a saturation, and a brightness of each pixel of the captured image. When the pixel corresponding to the predetermined color has been detected, a predetermined process is performed on the virtual object for which the predetermined color is set. An image of the virtual world where at least the virtual object is placed is displayed on a display device.

Claims

1. A computer-readable storage medium having stored thereon an image processing program to be executed by a computer of an image processing apparatus that processes an image to be displayed on a display device, the image processing program causing the computer to function as:

captured image acquisition means for acquiring a captured image captured by a real camera;

object placement means for placing in a virtual world at least one virtual object for which a predetermined color is set;

color detection means for, in the captured image acquired by the captured image acquisition means, detecting at least one pixel corresponding to the predetermined color set for the virtual object placed in the virtual world, using color information including at least one selected from the group including RGB values, a hue, a saturation, and a brightness of each pixel of the captured image;

object process means for, when the color detection means has detected the pixel corresponding to the predetermined color, performing a predetermined process on the virtual object for which the predetermined color is set; and

image display control means for displaying on the display device an image of the virtual world where at least the virtual object is placed.

2. The computer-readable storage medium having stored thereon the image processing program according to claim 1, the image processing program further causing the computer to function as:

image combination means for generating a combined image obtained by combining the captured image acquired by the captured image acquisition means with the image of the virtual world where the virtual object is placed, wherein

the image display control means displays the combined image generated by the image combination means on the display device.

3. The computer-readable storage medium having stored thereon the image processing program according to claim 2, wherein

when the image combination means combines the captured image with the image of the virtual world, the object process means performs the predetermined process on, among the virtual objects for which the predetermined color is set, a virtual object that overlaps the pixel corresponding to the predetermined color when combined with the captured image.

4. The computer-readable storage medium having stored thereon the image processing program according to claim 1, wherein

the object placement means places in the virtual world a plurality of virtual objects for which the predetermined color is set, and

the object process means performs the predetermined process on, among the plurality of virtual objects for which the predetermined color is set, all the virtual objects that, when combined with the captured image, overlap pixels corresponding to a predetermined color that is the same as the predetermined color.

5. The computer-readable storage medium having stored thereon the image processing program according to claim 1, the image processing program further causing the computer to function as:

operation signal acquisition means for acquiring an operation signal in accordance with an operation of a user, wherein

when the color detection means has detected the pixel corresponding to the predetermined color and the operation signal acquisition means has acquired an operation signal indicating an operation of making an attack on a virtual object, the object process means makes a predetermined attack on the virtual object for which the predetermined color is set.

6. The computer-readable storage medium having stored thereon the image processing program according to claim 1, wherein

when the color detection means has detected the pixel corresponding to the predetermined color, the object process means sets a predetermined sign for the virtual object for which the predetermined color is set, and

the image display control means assigns the sign set by the object process means to the virtual object, and displays on the display device an image of the virtual world where the virtual object to which the sign is assigned is placed.

7. The computer-readable storage medium having stored thereon the image processing program according to claim 5, wherein

the object process means causes the virtual object on which the predetermined attack has been made, to disappear from the virtual world.

8. The computer-readable storage medium having stored thereon the image processing program according to claim 1, wherein

the color detection means detects, as the pixel corresponding to the predetermined color, a pixel having items of the color information indicating the saturation and the brightness that are equal to or greater than predetermined thresholds, respectively, and also having an item of the color information indicating the hue indicative of a value within a predetermined range.

9. The computer-readable storage medium having stored thereon the image processing program according to claim 1, wherein

a display color of the virtual object for which the predetermined color is set is set to substantially the same color as the predetermined color, and

the image display control means displays on the display device the virtual object for which the predetermined color is set, such that the set display color is included at least in part of an image representing the virtual object.

10. The computer-readable storage medium having stored thereon the image processing program according to claim 1, wherein

a display color of the virtual object for which the predetermined color is set is set to a substantially complementary color of the predetermined color, and

the image display control means displays on the display device the virtual object for which the predetermined color is set, such that the set display color is included at least in part of an image representing the virtual object.

11. The computer-readable storage medium having stored thereon the image processing program according to claim 1, wherein

the color detection means includes:

block division means for dividing the captured image into blocks each including a plurality of pixels; and

block RGB average value calculation means for calculating average values of RGB values of pixels included in each block, wherein

the color detection means detects, in the captured image, pixels corresponding to the predetermined color, on the basis of the average values of each block such that the block is a detection unit.

12. The computer-readable storage medium having stored thereon the image processing program according to claim 2, wherein

the captured image acquisition means repeatedly acquires captured images of a real world captured in real time by a real camera available to the image processing apparatus,

the color detection means repeatedly detects pixels corresponding to the predetermined color in the captured images, respectively, repeatedly acquired by the captured image acquisition means,

the object process means repeatedly performs the predetermined process on the virtual object on the basis of results of the repeated detections of the color detection means,

the image combination means repeatedly generates combined images by combining each of the captured images repeatedly acquired by the captured image acquisition means, with the image of the virtual world where the virtual object is placed, and

the image display control means repeatedly displays on the display device the combined images obtained by combining each of the captured images repeatedly acquired by the captured image acquisition means, with the image of the virtual world.

13. The computer-readable storage medium having stored thereon the image processing program according to claim 1, the image processing program further causing the computer to function as;

color setting means for, after the object process means has performed the predetermined process on the virtual object, changing the predetermined color of the virtual object to a different color.

14. The computer-readable storage medium having stored thereon the image processing program according to claim 1, the image processing program further causing the computer to function as:

process setting means for, when the color detection means has detected the pixel corresponding to the predetermined color, changing a content of the predetermined process to be performed on the virtual object for which the predetermined color is set, on the basis of the color information of the pixel.

15. An image processing apparatus that processes an image to be displayed on a display device, the image processing apparatus comprising:

captured image acquisition means for acquiring a captured image captured by a real camera;

object placement means for placing in a virtual world at least one virtual object for which a predetermined color is set;

color detection means for, in the captured image acquired by the captured image acquisition means, detecting at least one pixel corresponding to the predetermined color set for the virtual object placed in the virtual world, using color information including at least one selected from the group including RGB values, a hue, a saturation, and a brightness of each pixel of the captured image;

object process means for, when the color detection means has detected the pixel corresponding to the predetermined color, performing a predetermined process on the virtual object for which the predetermined color is set; and

image display control means for displaying on the display device an image of the virtual world where at least the virtual object is placed.

16. An image processing system, including a plurality of apparatuses configured to communicate with each other, that processes an image to be displayed on a display device, the display control system comprising:

captured image acquisition means for acquiring a captured image captured by a real camera;

object placement means for placing in a virtual world at least one virtual object for which a predetermined color is set;

color detection means for, in the captured image acquired by the captured image acquisition means, detecting at least one pixel corresponding to the predetermined color set for the virtual object placed in the virtual world, using color information including at least one selected from the group including RGB values, a hue, a saturation, and a brightness of each pixel of the captured image;

object process means for, when the color detection means has detected the pixel corresponding to the predetermined color, performing a predetermined process on the virtual object for which the predetermined color is set; and

image display control means for displaying on the display device an image of the virtual world where at least the virtual object is placed.

17. An image processing method performed by a processor or a cooperation of a plurality of processors included in an image processing system including at least one information processing apparatus capable of performing image processing for processing an image to be displayed on a display device, the image processing method comprising:

a captured image acquisition step of acquiring a captured image captured by a real camera;

an object placement step of placing in a virtual world at least one virtual object for which a predetermined color is set;

a color detection step of, in the captured image acquired in the captured image acquisition step, detecting at least one pixel corresponding to the predetermined color set for the virtual object placed in the virtual world, using color information including at least one selected from the group including RGB values, a hue, a saturation, and a brightness of each pixel of the captured image;

an object process step of, when the pixel corresponding to the predetermined color has been detected in the color detection step, performing a predetermined process on the virtual object for which the predetermined color is set; and

an image display control step of displaying on the display device an image of the virtual world where at least the virtual object is placed.