IMAGE PROCESSING APPARATUS, METHOD FOR CONTROLLING THE SAME, AND PROGRAM THEREFOR

• US 20130100165A1
• Filed: 10/22/2012
• Published: 04/25/2013
• Est. Priority Date: 10/25/2011
• Status: Abandoned Application

• Current Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Original Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Inventors: Komiyama, Tomoaki
• Application Number:
  US13/656,914
• Publication Number:
  US 20130100165A1
• Time in Patent Office: Days
• US Class Current: 345/634
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 5/397 Arrangements specially adapted for transferring the contents of two or more bit-mapped memories to the screen simultaneously, e.g. for mixing or overlay G09G5/02 takes precedence

Abstract

An image processing apparatus includes an acquisition unit, a display control unit, and a determination unit. The acquisition unit acquires information about a plurality of markers included in an image. The display control unit performs control to superimpose on the image and display on a display unit a plurality of virtual objects corresponding to the plurality of markers. The determination unit determines whether the plurality of virtual objects is a specific combination. In response to the determination unit determining that the plurality of virtual objects is a specific combination, the display control unit performs control to superimpose on the image the plurality of virtual objects in a specific arrangement corresponding to the specific combination.

Claims

1. An image processing apparatus comprising:

an acquisition unit configured to acquire information about a plurality of markers included in an image;

a display control unit configured to perform control to superimpose on the image and display on a display unit a plurality of virtual objects corresponding to the plurality of markers; and

a determination unit configured to determine whether the plurality of virtual objects is a specific combination,

wherein, in response to the determination unit determining that the plurality of virtual objects is a specific combination, the display control unit performs control to superimpose on the image the plurality of virtual objects in a specific arrangement corresponding to the specific combination.

2. The image processing apparatus according to claim 1, wherein the specific arrangement is a feasible arrangement.

3. The image processing apparatus according to claim 1, wherein, in response to the determination unit determining that the plurality of virtual objects is a specific combination, the display control unit performs control to superimpose and display on the image the plurality of virtual objects in the specific arrangement based on positions of the corresponding markers on the image.

4. The image processing apparatus according to claim 1, wherein, in response to the determination unit determining that the plurality of virtual objects is a specific combination, the display control unit performs control to superimpose and display on the image the plurality of virtual objects in the specific arrangement based on at least any one of positions of the corresponding markers on the image.

5. The image processing apparatus according to claim 1, wherein, in response to the determination unit determining that the plurality of virtual objects is a specific combination, the display control unit performs control to superimpose and display on the image the plurality of virtual objects in the specific arrangement between positions of the corresponding markers on the image.

6. The image processing apparatus according to claim 1, wherein, in response to the determination unit determining that the plurality of virtual objects is not a specific combination, the display control unit performs control to superimpose and display on the image the plurality of virtual objects in an arrangement different from the specific arrangement based on positions of the corresponding markers on the image.

7. The image processing apparatus according to claim 1, wherein the plurality of markers include combination information indicating a combinational relation between virtual objects of the plurality of markers and virtual objects of other markers, and

wherein, based on the combination information, the determination unit determines whether the plurality of virtual objects corresponding to the plurality of markers is a specific combination.

8. The image processing apparatus according to claim 7, wherein, in response to the determination unit determining that a virtual object common to a plurality of specific combinations is included in the virtual objects, the display control unit displays any one combination of virtual objects, out of the plurality of combinations, in the specific arrangement.

9. The image processing apparatus according to claim 8, further comprising:

a receiving unit configured to receive a command for selecting any one combination out of the plurality of specific combinations,

wherein the display control unit displays in the specific arrangement a specific combination of visual objects selected by the command received by the receiving unit.

10. The image processing apparatus according to claim 9, wherein virtual objects included in combinations not selected by the command received by the receiving unit are not displayed other than the virtual object common to the plurality of specific combinations.

11. The image processing apparatus according to claim 1, wherein, via the display control unit, a user can select whether the plurality of virtual objects is to be displayed in the specific arrangement.

12. The image processing apparatus according to claim 1, further comprising:

an arrangement determination unit configured to, based on positions of the plurality of markers on the image, determine whether the plurality of virtual objects is to be displayed in the specific arrangement.

13. The image processing apparatus according to claim 1, wherein, in response to a distance between the plurality of markers being equal to or larger than a predetermined value, the display control unit performs control not to display the plurality of virtual objects in the specific arrangement, even if the plurality of virtual objects corresponding to the plurality of markers is a specific combination.

14. The image processing apparatus according to claim 1, wherein, in response to postures of the plurality of markers not being oriented in same direction, the display control unit performs control not to display the plurality of virtual objects in the specific arrangement, even if the plurality of virtual objects corresponding to the plurality of markers is a specific combination.

15. The image processing apparatus according to claim 1, wherein, in response to the plurality of markers both not being adjacently arranged and not facing each other, the display control unit performs control not to display the plurality of virtual objects in the specific arrangement, even if the plurality of virtual objects corresponding to the plurality of markers is a specific combination.

16. The image processing apparatus according to claim 1, wherein, in response to the plurality of markers not being adjacently arranged and oriented in opposite directions, the display control unit performs control not to display the plurality of virtual objects in the specific arrangement, even if the plurality of virtual objects corresponding to the plurality of markers is a specific combination.

17. The image processing apparatus according to claim 1, wherein the specific arrangement refers to an arrangement indicating a positional relation different from the on-image positional relation between the plurality of markers corresponding to the plurality of virtual objects determined to be a specific combination by the determination unit.

18. The image processing apparatus according to claim 1, wherein the plurality of markers include arrangement information indicating a positional relation between virtual objects corresponding to the plurality of markers and other virtual objects having a combinational relation with the virtual objects, and

wherein the display control unit displays, based on the arrangement information, the virtual objects corresponding to the plurality of markers and other virtual objects having a combinational relation with the virtual objects.

19. The image processing apparatus according to claim 18, wherein the arrangement information includes information defining an orientation of other virtual objects having a combinational relation with the virtual object corresponding to the marker, relative to the virtual object.

20. The image processing apparatus according to claim 18, wherein, in response to performing control to display the plurality of virtual objects corresponding to the plurality of markers in the specific arrangement, the display control unit performs control to display the virtual objects in an arrangement based on the on-image positions of the plurality of markers having the arrangement information, out of the on-image positions of the plurality of markers.

21. The image processing apparatus according to claim 1, further comprising:

a detection unit configured to detect the plurality of markers from the image.

22. The image processing apparatus according to claim 21, wherein, in response to the detection unit detecting the plurality of markers, the on-image positions of the plurality of markers are determined.

23. The image processing apparatus according to claim 21, further comprising:

a determination unit configured to determine whether the plurality of virtual objects are to be displayed in the specific arrangement, based on detection time of the plurality of markers detected by the detection unit.

24. The image processing apparatus according to claim 21, wherein, in response to a difference between detection time of the plurality of markers detected by the detection unit being equal to or larger than a predetermined period of time, the display control unit performs control not to display the virtual objects corresponding to the markers in the specific arrangement.

25. The image processing apparatus according to claim 21, further comprising:

an acquisition unit configured to acquire identification information indicated by the plurality of markers detected by the detection unit;

a transmission unit configured to transmit the identification information to an external apparatus; and

a receiving unit configured to receive virtual objects corresponding to the identification information from the external apparatus.

26. The image processing apparatus according to claim 1, further comprising:

a storage unit configured to store the plurality of virtual objects.

27. The image processing apparatus according to claim 26, wherein the storage unit stores a relation between the plurality of virtual objects and the plurality of markers.

28. The image processing apparatus according to claim 1, wherein, according to a user command, the display control unit changes display of the plurality of virtual objects superimposed and displayed on the image.

29. An information processing apparatus capable of communicating with an image processing apparatus, the information processing apparatus comprising:

a receiving unit configured to receive information about a plurality of markers included in an image from the image processing apparatus;

a storage unit configured to store virtual objects corresponding to the plurality of markers;

a determination unit configured to determine whether the plurality of virtual objects corresponding to the plurality of markers is a specific combination;

a processing unit configured to process, in response to the determination unit determining that the plurality of virtual objects is a specific combination, the plurality of virtual objects based on the information about the plurality of markers received from the image processing apparatus; and

a transmission unit configured to transmit the virtual objects processed by the processing unit to the image processing apparatus.

30. The information processing apparatus according to claim 29, wherein the storage unit stores a virtual object table which associates other virtual objects with the virtual object as recommended virtual objects, and

wherein, based on the virtual object table and the information about the plurality of markers received by the receiving unit, the transmission unit transmits to the image processing apparatus other virtual objects associated with the virtual objects corresponding to the plurality of markers received by the receiving unit, as a recommended virtual object, together with the virtual object corresponding to the marker received by the receiving unit.

31. The information processing apparatus according to claim 30, wherein the recommended virtual object is a virtual object most frequently displayed together with the virtual object.

32. A method to control an image processing apparatus, the method comprising:

acquiring information about a plurality of markers included in an image;

performing control to superimpose on the image and display a plurality of virtual objects corresponding to the plurality of markers; and

determining whether the plurality of virtual objects is a specific combination,

wherein, in response to determining that the plurality of virtual objects is a specific combination, performing control includes performing control to superimpose on the image the plurality of virtual objects in a specific arrangement corresponding to the specific combination.

34. A non-transitory storage medium storing a computer-readable program that causes a computer to perform the method according to claim 32.

33. A method to control an information processing apparatus capable of communicating with an image processing apparatus, the method comprising:

receiving information about a plurality of markers included in an image from the image processing apparatus;

storing virtual objects corresponding to the plurality of markers;

determining whether the plurality of virtual objects corresponding to the plurality of markers is a specific combination;

processing, in response to determining that the plurality of virtual objects is a specific combination, the plurality of virtual objects based on the information about the plurality of markers received from the image processing apparatus; and

transmitting the processed virtual objects to the image processing apparatus.

35. A non-transitory storage medium storing a computer-readable program that causes a computer to perform the method according to claim 33.