Information display apparatus, information display system, control method thereof, and program

• US 9,721,538 B2
• Filed: 12/26/2013
• Issued: 08/01/2017
• Est. Priority Date: 12/29/2012
• Status: Active Grant

• Current Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Original Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Inventors: Naganawa, Yoshihiro
• Primary Examiner(s): Tryder, Gregory J
• Assistant Examiner(s): Wang, Yuehan
• Application Number:
  US14/140,832
• Publication Number:
  US 20140184641A1
• Time in Patent Office: 1,314 Days
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G09G 2320/106 Determination of movement vectors or equivalent parameters within the image
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence

Abstract

An information display system capable of reducing a trouble about display caused by display of content information corresponding to numerous markers and reducing process loads and a control method of the information display system are provided. In this system, a movement speed and a movement direction of a client apparatus operated by a user are detected, and a size and a position of a marker recognition area for recognizing a marker in an imaged image are determined according to the detected movement speed and the detected movement direction. Then, the content information corresponding to the marker recognized in the marker recognition area is superposed on the imaged image and output to a display screen.

Claims

1. An information display apparatus which superposes information on an imaged image and displays them, comprising:

a detection unit configured to detect, with a sensor, a movement direction of the imaged image relative to an imaging target;

a determination unit configured to determine a location of a target area within the imaged image, based on the movement direction detected by the detection unit, wherein the location of the target area is shifted within the imaged image in the movement direction detected by the detection unit;

a recognition unit configured to recognize an object in the imaged image; and

a display control unit configured to superpose, on the imaged image, information related to the object when the object is in the target area, and not superpose, on the imaged image, the information related to the object when the object is outside the target area,

wherein, in a case where a ratio of the objects existing in the imaged image does not exceed a threshold value,

the detection unit does not measure the movement direction,

the determination unit determines the whole imaged image as the target area,

the recognition unit recognizes the objects recognizable in the imaged image, and

the display control unit superposes, on the imaged image, the information related to the recognized objects in the imaged image.

2. The information display apparatus according to claim 1, wherein

the detection unit further detects a speed of movement of the imaged image, and

in a case where the speed detected by the detection unit is equal to or lower than a first threshold value, the determination unit determines the location of the target area as a first area which is located near the center, and, in a case where the speed detected by the detection unit exceeds the first threshold value, the determination unit determines the location of the target area as a second area which is shifted within the imaging target in the movement direction detected by the detection unit.

3. The information display apparatus according to claim 2, wherein, in a case where the speed detected by the detection unit exceeds a second threshold value larger than the first threshold value, the determination unit determines the target area as a third area which is narrower than the second area.

4. The information display apparatus according to claim 1, wherein the recognition unit recognizes the object in the imaged image existing in the target area at a changed position within the imaged image.

5. An information display apparatus which superposes information on an imaged image and displays them, comprising:

a detection unit configured to detect a movement direction of the imaged image relative to an imaging target;

a determination unit configured to determine a location of a target area within the imaged image, based on the movement direction detected by the detection unit, wherein the location of the target area is shifted within the imaged image in the movement direction detected by the detection unit;

a recognition unit configured to recognize an object in the imaged image; and

a display control unit configured to superpose, on the imaged image, a display form of the information related to the object existing in the target area and a display form of the information related to the object existing outside the target area as respectively different display forms,

wherein the detection unit further detects a speed of movement of the imaged image, and

wherein, in a case where the speed detected by the detection unit is equal to or lower than a first threshold value, the determination unit determines the target area as a first area which is near the center, and, in a case where the speed detected by the detection unit exceeds the first threshold value, the determination unit determines the target area as a second area which is shifted within the imaging target in the movement direction detected by the detection unit, and

wherein, in a case where a ratio of the objects existing in the imaged image does not exceed a threshold value, the detection unit does not measure the movement direction and the speed, the determination unit determines the whole imaged image as the target area, the recognition unit recognizes the objects being recognizable in the imaged image, and the display control unit superposes, on the imaged image, the information related to the recognized objects in the imaged image.

6. The information display apparatus according to claim 5, wherein the recognition unit recognizes the object in the imaged image existing in the target area at a changed position within the imaged image.

7. The information display apparatus according to claim 5, wherein the display control unit displays the information related to the object existing in the target area in more detail as compared with the information related to the object existing outside the target area.

8. The information display apparatus according to claim 5, wherein the display control unit displays the information related to the object existing in the target area with more emphasis as compared with the information related to the object existing outside the target area.

9. The information display apparatus according to claim 5, wherein the display control unit displays the information related to the object when the object is in the target area with characters larger than those for displaying the information related to the object when the object is outside the target area.

10. The information display apparatus according to claim 5, wherein the display control unit displays the information related to the object when the object is in the target area and the information related to the object when the object is outside the target area respectively with different fonts.

11. The information display apparatus according to claim 5, wherein the display control unit displays the information related to the object when the object is in the target area and the information related to the object when the object is outside the target area respectively with different colors.

12. An information display system which includes an information display apparatus configured to superpose information on an imaged image and display them, and an information processing apparatus connected to the information display apparatus through a network, wherein:

the information display apparatus comprises

a measurement unit configured to measure a movement direction of an imaging direction relative to an imaging target,

a change unit configured to change a position of a target area identified from the imaged image based on the movement direction measured by the measurement unit, wherein the position of the target area is shifted within the imaged image in the movement direction measured by the measurement unit,

a recognition unit configured to recognize an object in the imaged image,

a transmission unit configured to transmit recognition information of the object recognized by the recognition unit, to the information processing apparatus,

a reception unit configured to receive information related to the object, from the information processing apparatus, and

a display control unit configured to superpose the received information related to the object on the target area in the imaged image,

wherein, in a case where a ratio of the objects existing in the imaged image does not exceed a threshold value, the detection unit does not measure the movement direction, the determination unit determines the whole imaged image as the target area, the recognition unit recognizes the objects recognizable in the imaged image, and the display control unit superposes, on the imaged image, the information related to the recognized objects in the imaged image; and

the information processing apparatus comprises

a reception unit configured to receive the recognition information of the object from the information display apparatus, and

a transmission unit configured to transmit the information related to the object to the information display apparatus.

13. A control method for an information display apparatus which superposes information on an imaged image of a client apparatus and displays them, the method comprising:

detecting, by a sensor, a direction in which the in-imaging client apparatus is moved relative to an imaging target;

determining a location of a target area within the imaged image based on the detected direction, wherein the location of the target area is shifted within the imaged image in the direction detected by the sensor;

recognizing an object in the imaged image; and

superposing, on the imaged image, the information related to the object existing in the target area, and not superposing, on the imaged image, the information related to the object existing outside the target area,

wherein, in a case where a ratio of the objects existing in the imaged image does not exceed a threshold value,

the detecting does not measure the movement direction,

the determining determines the whole imaged image as the target area,

the recognizing recognizes the objects recognizable in the imaged image, and

the superposing superposes, on the imaged image, the information related to the recognized objects in the imaged image.

14. A control method for an information display apparatus which superposes information on an imaged image of a client apparatus and displays them, the method comprising:

detecting a direction in which the in-imaging client apparatus is moved relative to an imaging target and a speed of movement of the imaged image;

determining a location of a target area within the imaged image based on the detected direction, wherein the location of the target area is shifted within the imaged image in the detected direction;

recognizing an object in the imaged image; and

superposing, on the imaged image, a display form of the information related to the recognized object existing in the target area and an display form of the information related to the recognized object existing outside the target area as respectively different display forms,

wherein, in a case where the speed detected is equal to or lower than a first threshold value, the determining determines the target area as a first area which is near the center, and, in a case where the speed detected exceeds the first threshold value, the determining determines the target area as a second area which is shifted within the imaging target in the detected movement direction, and

wherein, in a case where a ratio of the objects existing in the imaged image does not exceed a threshold value, the detecting does not measure the movement direction and the speed, the determining determines the whole imaged image as the target area, the recognition unit recognizes the objects being recognizable in the imaged image, and the superposing superposes, on the imaged image, the information related to the recognized objects in the imaged image.

15. A non-transitory computer-readable storage medium which stores a computer-executable program to achieve a control method for an information display apparatus which superposes information on an imaged image of a client apparatus and displays them, the control method comprising:

detecting, by a sensor, a direction in which the in-imaging client apparatus is moved relative to an imaging target;

determining a location of a target area within the imaged image based on the detected direction, wherein the location of the target area is shifted within the imaged image in the detected direction;

recognizing an object in the imaged image; and

superposing, on the imaged image, the information related to the object existing in the target area, and not superposing, on the imaged image, the information related to the object existing outside the target area,

wherein, in a case where a ratio of the objects existing in the imaged image does not exceed a threshold value,

the detecting does not measure the movement direction,

the determining determines the whole imaged image as the target area,

the recognizing recognizes the objects recognizable in the imaged image, and

the superposing superposes, on the imaged image, the information related to the recognized objects in the imaged image.

16. A non-transitory computer-readable storage medium which stores a computer-executable program to achieve a control method for an information display apparatus which superposes information on an imaged image of a client apparatus and displays them, the method comprising:

detecting a direction in which the in-imaging client apparatus is moved relative to an imaging target and a speed of movement of the imaged image;

determining a location of a target area within the imaged image based on the detected direction, wherein the location of the target area is shifted within the imaged image in the detected direction;

recognizing an object in the imaged image; and

superposing, on the imaged image, a display form of the information related to the recognized object existing in the target area and an display form of the information related to the recognized object existing outside the target area as respectively different display forms,

wherein, in a case where the speed detected is equal to or lower than a first threshold value, the determining determines the target area as a first area which is near the center, and, in a case where the speed detected exceeds the first threshold value, the determining determines the target area as a second area which is shifted within the imaging target in the detected movement direction, and

wherein, in a case where a ratio of the objects existing in the imaged image does not exceed a threshold value, the detecting does not measure the movement direction and the speed, the determining determines the whole imaged image as the target area, the recognition unit recognizes the objects being recognizable in the imaged image, and the superposing superposes, on the imaged image, the information related to the recognized objects in the imaged image.