IMAGE DISPLAY SYSTEM, IMAGE DISPLAY APPARATUS, AND CONTROL METHOD THEREOF

• US 20090109241A1
• Filed: 10/23/2008
• Published: 04/30/2009
• Est. Priority Date: 10/26/2007
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Tsujimoto, Takuya
• Granted Patent: US 8,681,177 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04W 36/322 by location data
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management

Abstract

Upon receiving a communication switching instruction from a first wireless access point used for communication with an image processing apparatus, an image display apparatus disconnects communication with the first wireless access point. Simultaneously, the image display apparatus transmits, to a second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction. The image display apparatus displays, on a display unit, a captured image continuously acquired from am image capturing unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

Claims

1. An image display system comprising:

an image display apparatus including an image capturing unit which captures an image of physical space, a wireless communication unit which communicates with an image processing apparatus, including transmission of the captured image obtained by the image capturing unit to the image processing apparatus, and a display unit which receives, from the image processing apparatus, a composite image of the captured image and a virtual image composited by the image processing apparatus and displays the composite image;

the image processing apparatus which communicates with the image display apparatus, the image processing apparatus including a generation unit which generates position and orientation information of the image display apparatus, a virtual image generation unit which generates the virtual image based on the position and orientation information, a composition unit which composites the captured image and the virtual image, and a communication unit which communicates with the image display apparatus, including transmission of the composite image obtained by the composition unit to the image display apparatus; and

a plurality of wireless access points for communication between the image display apparatus and the image processing apparatus,

wherein the image processing apparatus further comprises;

a determination unit that determines, based on a position of the image display apparatus, whether switching from a first wireless access point used for communication with the image display apparatus to a second wireless access point is necessary; and

a switching instruction unit for transmitting a switching instruction to the first wireless access point and the second wireless access point if the determination unit determines that the switching is necessary, and

the image display apparatus further comprises;

a communication control unit that, upon receiving the communication switching instruction from the first wireless access point used for communication with the image processing apparatus, disconnects communication with the first wireless access point and transmits, to the second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction; and

a display control unit that displays, on the display unit, the captured image continuously acquired from the image capturing unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

2. An image display system comprising:

an image display apparatus including an image capturing unit which captures an image of physical space, a wireless communication unit which communicates with an image processing apparatus, including transmission of the captured image obtained by the image capturing unit to the image processing apparatus, a display unit which receives, from the image processing apparatus, a composite image of the captured image and a virtual image composited by the image processing apparatus and displays the composite image, and an image superimposition display unit which superimposes a message image on a display image of the display unit;

the image processing apparatus which communicates with the image display apparatus, the image processing apparatus including a generation unit which generates position and orientation information of the image display apparatus, a virtual image generation unit which generates the virtual image based on the position and orientation information, a composition unit which composites the captured image and the virtual image, and a communication unit which communicates with the image display apparatus, including transmission of the composite image obtained by the composition unit to the image display apparatus; and

a plurality of wireless access points for communication between the image display apparatus and the image processing apparatus,

wherein the image processing apparatus further comprises;

a determination unit that determines, based a position of the image display apparatus, whether switching from a first wireless access point used for communication with the image display apparatus to a second wireless access point is necessary; and

a switching instruction unit that, if the determination unit determines that the switching is necessary, transmits a switching instruction to the first wireless access point and the second wireless access point, and

the image display apparatus further comprises;

a communication control unit that, upon receiving the communication switching instruction from the first wireless access point used for communication with the image processing apparatus, disconnects communication with the first wireless access point and transmits, to the second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction; and

a display control unit that causes the image superimposition display unit to superimpose the message image on the display image of the display unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

3. The image display system according to claim 2, wherein one of the image display apparatus and the image processing apparatus further comprises an input unit which instructs switching selection of superimposition of the message image.

4. An image display system comprising:

an image display apparatus including an image capturing unit which captures an image of physical space, a storage unit which stores a virtual image, a composition unit which composites the captured image captured by the image capturing unit and the virtual image stored in the storage unit, a display unit which displays a composite image composited by the composition unit, and a wireless communication unit which communicates with an image processing apparatus, including reception of the virtual image from the image processing apparatus;

the image processing apparatus which communicates with the image display apparatus, the image processing apparatus including a virtual image generation unit which generates the virtual image based on position and orientation information of the image display apparatus, and a communication unit which communicates with the image display apparatus, including transmission of the virtual image to the image display apparatus; and

a plurality of wireless access points for communication between the image display apparatus and the image processing apparatus,

wherein the image processing apparatus further comprises;

a determination unit that determines, based on a position of the image display apparatus, whether switching from a first wireless access point used for communication with the image display apparatus to a second wireless access point is necessary; and

a switching instruction unit to, if the determination unit determines that the switching is necessary, transmit a switching instruction to the first wireless access point and the second wireless access point, and

the image display apparatus further comprising;

a communication control unit that, upon receiving the communication switching instruction from the first wireless access point used for communication with the image processing apparatus, disconnects communication with the first wireless access point and transmits, to the second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction; and

a display control unit that displays, on the display unit, the composite image composited by the composition unit using the captured image and the virtual image received from the image processing apparatus immediately before the switching and stored in the storage unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

5. The image display system according to claim 4, wherein the image display apparatus further comprises an image superimposition display unit which superimposes a message image on a display image of the display unit,

wherein the display control unit causes the image superimposition display unit to superimpose the message image on the display image of the display unit.

6. The image display system according to claim 5, wherein one of the image display apparatus and the image processing apparatus further comprises an input unit which instructs switching selection of superimposition of the message image.

7. The image display system according to claim 4, wherein correction processing of the virtual image stored in the storage unit is executed based on the position and orientation information of the image display apparatus upon composition by the composition unit.

8. The image display system according to claim 7, wherein the correction processing is processing of translating a position of the virtual image.

9. The image display system according to claim 7, wherein the correction processing is one of an enlargement process and a reduction process of the virtual image.

10. The image display system according to claim 4, wherein the composition unit executes one of image composition by chromakey processing and semitransparent image composition using an α

channel.

11. An image display apparatus including an image capturing unit which captures an image of physical space, and a display unit which receives, from an image processing apparatus, a composite image of a virtual image and the captured image captured by the image capturing unit, which is composited by the image processing apparatus, and displays the composite image, the image display apparatus comprising:

a wireless communication unit that communicates with the image processing apparatus via one of a plurality of wireless access points;

a communication control unit that, upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnects communication with the first wireless access point and transmits, to a second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction; and

a display control unit that displays, on the display unit, the captured image continuously acquired from the image capturing unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

12. An image display apparatus including an image capturing unit which captures an image of physical space, a display unit which receives, from an image processing apparatus, a composite image of a virtual image and the captured image captured by the image capturing unit, which is composited by the image processing apparatus, and displays the composite image, and an image superimposition display unit which superimposes a message image on a display image of the display unit, comprising:

a wireless communication unit that communicates with the image processing apparatus via one of a plurality of wireless access points;

a communication control unit that, upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnects communication with the first wireless access point and transmits, to a second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction; and

a display control unit that causes the image superimposition display unit to superimpose the message image on the display image of the display unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

13. An image display apparatus including an image capturing unit which captures an image of physical space, a storage unit which stores a virtual image, a composition unit which composites the captured image captured by the image capturing unit and the virtual image stored in the storage unit, and a display unit which displays a composite image composited by the composition unit, the image display apparatus comprising:

a wireless communication unit that communicates with an image processing apparatus via one of a plurality of wireless access points;

a communication control unit that, upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnects communication with the first wireless access point and transmits, to a second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction; and

a display control unit that displays, on the display unit, the composite image composited by the composition unit using the captured image and the virtual image received from the image processing apparatus immediately before the switching and stored in the storage unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

14. A method of controlling an image display apparatus including an image capturing unit which captures an image of physical space, a display unit which receives, from an image processing apparatus, a composite image of a virtual image and the captured image captured by the image capturing unit, which is composited by the image processing apparatus, and displays the composite image, and a wireless communication unit which communicates with the image processing apparatus via one of a plurality of wireless access points, the method comprising:

upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnecting communication with the first wireless access point and transmitting, to a second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction; and

displaying, on the display unit, the captured image continuously acquired from the image capturing unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

15. A method of controlling an image display apparatus including an image capturing unit which captures an image of physical space, a display unit which receives, from an image processing apparatus, a composite image of a virtual image and the captured image captured by the image capturing unit, which is composited by the image processing apparatus, and displays the composite image, an image superimposition display unit which superimposes a message image on a display image of the display unit, and a wireless communication unit which communicates with the image processing apparatus via one of a plurality of wireless access points, the method comprising:

upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnecting communication with the first wireless access point and transmitting, to a second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction; and

causing the image superimposition display unit to superimpose the message image on the display image of the display unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

16. A method of controlling an image display apparatus including an image capturing unit which captures an image of physical space, a storage unit which stores a virtual image, a composition unit which composites the captured image captured by the image capturing unit and the virtual image stored in the storage unit, a display unit which displays a composite image composited by the composition unit, and a wireless communication unit which communicates with an image processing apparatus via one of a plurality of wireless access points, the comprising:

upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnecting communication with the first wireless access point and transmitting, to a second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction; and

displaying, on the display unit, the composite image composited by the composition unit using the captured image and the virtual image received from the image processing apparatus immediately before the switching and stored in the storage unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

17. A computer-readable medium having stored thereon a computer program stored to cause a computer to execute control of an image display apparatus including an image capturing unit which captures an image of physical space, a display unit which receives, from an image processing apparatus, a composite image of a virtual image and the captured image captured by the image capturing unit, which is composited by the image processing apparatus, and displays the composite image, and a wireless communication unit which communicates with the image processing apparatus via one of a plurality of wireless access points, by causing the computer to execute:

a communication control step of, upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnecting communication with the first wireless access point and transmitting, to a second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction; and

a display control step of displaying, on the display unit, the captured image continuously acquired from the image capturing unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

18. A computer-readable medium having stored thereon a computer program to cause a computer to execute control of an image display apparatus including an image capturing unit which captures an image of physical space, a display unit which receives, from an image processing apparatus, a composite image of a virtual image and the captured image captured by the image capturing unit, which is composited by the image processing apparatus, and displays the composite image, an image superimposition display unit which superimposes a message image on a display image of the display unit, and a wireless communication unit which communicates with the image processing apparatus via one of a plurality of wireless access points, by causing the computer to execute:

a communication control step of, upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnecting communication with the first wireless access point and transmitting, to a second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction; and

a display control step of causing the image superimposition display unit to superimpose the message image on the display image of the display unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

19. A computer-readable medium having stored thereon a computer program to cause a computer to execute control of an image display apparatus including an image capturing unit which captures an image of physical space, a storage unit which stores a virtual image, a composition unit which composites the captured image captured by the image capturing unit and the virtual image stored in the storage unit, a display unit which displays a composite image composited by the composition unit, and a wireless communication unit which communicates with an image processing apparatus via one of a plurality of wireless access points, by causing the computer to execute:

a communication control step of, upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnecting communication with the first wireless access point and transmitting, to a second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction; and

a display control step of displaying, on the display unit, the composite image composited by the composition unit using the captured image and the virtual image received from the image processing apparatus immediately before the switching and stored in the storage unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.