Virtual image display system, including wireless connection

• US 8,681,177 B2
• Filed: 10/23/2008
• Issued: 03/25/2014
• Est. Priority Date: 10/26/2007
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Tsujimoto, Takuya
• Primary Examiner(s): Richer, Aaron M
• Assistant Examiner(s): CRADDOCK, ROBERT J
• Application Number:
  US12/257,280
• Publication Number:
  US 20090109241A1
• Time in Patent Office: 1,979 Days
• Field of Search: 455436-444, 455/456.6, 715/123, 370/321, 370/328, 345/629, 345/633
• US Class Current: 345/633
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04W 36/32 by location or mobility data, e.g. speed data
• CPC Class Codes: H04W 36/322 by location data
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management

Abstract

Upon receiving a communication switching instruction from a first wireless access point used for communication with an image processing apparatus, an image display apparatus disconnects communication with the first wireless access point. Simultaneously, the image display apparatus transmits, to a second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction. The image display apparatus displays, on a display unit, a captured image continuously acquired from am image capturing unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

Claims

1. An image display system comprising:

an image display apparatus including an image capturing unit which captures an image of physical space in real time, a wireless communication unit which communicates with an image processing apparatus, including transmission of the captured image obtained by the image capturing unit to the image processing apparatus, and a display unit which receives from the image processing apparatus a composite image of the captured image and a virtual image composited by the image processing apparatus and displays the composite image;

the image processing apparatus which communicates with the image display apparatus, the image processing apparatus including a generation unit which generates position and orientation information of the image display apparatus, a virtual image generation unit which generates the virtual image based on the position and orientation information, a composition unit which composites the captured image and the virtual image, and a communication unit which communicates with the image display apparatus, including transmission of the composite image obtained by the composition unit to the image display apparatus; and

a plurality of wireless access points for communication between the image display apparatus and the image processing apparatus,

wherein the image processing apparatus further comprises;

a determination unit configured to determine whether switching from a first wireless access point used for communication with the image display apparatus to a second wireless access point is necessary; and

a switching instruction unit configured to transmit a switching instruction to the first wireless access point and the second wireless access point if the determination unit determines that the switching is necessary, and

wherein the image display apparatus further comprises;

a communication control unit configured to disconnect communication with the first wireless access point when the image display apparatus receives the communication switching instruction from the first wireless access point and to transmit a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction to the second wireless access point; and

a display control unit configured to display the captured image acquired in real time by the image capturing unit on the display unit instead of the composite image while a user of the image display apparatus is moving until the image display apparatus finishes switching from the first wireless access point to the second wireless access point,

wherein the display control unit, when the switching has finished and a composite image corresponding to the captured image transmitted to the second wireless access point of the new communication destination has been received, ends display control of an image which is currently displayed on the display unit, and displays the received composite image on the display unit.

2. An image display system comprising:

an image display apparatus including an image capturing unit which captures an image of physical space in real time, a wireless communication unit which communicates with an image processing apparatus, including transmission of the captured image obtained by the image capturing unit to the image processing apparatus, a display unit which receives from the image processing apparatus a composite image of the captured image and a virtual image composited by the image processing apparatus and displays the composite image, and an image superimposition display unit which superimposes a message image on a display image of the display unit;

the image processing apparatus which communicates with the image display apparatus, the image processing apparatus including a generation unit which generates position and orientation information of the image display apparatus, a virtual image generation unit which generates the virtual image based on the position and orientation information, a composition unit which composites the captured image and the virtual image, and a communication unit which communicates with the image display apparatus, including transmission of the composite image obtained by the composition unit to the image display apparatus; and

a plurality of wireless access points for communication between the image display apparatus and the image processing apparatus,

wherein the image processing apparatus further comprises;

a determination unit configured to determine whether switching from a first wireless access point used for communication with the image display apparatus to a second wireless access point is necessary; and

a switching instruction unit configured to transmit a switching instruction to the first wireless access point and the second wireless access point if the determination unit determines that the switching is necessary, and

wherein the image display apparatus further comprises;

a communication control unit configured to disconnect communication with the first wireless access point and transmit a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction to the second wireless access point when the image display apparatus receives the communication switching instruction from the first wireless access point; and

a display control unit configured to superimpose the message image on the display image of the display unit so as to call a user'"'"'s attention while the user of the image display apparatus is moving until the image display apparatus finishes switching from the first wireless access point to the second wireless access point,

wherein the display control unit, when the switching has finished and a composite image corresponding to the captured image transmitted to the second wireless access point of the new communication destination has been received, ends display control of an image which is currently displayed on the display unit, and displays the received composite image on the display unit.

3. The image display system according to claim 2, wherein one of the image display apparatus and the image processing apparatus further comprises an input unit configured to instruct switching selection of superimposition of the message image.

4. An image display system comprising:

an image display apparatus including an image capturing unit which captures an image of physical space in real time, a storage unit which stores a virtual image, a composition unit which composites the captured image captured by the image capturing unit and the virtual image stored in the storage unit, a display unit which displays a composite image composited by the composition unit, and a wireless communication unit which communicates with an image processing apparatus, including reception of the virtual image from the image processing apparatus;

the image processing apparatus which communicates with the image display apparatus, the image processing apparatus including a virtual image generation unit which generates the virtual image based on position and orientation information of the image display apparatus, and a communication unit which communicates with the image display apparatus, including transmission of the virtual image to the image display apparatus; and

a plurality of wireless access points for communication between the image display apparatus and the image processing apparatus,

wherein the image processing apparatus further comprises;

a determination unit configured to determine whether switching from a first wireless access point used for communication with the image display apparatus to a second wireless access point is necessary; and

a switching instruction unit configured to transmit a switching instruction to the first wireless access point and the second wireless access point if the determination unit determines that the switching is necessary, and

wherein the image display apparatus further comprises;

a communication control unit configured to disconnect communication with the first wireless access point and transmit a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction to the second wireless access point when the image display apparatus receives the communication switching instruction from the first wireless access point; and

a display control unit configured to display the composite image composited by the composition unit using the captured image and the virtual image stored in the storage unit on the display unit while a user of the image display apparatus is moving until the image display apparatus finishes switching from the first wireless access point to the second wireless access point,

wherein the display control unit, when the switching has finished and a virtual image corresponding to the position and orientation information transmitted to the second wireless access point of the new communication destination has been received, ends display control of an image which is currently displayed on the display unit, and displays a composite image composited by the composition unit using the captured image and the received virtual image.

5. The image display system according to claim 4, wherein the image display apparatus further comprises an image superimposition display unit configured to superimpose a message image on a display image of the display unit, and

wherein the display control unit causes the image superimposition display unit to superimpose the message image on the display image of the display unit.

6. The image display system according to claim 5, wherein one of the image display apparatus and the image processing apparatus further comprises an input unit configured to instruct switching selection of superimposition of the message image.

7. The image display system according to claim 4, wherein correction processing of the virtual image stored in the storage unit is executed based on the position and orientation information of the image display apparatus upon composition by the composition unit.

8. The image display system according to claim 7, wherein the correction processing is processing of translating a position of the virtual image.

9. The image display system according to claim 7, wherein the correction processing is one of an enlargement process and a reduction process of the virtual image.

10. The image display system according to claim 4, wherein the composition unit executes one of image composition by chromakey processing and semitransparent image composition using an α

channel.

11. An image display apparatus including an image capturing unit which captures an image of physical space in real time, and a display unit which receives, from an image processing apparatus, a composite image of a virtual image and the captured image captured by the image capturing unit, which is composited by the image processing apparatus, and displays the composite image, the image display apparatus comprising:

a wireless communication unit that communicates with the image processing apparatus via one of a plurality of wireless access points;

a communication control unit configured to disconnect communication with a first wireless access point used for communication with the image processing apparatus when the image display apparatus receives a communication switching instruction from the first wireless access point and to transmit a link request to establish communication with a second wireless access point of a new communication destination included in the switching instruction to the second wireless access point; and

a display control unit configured to display the captured image acquired in real time by the image capturing unit on the display unit instead of the composite image while a user of the image display apparatus is moving until the image display apparatus finishes switching from the first wireless access point to the second wireless access point,

wherein the display control unit, when the switching has finished and a composite image corresponding to the captured image transmitted to the second wireless access point of the new communication destination has been received, ends display control of an image which is currently displayed on the display unit, and displays the received composite image on the display unit.

12. An image display apparatus including an image capturing unit which captures an image of physical space in real time, a display unit which receives from an image processing apparatus a composite image of a virtual image and the captured image captured by the image capturing unit, which is composited by the image processing apparatus, and displays the composite image, and an image superimposition display unit which superimposes a message image on a display image of the display unit, the image display apparatus comprising:

a wireless communication unit that communicates with the image processing apparatus via one of a plurality of wireless access points;

a communication control unit configured to disconnect communication with a first wireless access point used for communication with the image processing apparatus and transmit a link request to establish communication with a second wireless access point of a new communication destination included in a switching instruction to the second wireless access point when the image display apparatus receives a communication switching instruction from the first wireless access point; and

a display control unit configured to superimpose the message image on the display image of the display unit so as to call a user'"'"'s attention while the user of the image display apparatus is moving until the image display apparatus finishes switching from the first wireless access point to the second wireless access point,

wherein the display control unit, when the switching has finished and a composite image corresponding to the captured image transmitted to the second wireless access point of the new communication destination has been received, ends display control of an image which is currently displayed on the display unit, and displays the received composite image on the display unit.

13. An image display apparatus including an image capturing unit which captures an image of physical space in real time, a storage unit which stores a virtual image, a composition unit which composites the captured image captured by the image capturing unit and the virtual image stored in the storage unit, and a display unit which displays a composite image composited by the composition unit, the image display apparatus comprising:

a wireless communication unit that communicates with an image processing apparatus via one of a plurality of wireless access points;

a communication control unit configured to disconnect communication with a first wireless access point used for communication with the image processing apparatus and transmit a link request to establish communication with a second wireless access point of a new communication destination included in a switching instruction to the second wireless access point when the image display apparatus receives a communication switching instruction from the first wireless access point; and

a display control unit configured to display the composite image composited by the composition unit while a user of the image display apparatus is moving on the display unit until the image display apparatus finishes switching from the first wireless access point to the second wireless access point,

wherein the display control unit, when the switching has finished and a virtual image corresponding to position and orientation information transmitted to the second wireless access point of the new communication destination has been received, ends display control of an image which is currently displayed on the display unit, and displays a composite image composited by the composition unit using the captured image and the received virtual image.

14. A method of controlling an image display apparatus including an image capturing unit which captures an image of physical space in real time, a display unit which receives from an image processing apparatus a composite image of a virtual image and the captured image captured by the image capturing unit, that is composited by the image processing apparatus and displays the composite image, and a wireless communication unit which communicates with the image processing apparatus via one of a plurality of wireless access points, the method comprising:

upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnecting communication with the first wireless access point when the image display apparatus receives the communication switching instruction from the first wireless access point and transmitting a link request to establish communication with a second wireless access point of a new communication destination included in the switching instruction to the second wireless access point; and

displaying the captured image acquired in real time by the image capturing unit on the display unit instead of the composite image while a user of the image display apparatus is moving until the image display apparatus finishes switching from the first wireless access point to the second wireless access point,

wherein when the switching has finished and a composite image corresponding to the captured image transmitted to the second wireless access point of the new communication destination has been received, display control of an image which is currently displayed on the display unit is ended, and the received composite image is displayed on the display unit.

15. A method of controlling an image display apparatus including an image capturing unit which captures an image of physical space in real time, a display unit which receives from an image processing apparatus a composite image of a virtual image and the captured image captured by the image capturing unit, that is composited by the image processing apparatus and displays the composite image, an image superimposition display unit which superimposes a message image on a display image of the display unit, and a wireless communication unit which communicates with the image processing apparatus via one of a plurality of wireless access points, the method comprising:

upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnecting communication with the first wireless access point and transmitting a link request to establish communication with a second wireless access point of a new communication destination included in the switching instruction to the second wireless access point when the image display apparatus receives the communication switching instruction from the first wireless access point; and

superimposing the message image on the display image of the display unit so as to call a user'"'"'s attention while the user of the image display apparatus is moving until the image display apparatus finishes switching from the first wireless access point to the second wireless access point,

wherein when the switching has finished and a composite image corresponding to the captured image transmitted to the second wireless access point of the new communication destination has been received, display control of an image which is currently displayed on the display unit is ended, and the received composite image is displayed on the display unit.

16. A method of controlling an image display apparatus including an image capturing unit which captures an image of physical space in real time, a storage unit which stores a virtual image, a composition unit which composites the captured image captured by the image capturing unit and the virtual image stored in the storage unit, a display unit which displays a composite image composited by the composition unit, and a wireless communication unit which communicates with an image processing apparatus via one of a plurality of wireless access points, the method comprising:

upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnecting communication with the first wireless access point and transmitting a link request to establish communication with a second wireless access point of a new communication destination included in the switching instruction to the second wireless access point when the image display apparatus receives the communication switching instruction from the first wireless access point; and

displaying the composite image composited by the composition unit on the display unit while a user of the image display apparatus is moving until the image display apparatus finishes switching from the first wireless access point to the second wireless access point,

wherein when the switching has finished and a virtual image corresponding to position and orientation information transmitted to the second wireless access point of the new communication destination has been received, display control of an image which is currently displayed on the display unit is ended, and a composite image composited by the composition unit is displayed using the captured image and the received virtual image.

17. A non-transitory computer-readable medium having stored thereon a computer program to cause a computer to execute control of an image display apparatus including an image capturing unit which captures an image of physical space in real time, a display unit which receives from an image processing apparatus a composite image of a virtual image and the captured image captured by the image capturing unit, which is composited by the image processing apparatus, and displays the composite image, and a wireless communication unit which communicates with the image processing apparatus via one of a plurality of wireless access points, by causing the computer to execute:

a communication control step of, upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnecting communication with the first wireless access point when the image display apparatus receives the communication switching instruction from the first wireless access point and transmitting a link request to establish communication with a second wireless access point of a new communication destination included in the switching instruction to the second wireless access point; and

a display control step of displaying the captured image acquired in real time by the image capturing unit on the display unit instead of the composite image while a user of the image display apparatus is moving until the image display apparatus finishes switching from the first wireless access point to the second wireless access point,

wherein when the switching has finished and a composite image corresponding to the captured image transmitted to the second wireless access point of the new communication destination has been received, display control of an image which is currently displayed on the display unit is ended, and the received composite image is displayed on the display unit.

18. A non-transitory computer-readable medium having stored thereon a computer program to cause a computer to execute control of an image display apparatus including an image capturing unit which captures an image of physical space in real time, a display unit which receives from an image processing apparatus a composite image of a virtual image and the captured image captured by the image capturing unit, that is composited by the image processing apparatus, and displays the composite image, an image superimposition display unit which superimposes a message image on a display image of the display unit, and a wireless communication unit which communicates with the image processing apparatus via one of a plurality of wireless access points, by causing the computer to execute:

a communication control step of, upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnecting communication with the first wireless access point and transmitting a link request to establish communication with a second wireless access point of a new communication destination included in the communication switching instruction to the second wireless access point when the image display apparatus receives the communication switching instruction from the first wireless access point; and

a display control step of superimposing the message image on the display image of the display unit so as to call a user'"'"'s attention while the user of the image display apparatus is moving until the image display apparatus finishes switching from the first wireless access point to the second wireless access point,

wherein when the switching has finished and a composite image corresponding to the captured image transmitted to the second wireless access point of the new communication destination has been received, display control of an image which is currently displayed on the display unit is ended, and the received composite image is displayed on the display unit.

19. A non-transitory computer-readable medium having stored thereon a computer program to cause a computer to execute control of an image display apparatus including an image capturing unit which captures an image of physical space in real time, a storage unit which stores a virtual image, a composition unit which composites the captured image captured by the image capturing unit and the virtual image stored in the storage unit, a display unit which displays a composite image composited by the composition unit, and a wireless communication unit which communicates with an image processing apparatus via one of a plurality of wireless access points, by causing the computer to execute:

a communication control step of, upon receiving a communication switching instruction from a first wireless access point used for communication with the image processing apparatus, disconnecting communication with the first wireless access point and transmitting a link request to establish communication with a second wireless access point of a new communication destination included in the communication switching instruction to the second wireless access point when the image display apparatus receives the communication switching instruction from the first wireless access point; and

a display control step of displaying the composite image composited by the composition unit on the display unit while a user of the image display apparatus is moving until the image display apparatus finishes switching from the first wireless access point to the second wireless access point,

wherein when the switching has finished and a virtual image corresponding to position and orientation information transmitted to the second wireless access point of the new communication destination has been received, display control of an image which is currently displayed on the display unit is ended, and a composite image composited by the composition unit is displayed using the captured image and the received virtual image.

20. An image display system comprising:

an image display apparatus including an image capturing unit which captures an image of physical space in real time, a communication unit which communicates with an image processing apparatus and a display unit which receives from the image processing apparatus a composite image and displays the composite image;

the image processing apparatus including a composition unit which composites the captured image and a virtual image as the composite image, and a communication unit which communicates with the image display apparatus; and

a plurality of wireless access points for communication between the image display apparatus and the image processing apparatus,

wherein the image processing apparatus further comprises;

a determination unit configured to determine whether or not switching from a first wireless access point used for communication with the image display apparatus to a second wireless access point is necessary; and

a switching instruction unit configured to transmit a switching instruction to the first wireless access point and the second wireless access point if the determination unit determines that the switching is necessary, and

wherein the image display apparatus further comprises;

a display control unit configured to display the captured image on the display unit instead of the composite image while a user of the image display apparatus is moving until the image display apparatus finishes switching from the first wireless access point to the second wireless access point,

wherein the display control unit, when the switching has finished and a composite image corresponding to the captured image transmitted to the second wireless access point has been received, ends display control of an image which is currently displayed on the display unit, and displays the received composite image on the display unit.

21. An image display apparatus comprising:

an image capturing unit configured to capture an image of physical space in real time;

a communication unit configured to communicate with an image processing apparatus via one of a plurality of wireless access points; and

a display unit configured to display the captured image on the display unit instead of a composite image which is composited with the captured image and a virtual image while a user of the image display apparatus is moving until the image display apparatus finishes switching from a first wireless access point to a second wireless access point,

wherein the display unit, when the switching has finished and a composite image corresponding to the captured image transmitted to the second wireless access point has been received, ends display control of an image which is currently displayed on the display unit, and displays the received composite image on the display unit.

22. An image display apparatus comprising:

a communication unit configured to communicate with an image processing apparatus via one of a plurality of wireless access points;

an obtaining unit configured to obtain a virtual image from the image processing apparatus; and

a display unit configured to display the virtual image obtained by the obtaining unit,

wherein the display unit does not display the virtual image while a user of the image display apparatus is moving until the image display apparatus finishes switching from a first wireless access point to a second wireless access point, and

wherein a display control unit, when the switching has finished and the virtual image has been obtained from the image processing apparatus via the second wireless access point, ends display control of an image which is currently displayed on the display unit, and displays the obtained virtual image on the display unit.