Image display system, image display apparatus, and control method thereof

• US 9,418,630 B2
• Filed: 02/11/2014
• Issued: 08/16/2016
• Est. Priority Date: 10/26/2007
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Tsujimoto, Takuya
• Primary Examiner(s): Drennan, Barry
• Assistant Examiner(s): CRADDOCK, ROBERT J
• Application Number:
  US14/177,321
• Publication Number:
  US 20140160164A1
• Time in Patent Office: 917 Days
• Field of Search: 345/629, 345/633
• US Class Current: 1/1
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04W 36/32 by location or mobility data, e.g. speed data
• CPC Class Codes: H04W 36/322 by location data
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management

Abstract

Upon receiving a communication switching instruction from a first wireless access point used for communication with an image processing apparatus, an image display apparatus disconnects communication with the first wireless access point. Simultaneously, the image display apparatus transmits, to a second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction. The image display apparatus displays, on a display unit, a captured image continuously acquired from am image capturing unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

Claims

1. A system comprising:

a display apparatus; and

an image processing apparatus which connects with the display apparatus via a first wireless access point, wherein the display apparatus comprises;

an image capturing unit configured to capture an image of physical space; and

a first transmitter configured to transmit the image of physical space to the image processing apparatus via the first wireless access point,

wherein the image processing apparatus comprises;

a first receiver configured to receive the image of physical space from the display apparatus via the first wireless access point;

a computer that functions as;

an obtaining unit configured to obtain a position and an orientation of the display apparatus;

a generation unit configured to generate a virtual image based on the obtained position and orientation of the display apparatus; and

a synthesizing unit configured to generate a synthesized image by synthesizing the image of physical space with the virtual image; and

a second transmitter configured to transmit the synthesized image to the display apparatus via the first wireless access point, and

wherein the display apparatus further comprises;

a second receiver configured to receive the synthesized image from the image processing apparatus via the first wireless access point; and

a display configured to, when the display apparatus moves in a communication area of the first wireless access point, display the synthesized image, and when the display apparatus goes out of the communication area of the first wireless access point, display the image of physical space captured by the image capturing unit.

2. The system according to claim 1, wherein the image processing apparatus further comprises a computer that functions as a determination unit configured to determine whether or not the display apparatus goes out of the communication area of the first wireless access point.

3. The system according to claim 2, wherein the determination unit determines whether or not the display apparatus goes out of the communication area of the first wireless access point based on distance between the display apparatus and the first wireless access point.

4. The system according to claim 1, wherein the display superimposes a message image with the image of physical space captured by the image capturing unit and displays the superimposed image when a link with the first wireless access point is not established.

5. The system according to claim 1, wherein the display is attached on a head of a user.

6. The system according to claim 1, wherein the obtaining unit obtains the position and orientation of the display apparatus based on the image of physical space.

7. The system according to claim 1, wherein the display apparatus further comprises a computer that functions as a measurement unit configured to measure a position and an orientation of the display apparatus,

wherein the first transmitter further transmits the position and orientation of the display apparatus measured by the measurement unit to the image processing apparatus, and

wherein the obtaining unit configured to obtain the position and orientation of the display apparatus from the position and orientation of the display apparatus measured by the measurement unit.

8. A display apparatus which connects with an image processing apparatus via a first wireless access point, comprising:

an image capturing unit configured to capture an image of physical space;

a transmitter configured to transmit the image of physical space to the image processing apparatus via the first wireless access point;

a receiver configured to receive a synthesized image, in which the image of physical space is synthesized with a virtual image from the image processing apparatus via the first wireless access point; and

a display configured to, when the display apparatus moves in a communication area of the first wireless access point, display the synthesized image, and when the display apparatus goes out of the communication area of the first wireless access point, display the image of physical space captured by the image capturing unit without displaying the synthesized image.

9. The apparatus according to claim 8, wherein the display superimposes a message image with the image of physical space captured by the image capturing unit and displays the superimposed image when a link with the first wireless access point is not established.

10. The apparatus according to claim 8, wherein the display is attached on a head of a user.

11. A method of controlling a system comprising a display apparatus and an image processing apparatus, which connects with the display apparatus via a first wireless access point, the method comprising:

capturing, by the display apparatus, an image of physical space;

transmitting, by the display apparatus, the image of physical space to the image processing apparatus via the first wireless access point;

receiving, by the image processing apparatus, the image of physical space from the display apparatus via the first wireless access point;

obtaining, by the image processing apparatus, a position and an orientation of the display apparatus;

generating, by the image processing apparatus, a virtual image based on the obtained position and orientation of the display apparatus;

generating, by the image processing apparatus, a synthesized image by synthesizing the image of physical space with the virtual image;

transmitting, by the image processing apparatus, a synthesized image to the display apparatus via the first wireless access point;

receiving, by the display apparatus, the synthesized image from the image processing apparatus via the first wireless access point; and

when the display apparatus moves in a communication area of the first wireless access point, displaying the synthesized image, and when the display apparatus goes out of the communication area of the first wireless access point, displaying the image of physical space captured by an image capturing unit.

12. A non-transitory computer-readable medium having stored thereon a computer program to cause a computer to perform a method of controlling a system comprising a display apparatus and an image processing apparatus, which connects with the display apparatus via a first wireless access point, the method comprising:

capturing, by the display apparatus an image of physical space;

transmitting, by the display apparatus, the image of physical space to the image processing apparatus via the first wireless access point;

receiving, by the image processing apparatus, the image of physical space from the display apparatus via the first wireless access point;

obtaining, by the image processing apparatus, a position and orientation of the display apparatus;

generating, by the image processing apparatus, a virtual image based on the obtained position and orientation of the display apparatus;

generating, by the image processing apparatus, a synthesized image by synthesizing the image of physical space with the virtual image;

transmitting, by the image processing apparatus, a synthesized image to the display apparatus via the first wireless access point;

receiving, by the display apparatus, the synthesized image from the image processing apparatus via the first wireless access point; and

when the display apparatus moves in a communication area of the first wireless access point, displaying the synthesized image, and when the display apparatus goes out of the communication area of the first wireless access point, displaying the image of physical space captured by an image capturing unit.

13. A method of controlling a display apparatus which connects with an image processing apparatus via a first wireless access point, comprising:

capturing an image of physical space;

transmitting the image of physical space to the image processing apparatus via the first wireless access point;

receiving a synthesized image, in which the image of physical space is synthesized with a virtual image from the image processing apparatus via the first wireless access point; and

when the display apparatus moves in a communication area of the first wireless access point displaying the synthesized image, and when the display apparatus goes out of the communication area of the first wireless access point, displaying the image of physical space captured by an image capturing unit without displaying the synthesized image.

14. A non-transitory computer-readable medium having stored thereon a computer program to cause a computer to execute control of a display apparatus which connects with an image processing apparatus via a first wireless access point, by causing the computer to execute:

capturing an image of physical space;

transmitting the image of physical space to the image processing apparatus via the first wireless access point;

receiving a synthesized image, in which the image of physical space is synthesized with a virtual image from the image processing apparatus via the first wireless access point; and

when the display apparatus moves in a communication area of the first wireless access point, displaying the synthesized image, and when the display apparatus goes out of the communication area of the first wireless access point, displaying the image of physical space captured by an image capturing unit without displaying the synthesized image.

15. A system comprising:

a display apparatus; and

an image processing apparatus which connects with the display apparatus via a first wireless access point,

wherein the display apparatus comprises;

an image capturing unit configured to capture an image of physical space; and

a first transmitter configured to transmit the image of physical space to the image processing apparatus via the first wireless access point,

wherein the image processing apparatus comprises;

a first receiver configured to receive the image of physical space from the display apparatus via the first wireless access point;

a computer that functions as;

an obtaining unit configured to obtain a position and an orientation of the display apparatus;

a generation unit configured to generate a virtual image based on the obtained position and orientation of the display apparatus; and

a synthesizing unit configured to generate a synthesized image by synthesizing the image of physical space with the virtual image; and

a second transmitter configured to transmit the synthesized image to the display apparatus via the first wireless access point, and

wherein the display apparatus further comprises;

a second receiver configured to receive the synthesized image from the image processing apparatus via the first wireless access point; and

a display configured to, when the second receiver cannot receive the synthesized image from the image processing apparatus via the first wireless access point, display the image of physical space captured by the image capturing unit.

16. The system according to claim 15, wherein the display is attached on a head of a user.

17. A display apparatus which connects with an image processing apparatus via a first wireless access point, comprising:

an image capturing unit configured to capture an image of physical space;

a transmitter configured to transmit the image of physical space to the image processing apparatus via the first wireless access point;

a receiver configured to receive a synthesized image, in which the image of physical space is synthesized with a virtual image from the image processing apparatus via the first wireless access point; and

a display configured to, when the receiver cannot receive the synthesized image from the image processing apparatus via the first wireless access point, display the image of physical space captured by the image capturing unit without displaying the synthesized image.

18. The apparatus according to claim 17, wherein the display is attached on a head of a user.

19. A method of controlling a system comprising a display apparatus and an image processing apparatus, which connects with the display apparatus via a first wireless access point, the method comprising:

capturing, by the display apparatus, an image of physical space;

transmitting, by the display apparatus, the image of physical space to the image processing apparatus via the first wireless access point;

receiving, by the image processing apparatus, the image of physical space from the display apparatus via the first wireless access point;

obtaining, by the image processing apparatus, a position and an orientation of the display apparatus;

generating, by the image processing apparatus, a virtual image based on the obtained position and orientation of the display apparatus;

generating, by the image processing apparatus, a synthesized image by synthesizing the image of physical space with the virtual image;

transmitting, by the image processing apparatus, a synthesized image to the display apparatus via the first wireless access point;

receiving, by the display apparatus, the synthesized image from the image processing apparatus via the first wireless access point; and

when the display apparatus cannot receive the synthesized image from the image processing apparatus via the first wireless access point, displaying the image of physical space captured by an image capturing unit.

20. A method of controlling a display apparatus which connects with an image processing apparatus via a first wireless access point, comprising:

capturing an image of physical space;

transmitting the image of physical space to the image processing apparatus via the first wireless access point;

receiving a synthesized image, in which the image of physical space is synthesized with a virtual image from the image processing apparatus via the first wireless access point; and

when the display apparatus cannot receive the synthesized image from the image processing apparatus via the first wireless access point, display the image of physical space captured by an image capturing unit without displaying the synthesized image.

21. A system comprising:

a display apparatus; and

an image processing apparatus which connects with the display apparatus via a first wireless access point,

wherein the display apparatus comprises;

an image capturing unit configured to capture an image of physical space; and

a first transmitter configured to transmit the image of physical space to the image processing apparatus via the first wireless access point,

wherein the image processing apparatus comprises;

a first receiver configured to receive the image of physical space from the display apparatus via the first wireless access point;

a computer that functions as;

an obtaining unit configured to obtain a position and an orientation of the display apparatus;

a generation unit configured to generate a virtual image based on the obtained position and orientation of the display apparatus; and

a synthesizing unit configured to generate a synthesized image by synthesizing the image of physical space with the virtual image; and

a second transmitter configured to transmit the synthesized image to the display apparatus via the first wireless access point, and

wherein the display apparatus further comprises;

a second receiver configured to receive the synthesized image from the image processing apparatus via the first wireless access point; and

a display configured to, when connection condition between the display apparatus and the first wireless access point is predetermined condition, display the image of physical space captured by the image capturing unit.

22. The system according to claim 21, wherein the display is attached on a head of a user.

23. A display apparatus which connects with an image processing apparatus via a first wireless access point, comprising:

an image capturing unit configured to capture an image of physical space;

a transmitter configured to transmit the image of physical space to the image processing apparatus via the first wireless access point;

a receiver configured to receive a synthesized image, in which the image of physical space is synthesized with a virtual image from the image processing apparatus via the first wireless access point; and

a display configured to, when connection condition between the display apparatus and the first wireless access point is predetermined condition, display the image of physical space captured by the image capturing unit without displaying the synthesized image.

24. The apparatus according to claim 23, wherein the display is attached on a head of a user.

25. A display apparatus which connects with an image processing apparatus via a first wireless access point, comprising:

an image capturing unit configured to capture an image of physical space;

a transmitter configured to transmit the image of physical space to the image processing apparatus via the first wireless access point;

a receiver configured to receive a synthesized image, in which the image of physical space is synthesized with a virtual image from the image processing apparatus via the first wireless access point; and

a selector configured to select a source of an image to be displayed from the receiver and the image capturing unit, wherein the synthesized image received by the transmitter is selected when the display apparatus moves in a communication area of the first wireless access point and the image of physical space captured by the image capturing units is selected when the display apparatus goes out of the communication area of the first wireless access point, wherein the selector is further configured to input the selected image to a display of the display apparatus.

26. The apparatus according to claim 25, wherein the display is attached on a head of a user.

27. A method of controlling a display apparatus which connects with an image processing apparatus via a first wireless access point, comprising:

capturing an image of physical space;

transmitting the image of physical space to the image processing apparatus via the first wireless access point;

receiving a synthesized image, in which the image of physical space is synthesized with a virtual image from the image processing apparatus via the first wireless access point;

selecting an image to be displayed from the received synthesized image and the captured image of physical space, wherein the received synthesized image is selected when the display apparatus moves in a communication area of the first wireless access point and the captured image of physical space is selected when the display apparatus goes out of the communication area of the first wireless access point; and

inputting the selected image to a display of the display apparatus.

28. A non-transitory computer-readable medium having stored thereon a computer program to cause a computer to execute control of a display apparatus which connects with an image processing apparatus via a first wireless access point, by causing the computer to execute:

capturing an image of physical space;

transmitting the image of physical space to the image processing apparatus via the first wireless access point;

receiving a synthesized image, in which the image of physical space is synthesized with a virtual image from the image processing apparatus via the first wireless access point;

selecting an image to be displayed from the received synthesized image and the captured image of physical space, wherein the received synthesized image is selected when the display apparatus moves in a communication area of the first wireless access point and the captured image of physical space is selected when the display apparatus goes out of the communication area of the first wireless access point; and

inputting the selected image to a display of the display apparatus.