Image display system, image display apparatus, and control method thereof

• US 9,721,539 B2
• Filed: 07/19/2016
• Issued: 08/01/2017
• Est. Priority Date: 10/26/2007
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Tsujimoto, Takuya
• Primary Examiner(s): CRADDOCK, ROBERT J
• Application Number:
  US15/213,881
• Publication Number:
  US 20160329032A1
• Time in Patent Office: 378 Days
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04W 36/32 by location or mobility data, e.g. speed data
• CPC Class Codes: H04W 36/322 by location data
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management

Abstract

Upon receiving a communication switching instruction from a first wireless access point used for communication with an image processing apparatus, an image display apparatus disconnects communication with the first wireless access point. Simultaneously, the image display apparatus transmits, to a second wireless access point, a link request to establish communication with the second wireless access point of a new communication destination included in the switching instruction. The image display apparatus displays, on a display unit, a captured image continuously acquired from am image capturing unit until switching from the first wireless access point to the second wireless access point finishes as communication destination switching.

Claims

1. A system comprising:

a display apparatus comprising;

a first processor; and

a first memory having instructions stored thereon, which, when executed by the first processor, cause the display apparatus to;

capture an image of physical space; and

transmit the captured image of physical space; and

an image processing apparatus which communicates with the display apparatus, the image processing apparatus comprising;

a second processor; and

a second memory storing instructions that, when executed by the second processor, cause the image processing apparatus to;

receive the captured image of physical space;

generate a virtual image based on position and orientation information of the display apparatus;

generate a synthesized image based on the captured image of physical space and the generated virtual image; and

transmit the generated synthesized image to the display apparatus,

wherein the first memory has further instructions stored thereon which, when executed by the first processor, cause the display apparatus to;

receive the generated synthesized image; and

display the generated synthesized image in a first display mode and the captured image of physical space in a second display mode, and

wherein the display apparatus uses the second display mode when the display apparatus cannot communicate with the image processing apparatus.

2. The system according to claim 1, wherein the image processing apparatus communicates wirelessly with the display apparatus.

3. The system according to claim 1, wherein the second display mode is used to further display the captured image of physical space while at least a part of the synthesized image is not received from the image processing apparatus.

4. The system according to claim 1, wherein a message superimposed on the captured image of physical space is displayed in the second display mode.

5. The system according to claim 1, wherein the display apparatus is used while mounted on a head of a user.

6. A system comprising:

a display apparatus comprising;

a first processor; and

a first memory having instructions stored thereon which, when executed by the first processor, cause the display apparatus to;

capture an image of physical space; and

transmit the captured image of physical space; and

an image processing apparatus which communicates with the display apparatus, the image processing apparatus comprising;

a second processor; and

a second memory having instructions stored thereon, which, when executed by the second processor, cause the image processing apparatus to;

receive the captured image of physical space;

generate a virtual image based on position and orientation information of the display apparatus; and

transmit the generated virtual image to the display apparatus,

wherein the first memory has further instructions stored thereon, which, when executed by the first processor, cause the display apparatus to;

receive the generated virtual image;

generate a synthesized image based on the captured image of physical space and the generated virtual image; and

display the generated synthesized image in a first display mode and the captured image of physical space in a second display mode, and

wherein the display apparatus uses the second display mode when the display apparatus cannot communicate with the image processing apparatus.

7. The system according to claim 6, wherein the image processing apparatus communicates wirelessly with the display apparatus.

8. The system according to claim 6, wherein the second display mode is used to further display the captured image of physical space while at least a part of the virtual image is not received from the image processing apparatus.

9. The system according to claim 6, wherein a message superimposed on the captured image of physical space is displayed in the second display mode.

10. The system according to claim 6, wherein the display apparatus is used while mounted on a head of a user.

11. A display apparatus which communicates with an image processing apparatus, comprising:

a first processor; and

a first memory having instructions stored thereon, which, when executed by the first processor, cause the display apparatus to;

capture an image of physical space; and

transmit the captured image of physical space,

wherein the image processing apparatus comprises;

a second processor; and

a second memory having instructions stored thereon, which, when executed by the second processor, cause the image processing apparatus to;

receive the captured image of physical space;

generate a virtual image based on position and orientation information of the display apparatus;

generate a synthesized image based on the captured image of physical space and the generated virtual image; and

transmit the generated synthesized image to the display apparatus,

wherein the first memory has further instructions, which, when executed by the first processor, cause the display apparatus to;

receive the generated synthesized image; and

display the generated synthesized image in a first display mode and the captured image of physical space in a second display mode, and

wherein the display apparatus uses the second display mode when the display apparatus cannot communicate with the image processing apparatus.

12. The display apparatus according to claim 11, wherein the display apparatus communicates wirelessly with the image processing apparatus.

13. The display apparatus according to claim 11, wherein the second display mode is used to further display the captured image of physical space while at least a part of the generated synthesized image is not received from the image processing apparatus.

14. The display apparatus according to claim 11, wherein a message superimposed on the captured image of physical space is displayed in the second display mode.

15. The display apparatus according to claim 11, wherein the display apparatus is used while mounted on a head of a user.

16. A display apparatus which communicates with an image processing apparatus, comprising:

a first processor; and

a first memory having instructions stored thereon, which, when executed by the first processor, cause the display apparatus to;

capture an image of physical space; and

transmit the captured image of physical space,

wherein the image processing apparatus comprises;

a second processor; and

a second memory having instructions stored thereon, which, when executed by the second processor, cause the image processing apparatus to;

receive the captured image of physical space;

generate a virtual image based on position and orientation information of the display apparatus; and

transmit the generated virtual image to the display apparatus,

wherein the first memory has further instructions stored thereon, which, when executed by the first processor, cause the display apparatus to;

receive the generated virtual image;

generate a synthesized image based on the captured image of physical space and the generated virtual image; and

display the generated synthesized image in a first display mode and the captured image of physical space in a second display mode, and

wherein the display apparatus uses the second display mode when the display apparatus cannot communicate with the image processing apparatus.

17. The display apparatus according to claim 16, wherein the display apparatus communicates wirelessly with the image processing apparatus.

18. The display apparatus according to claim 16, wherein the second display mode is further used to display the captured image of physical space while at least a part of the generated virtual image is not received from the image processing apparatus.

19. The display apparatus according to claim 16, wherein a message superimposed on the captured image of physical space is displayed in the second display mode.

20. The display apparatus according to claim 16, wherein the display apparatus is used while mounted on a head of a user.

21. A control method of a system comprising a display apparatus and an image processing apparatus which communicates with the display apparatus, comprising:

at the display apparatus;

capturing an image of physical space; and

transmitting the captured image of physical space;

at the image processing apparatus;

receiving the captured image of physical space;

generating a virtual image based on position and orientation information of the display apparatus;

generating a synthesized image based on the image of physical space and the generated virtual image; and

transmitting the generated synthesized image to the display apparatus; and

at the display apparatus;

receiving the generated synthesized image;

displaying the generated synthesized image in a first display mode and the captured image of physical space in a second display mode; and

using the second display mode when the display apparatus cannot communicate with the image processing apparatus.

22. A control method of a system comprising a display apparatus and an image processing apparatus which communicates with the display apparatus, comprising:

at the display apparatus;

capturing an image of physical space; and

transmitting the captured image of physical space;

at the image processing apparatus;

receiving the captured image of physical space;

generating a virtual image based on position and orientation information of the display apparatus; and

transmitting the generated virtual image to the display apparatus; and

at the display apparatus;

receiving the generated virtual image;

generating a synthesized image based on the captured image of physical space and the generated virtual image;

displaying the generated synthesized image in a first display mode and the captured image of physical space in a second display mode; and

using the second display mode when the display apparatus cannot communicate with the image processing apparatus.