Imaging device and imaging method for augmented reality apparatus

• US 10,580,214 B2
• Filed: 03/23/2018
• Issued: 03/03/2020
• Est. Priority Date: 09/29/2017
• Status: Active Grant

• Current Assignee: BOE Technology Group Co., Ltd.
• Original Assignee: BOE Technology Group Co., Ltd.
• Inventors: Ma, Xitong, Wu, Naifu, Li, Dianmeng, Sun, Rui
• Primary Examiner(s): Wang, Jin Cheng
• Application Number:
  US15/934,215
• Publication Number:
  US 20190102945A1
• Time in Patent Office: 711 Days
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence

Abstract

The present disclosure provides in some embodiments an imaging device and an imaging method for an AR apparatus. The imaging method includes steps of: achieving, by the AR apparatus, to be in communication with a terminal device; receiving, by the AR apparatus, image data from the terminal device; and displaying, by the AR apparatus, the image data.

Claims

1. An imaging method for an Augmented Reality (AR) apparatus, comprising:

achieving, by the AR apparatus, communication with a terminal device;

receiving, by the AR apparatus, image data from the terminal device, while disenabling a display function of the terminal device;

determining, by the AR apparatus, whether or not the terminal device has been unlocked; and

in the case that the terminal device has been unlocked, displaying, by the AR apparatus, the image data,

wherein the step of determining, by the AR apparatus, whether or not the terminal device has been unlocked comprises;

capturing, by the AR apparatus, a first image containing a current image displayed on the terminal device;

comparing, by the AR apparatus, the first image with a pre-stored target image; and

in the case that the first image matches the target image, determining, by the AR apparatus, that the terminal device has been unlocked,

wherein the target image contains an image displayed on the terminal device during or after an unlocking operation.

2. The imaging method according to claim 1, wherein the step of displaying, by the AR apparatus, the image data comprises displaying, by the AR apparatus, the image data in accordance with a distance between the AR apparatus and the terminal device and a size of an imaging region of the terminal device, so as to display the image data at the imaging region of the terminal device, wherein the imaging region of the terminal device is located within a contour of the terminal device.

3. The imaging method according to claim 2, wherein the step of displaying, by the AR apparatus, the image data in accordance with the distance between the AR apparatus and the terminal device and the size of the imaging region of the terminal device comprises:

scaling, by the AR apparatus, the image data in accordance with a proportion of a size of an image currently configured by the AR apparatus to the size of the imaging region of the terminal device;

adjusting, by the AR apparatus, an imaging distance of the AR apparatus in accordance with the distance between the AR apparatus and the terminal device; and

displaying, by the AR apparatus, the resultant image data.

4. The imaging method according to claim 3, wherein prior to the step of scaling, by the AR apparatus, the image data, the imaging method further comprises:

acquiring, by the AR apparatus, two second images of the terminal device from different perspectives;

determining, by the AR apparatus, vertex coordinates of the imaging region on each of the two second images;

subjecting, by the AR apparatus, the vertex coordinates to treatment on the basis of a disparity principle, so as to acquire contour coordinates of the imaging region; and

acquiring, by the AR apparatus, a length and a width of the imaging region in accordance with the contour coordinates.

5. The imaging method according to claim 3, wherein the step of adjusting, by the AR apparatus, the imaging distance of the AR apparatus in accordance with the distance between the AR apparatus and the terminal device comprises subjecting, by the AR apparatus, the acquired image data to treatment so as to acquire a left-eye image and a right-eye image, and adjusting a pixel offset of the left-eye image displayed on a left-eye lens of the AR apparatus and/or a pixel offset of the right-eye image displayed on a right-eye lens of the AR apparatus in accordance with the distance between the AR apparatus and the terminal device.

6. The imaging method according to claim 3, wherein the step of adjusting, by the AR apparatus, the imaging distance of the AR apparatus in accordance with the distance between the AR apparatus and the terminal device comprises subjecting, by the AR apparatus, the acquired image data to treatment so as to acquire a left-eye image and a right-eye image, and adjusting, by the AR apparatus, a distance between each of the left-eye lens and the right-eye lens and a corresponding objective lens of the AR apparatus in accordance with the distance between the AR apparatus and the terminal device.

7. The imaging method according to claim 1, wherein the step of determining, by the AR apparatus, whether or not the terminal device has been unlocked precedes the step of achieving, by the AR apparatus, communication with the terminal device.

8. The imaging method according to claim 1, wherein the step of determining, by the AR apparatus, whether or not the terminal device has been unlocked follows the step of achieving, by the AR apparatus, communication with the terminal device and precedes the step of receiving, by the AR apparatus, the image data from the terminal device.

9. A non-transitory computer-readable storage medium storing therein an instruction, which is operated on an Augmented Reality (AR) apparatus, so as to enable the AR apparatus to implement the imaging method according to claim 1.

10. An imaging method for an Augmented Reality (AR) apparatus, comprising:

achieving, by a terminal device, communication with the AR apparatus; and

transmitting, by the terminal device, image data containing a to-be-displayed image generated during the operation of the terminal device to the AR apparatus, such that the AR apparatus displays the image data containing the to-be-displayed image, and disenabling a display function of the terminal device while the AR apparatus displays the image data containing the to-be-displayed image,

wherein the AR apparatus determines whether or not the terminal device has been unlocked, and in the case that the terminal device has been unlocked, the AR apparatus displays the image data,

wherein the AR apparatus captures a first image containing a current image displayed on the terminal device, compares the first image with a pre-stored target image; and

in the case that the first image matches the target image, determines that the terminal device has been unlocked,

wherein the target image contains an image displayed on the terminal device during or after an unlocking operation.

11. A terminal device, comprising a memory, a processor and a transceiver connected to each other via a bus, wherein the processor is configured to execute a program stored in the memory, so as to implement the imaging method according to claim 10.

12. A non-transitory computer-readable storage medium storing therein an instruction, which is operated on a terminal device, so as to enable the terminal device to implement the imaging method for the AR apparatus according to claim 10.

13. An Augmented Reality (AR) apparatus, comprising:

a communication circuit configured to be in communication with a terminal device and receive image data from the terminal device, while disenabling a display function of the terminal device; and

a display circuit configured to display the image data,

wherein the AR apparatus further comprises;

an image capturing circuit configured to capture a first image containing a current image displayed on the terminal device;

a storage circuit configured to store therein a target image containing an image displayed on the terminal device during or after an unlocking operation; and

a processing circuit configured to determine whether or not the first image matches the target image,

wherein in the case that the first image matches the target image, the processing circuit determines that the terminal device has been unlocked,

wherein the target image contains an image displayed on the terminal device during or after an unlocking operation.

14. The AR apparatus according to claim 13, wherein the processing circuit is further configured to scale the image data in accordance with a proportion of a size of an image currently configured by the AR apparatus to the size of an imaging region of the terminal device, and adjust an imaging distance of the AR apparatus in accordance with a distance between the AR apparatus and the terminal device.

15. The AR apparatus according to claim 13, wherein the image capturing circuit comprises two cameras.

16. The AR apparatus according to claim 13, wherein the image capturing circuit comprises one camera.