Apparatuses and methods for providing a 3D man-machine interface (MMI)

• US 9,632,626 B2
• Filed: 01/15/2014
• Issued: 04/25/2017
• Est. Priority Date: 01/17/2011
• Status: Active Grant

• Current Assignee: Xueshan Technologies Inc. (Wi-LAN Inc.)
• Original Assignee: MediaTek, Inc.
• Inventors: Hu, Yueh-Wei, Chen, Cheng-Che, Chiou, Ting
• Primary Examiner(s): ELAHI, SHAN E
• Application Number:
  US14/156,010
• Publication Number:
  US 20140125587A1
• Time in Patent Office: 1,196 Days
• Field of Search: 348 47
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

An electronic apparatus having at least two camera devices and a processing device. The processing device: determines a first length between an object positioned at a first time and a surface formed by the two camera devices, determines a second length between the object positioned at a second time and the surface, determines a third length between the object positioned at a third time and the surface, and determines a depth in a virtual space corresponding to the object positioned at the third time according to the first length, the second length, and the third length. In operation, the third time is later than the first time and the second time, and the third length is longer than the first length and shorter than the second length.

Claims

1. An electronic apparatus for providing a Man-Machine Interface (MMI) in a 3D virtual space, comprising:

a display device configured to display the 3D virtual space;

at least two camera devices configured to capture images of an object placed in front of the display device; and

a processing device, configured to;

perform a calibration of the 3D virtual space comprising;

determining a first length (L_far) between an object, positioned at a first time instant (t₁), and a surface formed by the two camera devices to be a farthest end of an operation area, by processing two images of the object captured by the two camera devices at the first time instant (t₁), and determining a second length (L_near) between the object, positioned at a second time instant (t₂), and the surface to be a nearest end of the operation area, by processing two images of the object captured by the two camera devices at the second time instant (t₂); and

after the calibration, provide the MMI by;

determining a third length (L_current) between the object, positioned at a third time instant (t₃), and the surface to be a position within the operation area, by processing two images of the object captured by the camera devices at the third time instant (t₃), transforming the position within the operation area into a depth in the 3D virtual space by calculating the depth according to the first length (L_far), the second length (L_near), and the third length (L_current), and providing the MMI in the 3D virtual space displayed by the display device according to the depth calculated,

wherein the third length (L_current) is longer than the first length (L_far) and shorter than the second length (L_near).

2. The electronic apparatus of claim 1, wherein the two camera devices are disposed to be at a first distance from each other, and the processing device further obtains two images of the object at each of the first time instant (t₁), the second time instant (t₂), and the third time instant (t₃) via the two camera devices.

3. The electronic apparatus of claim 2, wherein the object in one of the images captured at the first time instant (t₁), the second time instant (t₂) or the third time instant (t₃) is positioned at a second distance from the object in the other one of the images, and the processing device further determines the first length (L_far), the second length (L_near) the third length (L_current) according to the first distance and the corresponding second distance.

4. The electronic apparatus of claim 3, wherein the processing device further determines the first length (L_far), the second length (L_near), or the third length (L_current) using the following formula:

5. The electronic apparatus of claim 2, wherein the processing device further determines the first length (L_far), the second length (L_near), or the third length (L_current)according to a predetermined lookup table storing mapping information between a plurality of predetermined lengths from the surface, and a plurality of differences between the positions of an subject in two images captured by the two camera devices.

6. The electronic apparatus of claim 5, wherein the processing device further searches the lookup table for a match for the corresponding second distance and obtains the first length (L_far), the second length (L_near), or the third length (L_current) in accordance with the match.

7. The electronic apparatus of claim 1, wherein processing device further configures the display screen to display a first indication and a second indication in the 3D virtual space, informing user to place the object to point to the farthest end and the nearest end of the 3D virtual space, respectively.

8. The electronic apparatus of claim 1, wherein the processing device determines the depth in the 3D virtual space using the following formula:

9. A method for providing a Man-Machine Interface (MMI) in a 3D virtual space by an electronic apparatus comprising a display device displaying the 3D virtual space and at least two camera devices capturing images of an object placed in front of the display device, comprising:

performing a calibration of the 3D virtual space comprising;

determining a first length (L_far) between an object, positioned at a first time instant (t₁), and a surface formed by the two camera devices to be a farthest end of an operation area, by processing two images of the object captured by the two camera devices at the first time instant (t₁), determining a second length (L_near) between the object, positioned at a second time instant (t₂) and the surface to be a nearest end of the operation area, by processing two images of the object captured by the two camera devices at the second time instant (t₂); and

after the calibration, providing the MMI by;

determining a third length (L_current) between the object, positioned at a third time instant (t₃) and the surface to be a position within the operation area, by processing two images of the object captured by the camera devices at the third time instant (t₃), wherein the third length (L_current) is longer than the first length (L_far) and shorter than the second length (L_near), transforming the position within the operation area into a depth in the 3D virtual space by calculating the depth according to the first length (L_far), the second length (L_near), and the third length (L_current), and providing the MMI in the 3D virtual space displayed by the display device according to the depth calculated.

10. The method of claim 9, wherein the two camera devices are disposed to be at a first distance from each other, and each determination step for the first length(L_far), the second length (L_near), and the third length (L_current) further comprises capturing two images of the object at the first time instant (t₁) the second time instant (t₂), or the third time instant (t₃) by the two camera devices.

11. The method of claim 10, wherein the object in one of the images captured at the first instant (t₁), the second instant (t₂), or the third time instant (t₃) is positioned at a second distance from the object in the other one of the images, and each determination step for the first length (L_far), the second length (L_near), and the third length (L_current) further comprises determining the first length (L_far), the second length (L_near), or the third length (L_current) according to the first distance and the corresponding second distance.

12. The method of claim 9, wherein the determination step for the first length (L_far), the second length (L_near), or the third length (L_current) further comprises using the following formula to determine the first length (L_far), the second length (L_near), or the third length (L_current),

13. The method of claim 10, wherein the determination step for the first length (L_far), the second (L_near), or the third length (L_current) further comprises determining the first length (L_far), the second length (L_near), or the third length (L_current) according to a predetermined lookup table storing mapping information between a plurality of predetermined lengths from the surface, and a plurality of differences between the positions of an subject in two images captured by the two camera devices.

14. The method of claim 13, wherein determination step for the first length (L_far), the second length (L_near), or the third length (L_current) further comprises:

searching the lookup table for a match for the corresponding second distance; and

obtaining the first length (L_far) , the second length (L_near), or the third length (L_current) in accordance with the match.

15. The method of claim 9, wherein the determining step for the first length (L_far) further comprising displaying a first indication in the 3D virtual space, informing user to place the object to point to the farthest end of the 3D virtual space, and the determination step for the second length (L_near) further comprising displaying a second indication in the 3D virtual space, informing user to place the object to point to the nearest end of the 3D virtual space.

16. The method of claim 9, wherein the determination step for the depth further comprising determines the depth in the 3D virtual space using the following formula: