Interactive module applied in 3D interactive system and method

• US 20110187638A1
• Filed: 05/21/2010
• Published: 08/04/2011
• Est. Priority Date: 02/01/2010
• Status: Abandoned Application

• Current Assignee: PixArt Imaging Incorporated
• Original Assignee: PixArt Imaging Incorporated
• Inventors: Chao, Tzu-Yi
• Application Number:
  US12/784,512
• Publication Number:
  US 20110187638A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

An interactive module applied in a 3D interactive system calibrates a location of an interactive component or calibrates a location and an interactive condition of a virtual object in a 3D image, according to a location of a user. In this way, even the location of the user changes so that the location of the virtual object seen by the user changes as well, the 3D interactive system still can correctly decide an interactive result according to the corrected location of the interactive component, or according to the corrected location and corrected interactive condition of the virtual object.

Claims

1. An interactive module applied in a 3D interactive system, the 3D interactive system having a 3D display system, the 3D display system being utilized for providing a 3D image, the 3D image having a virtual object, the virtual object having a virtual coordinate and an interaction determining condition, the interactive module comprising:

a positioning module, for detecting a location of a user in a scene so as to generate a 3D reference coordinate;

an interactive component;

an interactive component positioning module, for detecting a location of the interactive component so as to generate a 3D interactive coordinate; and

an interaction determining circuit, for converting the virtual coordinate into a corrected virtual coordinate according to the 3D reference coordinate, and deciding an interactive result between the interactive component and the 3D image according to the 3D interactive coordinate, the corrected virtual coordinate, and the interaction determining condition.

2. The interactive module of claim 1, wherein the interaction determining circuit converts the interaction determining condition into a corrected interaction determining condition according to the 3D reference coordinate;

the interaction determining circuit decides the interactive result according the 3D interactive coordinate, the corrected virtual coordinate, and the corrected interaction determining condition;

the interaction determining circuit calculates a threshold surface according to a interactive threshold distance and the virtual coordinate;

the interaction determining circuit converts the threshold surface into a corrected threshold surface according to the 3D reference coordinate;

the corrected interaction determining condition indicates that when the 3D interactive coordinate is within a region covered by the corrected threshold surface, the interactive result represents contact.

3. The interactive module of claim 1, wherein the positioning module is an eye positioning module;

the eye positioning module is utilized for detecting locations of user'"'"'s eyes in the scene so as to generate a 3D eye coordinate as the 3D reference coordinate;

wherein the 3D display system comprises a display screen and an assistant glass;

the display screen is utilized for providing a left image and a right image;

the assistant glass is utilized for helping the user'"'"'s eyes to receive the left image and the right image respectively so that the user obtains the 3D image;

wherein the eye positioning module comprises;

a first image sensor, for sensing the scene so as to generate a first 2D sensing image;

a second image sensor, for sensing the scene so as to generate a second 2D sensing image;

an eye positioning circuit, comprising;

a glass detecting circuit, for detecting the assistant glass in the first 2D sensing image so as to obtain a first 2D glass coordinate and a first glass slope, and detecting the assistant glass in the second 2D sensing image so as to obtain a second 2D glass coordinate and a second glass slope; and

a glass coordinate converting circuit, for calculating a first 2D eye coordinate and a second 2D eye coordinate according to the first 2D glass coordinate, the first glass slope, the second 2D glass coordinate, the second glass slope, and a predetermined eye spacing; and

a 3D coordinate converting circuit, for calculating the 3D eye coordinate according to the first 2D eye coordinate, the second 2D eye coordinate, a first sensing location of the first image sensor, and a second sensing location of the second image sensor.

4. The interactive module of claim 3, wherein the eye positioning circuit further comprises a tilt detector;

the tilt detector is disposed on the assistant glass;

the tilt detector is utilized for generating a tilt information according to a tilt angle of the assistant glass;

the glass coordinate converting circuit calculates the first 2D eye coordinate and the second 2D eye coordinate according to the tilt information, the first 2D glass coordinate, the first glass slope, the second 2D glass coordinate, the second glass slope, and the predetermined eye spacing.

5. The interactive module of claim 3, wherein the eye positioning circuit further comprises:

a first infra-red light emitting component, for emitting a first detecting light; and

an infra-red light sensing circuit, for generating a 2D infra-red light coordinate and an infra-red light slope;

wherein the glass coordinate converting circuit calculates the first 2D eye coordinate and the second 2D eye coordinate according to the 2D infra-red light coordinate, the infra-red light slope, the first 2D glass coordinate, the first glass slope, the second 2D glass coordinate, the second glass slope, and the predetermined eye spacing.

6. The interactive module of claim 1, wherein the positioning module is an eye positioning module;

the eye positioning module is utilized for detecting locations of user'"'"'s eyes in the scene so as to generate a 3D eye coordinate as the 3D reference coordinate;

wherein the 3D display system comprises a display screen and an assistant glass;

the display screen is utilized for providing a left image and a right image;

the assistant glass is utilized for helping the user'"'"'s eyes to receive the left image and the right image respectively so that the user obtains the 3D image;

wherein the eye positioning module comprises;

a 3D scene sensor, comprising;

a third image sensor, for sensing the scene so as to generate a third 2D sensing image;

an infra-red light emitting component, for emitting a detecting light to the scene so that the scene generates a reflecting light; and

a light-sensing distance-measuring device, for sensing the reflecting light so as to generate a distance information;

wherein the distance information has data of distance between each point of the third 2D sensing image and the 3D scene sensor; and

an eye coordinate generating circuit, comprising;

a glass detecting circuit, for detecting the assistant glass in the third 2D sensing image so as to obtain a third 2D glass coordinate and a third glass slope; and

a glass coordinate converting circuit, for calculating the 3D eye coordinate according to the third 2D glass coordinate, the third glass slope, a predetermined eye spacing, and the distance information.

7. The interactive module of claim 1, wherein the positioning module is an eye positioning module;

the eye positioning module is utilized for detecting locations of user'"'"'s eyes in the scene so as to generate a 3D eye coordinate as the 3D reference coordinate;

wherein the eye positioning module comprises;

a 3D scene sensor, comprising;

a third image sensor, for sensing the scene so as to generate a third 2D sensing image;

an infra-red light emitting component, for emitting a detecting light to the scene so that the scene generates a reflecting light; and

a light-sensing distance-measuring device, for sensing the reflecting light so as to generate a distance information;

wherein the distance information has the data of distance between each point of the third 2D sensing image and the 3D scene sensor; and

an eye coordinate generating circuit, comprising;

an eye detecting circuit, for detecting the user'"'"'s eyes in the third 2D sensing image so as to obtain a third 2D eye coordinate; and

a 3D coordinate converting circuit, for calculating the 3D eye coordinate according to the third 2D eye coordinate, the distance information, a distance-measuring location of the light-sensing distance-measuring device, and a third sensing location of the third image sensor.

8. An interactive module applied in a 3D interactive system, the 3D interactive system having a 3D display system, the 3D display system being utilized for providing a 3D image, the 3D image having a virtual object, the virtual object having a virtual coordinate and an interaction determining condition, the interactive module comprising:

a positioning module, for detecting a location of a user in a scene so as to generate a 3D reference coordinate;

an interactive component;

an interactive component positioning module, for detecting a location of the interactive component so as to generate a 3D interactive coordinate; and

an interaction determining circuit, for converting the 3D interactive coordinate into a corrected 3D interactive coordinate according to the 3D reference coordinate, and deciding an interactive result between the interactive component and the 3D image according to the corrected 3D interactive coordinate, the virtual coordinate, and the interaction determining condition.

9. The interactive module of claim 8, wherein the positioning module is an eye positioning module;

the eye positioning module is utilized for detecting locations of user'"'"'s eyes in the scene so as to generate a 3D eye coordinate as the 3D reference coordinate;

the interaction determining circuit obtains a 3D left interactive projected coordinate and a 3D right interactive projected coordinate according to the 3D eye coordinate and the 3D interactive coordinate;

the interaction determining circuit determines a left reference straight line according to the 3D left interactive projected coordinate and a predetermined left eye coordinate, and determines a right reference straight line according to the 3D right interactive projected coordinate and a predetermined right eye coordinate;

the interaction determining circuit obtains the corrected 3D interactive coordinate according to the left reference straight line and the right reference straight line.

10. The interactive module of claim 9, wherein when the left reference straight line and the right reference straight line cross at a cross point, the interaction determining circuit obtains the corrected 3D interactive coordinate according to a coordinate of the cross point;

when the left reference straight line and the right reference do not cross, the interaction determining circuit obtains a reference middle point having a minimal sum of distance to the left reference straight line and to the right reference straight line according to the left reference straight line and the right reference straight line;

a distance between the reference middle point and the left reference straight line equals to a distance between the reference middle point and the right reference straight line;

the interaction determining circuit obtains the corrected 3D interactive coordinate according to a coordinate of the reference middle point.

11. The interactive module of claim 9, wherein the interaction determining circuit obtains a center point according to the left reference straight light and the right reference straight line;

the interaction determining circuit determines a search range according to the center point;

M search points exist in the search range;

the interaction determining circuit determines M points in a coordinate system of the 3D eye coordinate corresponding to the M search points according to the predetermined eye coordinate, the M search points, and the 3D eye coordinate;

the interaction determine circuit determines M error distances corresponding to the M points according to locations of the M points and the 3D interactive coordinate, respectively;

the interaction determining circuit determines the corrected 3D interactive coordinate according to a K^th point of the M points having a minimal error distance;

M and K are positive integers, and K≦

M;

wherein the interaction determining circuit determines a left search projected coordinate and a right search projected coordinate according to a K^th search point of the M search points and the predetermined eye coordinate;

the interaction determining circuit obtains the K^th point of the M points corresponding to the K^th search point of the M search points according to the left search projected coordinate, the right search projected coordinate, and the 3D eye coordinate.

12. The interactive module of claim 8, wherein the positioning module is an eye positioning module;

the eye positioning module is utilized for detecting locations of user'"'"'s eyes in the scene so as to generate a 3D eye coordinate as the 3D reference coordinate;

Wherein M search points exist in a coordinate system of the predetermined eye coordinate;

the interaction determining circuit determines M points in a coordinate system of the 3D eye coordinate corresponding to the M search points according to the predetermined eye coordinate, the M search points, and the 3D eye coordinate;

the interaction determine circuit determines M error distances corresponding to the M points according to locations of the M points and the 3D interactive coordinate, respectively;

the interaction determining circuit determines the corrected 3D interactive coordinate according to a K^th point of the M points having a minimal error distance;

M and K are positive integers, and K≦

M;

wherein the interaction determining circuit determines a left search projected coordinate and a right search projected coordinate according to a K^th search point of the M search points and the predetermined eye coordinate;

the interaction determining circuit obtains the K^th point of the M points corresponding to the K^th search point of the M search points according to the left search projected coordinate, the right search projected coordinate, and the 3D eye coordinate.

13. The interactive module of claim 8, wherein the positioning module is an eye positioning module;

the eye positioning module is utilized for detecting locations of user'"'"'s eyes in the scene so as to generate a 3D eye coordinate as the 3D reference coordinate;

wherein the 3D display system comprises a display screen and an assistant glass;

the display screen is utilized for providing a left image and a right image;

the assistant glass is utilized for helping the user'"'"'s eyes to receive the left image and the right image respectively so that the user obtains the 3D image;

wherein the eye positioning module comprises;

a first image sensor, for sensing the scene so as to generate a first 2D sensing image;

a second image sensor, for sensing the scene so as to generate a second 2D sensing image;

an eye positioning circuit, comprising;

a glass detecting circuit, for detecting the assistant glass in the first 2D sensing image so as to obtain a first 2D glass coordinate and a first glass slope, and detecting the assistant glass in the second 2D sensing image so as to obtain a second 2D glass coordinate and a second glass slope; and

a glass coordinate converting circuit, for calculating a first 2D eye coordinate and a second 2D eye coordinate according to the first 2D glass coordinate, the first glass slope, the second 2D glass coordinate, the second glass slope, and a predetermined eye spacing; and

a 3D coordinate converting circuit, for calculating the 3D eye coordinate according to the first 2D eye coordinate, the second 2D eye coordinate, a first sensing location of the first image sensor, and a second sensing location of the second image sensor.

14. The interactive module of claim 13, wherein the eye positioning circuit further comprises a tilt detector;

the tilt detector is disposed on the assistant glass;

the tilt detector is utilized for generating a tilt information according to a tilt angle of the assistant glass;

the glass coordinate converting circuit calculates the first 2D eye coordinate and the second 2D eye coordinate according to the tilt information, the first 2D glass coordinate, the first glass slope, the second 2D glass coordinate, the second glass slope, and the predetermined eye spacing.

15. The interactive module of claim 13, wherein the eye positioning circuit further comprises:

a first infra-red light emitting component, for emitting a first detecting light; and

an infra-red light sensing circuit, for generating a 2D infra-red light coordinate and an infra-red light slope;

wherein the glass coordinate converting circuit calculates the first 2D eye coordinate and the second 2D eye coordinate according to the 2D infra-red light coordinate, the infra-red light slope, the first 2D glass coordinate, the first glass slope, the second 2D glass coordinate, the second glass slope, and the predetermined eye spacing.

16. The interactive module of claim 8, wherein the positioning module is an eye positioning module;

the eye positioning module is utilized for detecting locations of eyes of a user in the scene so as to generate a 3D eye coordinate as the 3D reference coordinate;

wherein the 3D display system comprises a display screen and an assistant glass;

the display screen is utilized for providing a left image and a right image;

the assistant glass is utilized for helping the user'"'"'s eyes to receive the left image and the right image respectively so that the user obtains the 3D image;

wherein the eye positioning module comprises;

a 3D scene sensor, comprising;

a third image sensor, for sensing the scene so as to generate a third 2D sensing image;

an infra-red light emitting component, for emitting a detecting light to the scene so that the scene generates a reflecting light; and

a light-sensing distance-measuring device, for sensing the reflecting light so as to generate a distance information;

−

wherein the distance information has the data of distance between each point of the third 2D sensing image and the 3D scene sensor; and

an eye coordinate generating circuit, comprising;

a glass detecting circuit, for detecting the assistant glass in the third 2D sensing image so as to obtain a third 2D glass coordinate and a third glass slope; and

a glass coordinate converting circuit, for calculating the 3D eye coordinate according to the third 2D glass coordinate, the third glass slope, a predetermined eye spacing, and the distance information.

17. The interactive module of claim 8, wherein the positioning module is an eye positioning module;

the eye positioning module is utilized for detecting locations of eyes of a user in the scene so as to generate a 3D eye coordinate as the 3D reference coordinate;

wherein the eye positioning module comprises;

a 3D scene sensor, comprising;

a third image sensor, for sensing the scene so as to generate a third 2D sensing image;

an infra-red light emitting component, for emitting a detecting light to the scene so that the scene generates a reflecting light; and

a light-sensing distance-measuring device, for sensing the reflecting light so as to generate a distance information;

wherein the distance information has the data of distance between each point of the third 2D sensing image and the 3D scene sensor; and

an eye coordinate generating circuit, comprising;

an eye detecting circuit, for detecting the user'"'"'s eyes in the third 2D sensing image so as to obtain a third 2D eye coordinate; and

a 3D coordinate converting circuit, for calculating the 3D eye coordinate according to the third 2D eye coordinate, the distance information, a distance-measuring location of the light-sensing distance-measuring device, and a third sensing location of the third image sensor.

18. A method of deciding an interactive result of a 3D interactive system, the 3D interactive system having a 3D display system and an interactive component, the 3D display system being utilized for providing a 3D image, the 3D image having a virtual object, the virtual object having a virtual coordinate and an interaction determining condition, the method comprising:

detecting a location of a user in a scene so as to generate a 3D reference coordinate;

detecting a location of the interactive component so as to generate a 3D interactive coordinate; and

deciding the interactive result between the interactive component and the 3D image according to the 3D reference coordinate, the 3D interactive coordinate, the virtual coordinate, and the interaction determining condition.

19. The method of claim 18, wherein detecting the location of the user in the scene so as to generate the 3D reference coordinate comprises detecting locations of user'"'"'s eyes in the scene so as to generate a 3D eye coordinate as the 3D reference coordinate;

wherein deciding the interactive result between the interactive component and the 3D image according to the 3D reference coordinate, the 3D interactive coordinate the virtual coordinate, and the interaction determining condition comprises;

converting the virtual coordinate into a corrected virtual coordinate according to the 3D eye coordinate; and

deciding the interactive result according to the 3D interactive coordinate, the corrected virtual coordinate, and the interaction determining condition.

20. The method of claim 18, wherein detecting the location of the user in the scene so as to generate the 3D reference coordinate comprises detecting locations of user'"'"'s eyes in the scene so as to generate a 3D eye coordinate as the 3D reference coordinate;

wherein deciding the interactive result between the interactive component and the 3D image according to the 3D reference coordinate, the 3D interactive coordinate the virtual coordinate, and the interaction determining condition comprises;

converting the virtual coordinate into a corrected virtual coordinate according to the 3D eye coordinate;

converting the interaction determining condition into a corrected interaction determining condition; and

deciding the interactive result according to the 3D interactive coordinate, the corrected virtual coordinate, and the corrected interaction determining condition;

wherein converting the interaction determining condition into the corrected interaction determining condition comprises;

calculating a threshold surface according to an interactive threshold distance and the virtual coordinate; and

converting the threshold surface into a corrected threshold surface according to the 3D eye coordinate;

wherein the corrected interaction determining condition indicates that when the 3D interactive coordinate is within a region covered by the corrected threshold surface, the interactive result represents contact.

21. The method of claim 18, wherein detecting the location of the user in the scene so as to generate the 3D reference coordinate comprises detecting locations of user'"'"'s eyes in the scene so as to generate a 3D eye coordinate as the 3D reference coordinate;

wherein deciding the interactive result between the interactive component and the 3D image according to the 3D reference coordinate, the 3D interactive coordinate the virtual coordinate, and the interaction determining condition comprises;

converting the 3D interactive coordinate into a corrected 3D interactive coordinate according to the 3D eye coordinate; and

deciding the interactive result according to the corrected 3D interactive coordinate, the virtual coordinate, and the interaction determining condition;

wherein the interaction determining condition indicates that when a distance between the corrected 3D interactive coordinate and the virtual coordinate is shorter than a interactive threshold distance, the interactive result represents contact.

22. The method of claim 21, wherein converting the 3D interactive coordinate into the corrected 3D interactive coordinate according to the 3D eye coordinate comprises:

obtaining a 3D left interactive projected coordinate and a 3D right interactive projected coordinate which the interactive component projects to the 3D display system according to the 3D eye coordinate and the 3D interactive coordinate;

determining a left reference straight line according to the 3D left interactive projected coordinate and a predetermined left eye coordinate, and determining a right reference straight line according to the 3D right interactive projected coordinate and a predetermined right eye coordinate; and

obtaining the corrected 3D interactive coordinate according to the left reference straight line and the right reference straight line.

23. The method of claim 22, wherein obtaining the corrected 3D interactive coordinate according to the left reference straight line and the right reference straight line comprises:

when the left reference straight line and the right reference straight line cross at a cross point, obtaining the corrected 3D interactive coordinate according to a coordinate of the cross point; and

when the left reference straight line and the right reference do not cross, obtaining a reference middle point having a minimal sum of distance to the left reference straight line and to the right reference straight line according to the left reference straight line and the right reference straight line, and obtaining the corrected 3D interactive coordinate according to a coordinate of the reference middle point;

wherein a distance between the reference middle point and the left reference straight line equals to a distance between the reference middle point and the right reference straight line.

24. The method of claim 22, wherein obtaining the corrected 3D interactive coordinate according to the left reference straight line and the right reference straight line comprises:

obtaining a center point according to the left reference straight line and the right reference straight line;

determining a search range according to the center point;

wherein M search points exist in the search range;

determining M points corresponding to the M search points according to the predetermined eye coordinate, the M search points, and the 3D eye coordinate;

respectively determining M error distances, which corresponds to the M points, between locations of the M points and the 3D interactive coordinate; and

determining the corrected 3D interactive coordinate according to a K^th point of the M points having a minimal error distance;

wherein M and K are positive integers, and K≦

M;

wherein determining the M points corresponding to the M search points according to the predetermined eye coordinate, the M search points, and the 3D eye coordinate comprises;

determining a left search projected coordinate and a right search projected coordinate according to a K^th search point of the M search points and the predetermined eye coordinate; and

obtaining the K^th point of the M points corresponding to the K^th search point of the M search points according to the left search projected coordinate, the right search projected coordinate, and the 3D eye coordinate.

25. The method of claim 21, wherein converting the 3D interactive coordinate into the corrected 3D interactive coordinate according to the 3D eye coordinate comprises:

In a coordinate system of the 3D eye coordinate, determining M points corresponding to the M search points according to the predetermined eye coordinate, the M search points in a coordinate system of the predetermined eye coordinate, and the 3D eye coordinate;

respectively determining M error distances, which corresponds to the M points, between locations of the M points and the 3D interactive coordinate; and

determining the corrected 3D interactive coordinate according to a K^th point of the M points having a minimal error distance;

wherein M and K are positive integers, and K≦

M;

wherein in the coordinate system of the 3D eye coordinate, determining the M points corresponding to the M search points according to the predetermined eye coordinate, the M search points in the coordinate system of the predetermined eye coordinate, and the 3D eye coordinate comprises;

determining a left search projected coordinate and a right search projected coordinate according to a K^th search point of the M search points and the predetermined eye coordinate; and

obtaining the K^th point of the M points corresponding to the K^th search point of the M search points according to the left search projected coordinate, the right search projected coordinate, and the 3D eye coordinate.