SYSTEM AND METHOD FOR USER INTERACTION

• US 20120229509A1
• Filed: 11/08/2011
• Published: 09/13/2012
• Est. Priority Date: 03/07/2011
• Status: Abandoned Application

• Current Assignee: Guangsong Liu
• Original Assignee: Guangsong Liu
• Inventors: LIU, Guangsong
• Application Number:
  US13/291,750
• Publication Number:
  US 20120229509A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A system is used for user interaction. When the system is in use, a signal source is configured to provide an image signal to a retina display unit. The retina display unit is configured to project the image signal provided by the signal source onto a user'"'"'s retina such that the user visually senses a virtual interface. The image signal is displayed on the virtual interface. A camera unit is configured to capture the user'"'"'s body motion. An identification-interaction unit is configured to determine an interactive operation command corresponding to the user'"'"'s body motion and transmit the interactive operation command to the signal source.

Claims

1. A system for user interaction, comprising a signal source, a retina display unit, a camera unit and an identification-interaction unit, wherein

the signal source is configured to provide an image signal to the retina display unit;

the retina display unit is configured to project the image signal provided by the signal source onto the user'"'"'s retina such that the user visually senses a virtual interface, and said image signal is displayed on said virtual interface;

the camera unit is configured to capture the user'"'"'s body motion;

the identification-interaction unit is configured to determine an interactive operation command corresponding to the user'"'"'s body motion and transmit said interactive operation command to the signal source.

2. The system according to claim 1, wherein the signal source is further configured to provide the image signal, corresponding to that after execution of said interactive operation command, to the retina display unit in real time.

3. The system according to claim 1, wherein the said retina display unit is a glasses-type displayer or a direct retinal projection device.

4. The system according to claim 1, wherein the said signal source is a mobile terminal, a computer or an information service platform based on cloud computing.

5. The system according to claim 1, wherein said retina display unit, camera unit and identification-interaction unit are physically integrated into a whole.

6. The system according to claim 1, wherein said identification-interaction unit and signal source are physically integrated into a whole, and said retina display unit and camera unit are physically integrated into a whole.

7. The system according to claim 1, wherein said retina display unit is configured to superimpose the virtual interface on the real environment viewing field formed by real environment light around the user.

8. The system according to claim 7, wherein

said retina display unit is further configured to display a spatial virtual pointer element corresponding to user'"'"'s hand on the virtual interface and/or the real environment viewing field;

the identification-interaction unit is further configured to locate and track position and shape information of user'"'"'s hand in real time based on image data of user'"'"'s hand motion, and feed back the position and shape information of user'"'"'s hand to the signal source; and

the signal source is further configured to output the image signal of the spatial virtual pointer element to the retina display unit in real time to achieve that a moving trajectory of the spatial virtual pointer element is in conformity with the user'"'"'s hand motion on the virtual interface, according to the position and shape information of user'"'"'s hand provided by the identification-interaction unit.

9. The system according to claim 1, wherein the system further comprises a voice acquisition unit, configured to acquire user'"'"'s voice interactive command and transmit the voice interactive command to the identification-interaction unit;

wherein

the identification-interaction unit is configured to identify user'"'"'s interactive intention command by a voice identification process to the user'"'"'s voice interactive command; and

the signal source is configured to provide the retina display unit in real time with the image signal corresponding to that after execution of the identified user'"'"'s interactive intention command, based on the user'"'"'s interactive intention command identified by the identification-interaction unit.

18. The system according to claim 2, wherein said retina display unit is configured to superimpose the virtual interface on the real environment viewing field formed by real environment light around the user.

19. The system according to claim 3, wherein said retina display unit is configured to superimpose the virtual interface on the real environment viewing field formed by real environment light around the user.

20. The system according to claim 4, wherein said retina display unit is configured to superimpose the virtual interface on the real environment viewing field formed by real environment light around the user.

10. A method for user interaction comprising:

by a signal source, providing an image signal to a retina display unit;

by the retina display unit, projecting the image signal provided by the signal source onto the user'"'"'s retina such that the user visually senses a virtual interface, and the image signal being displayed on the virtual interface;

by a camera unit, capturing the user'"'"'s body motion;

by an identification-interaction unit, determining an interactive operation command corresponding to the user'"'"'s body motion and transmitting the interactive operation command to the signal source.

11. The method according to claim 10, further comprising:

by the signal source, providing the image signal, corresponding to that after execution of the interactive operation command, to retina display unit in real time.

12. The method according to claim 10, wherein by the camera unit, capturing the user'"'"'s body motion comprises:

by the camera unit, capturing the user'"'"'s precise positioning operation and/or non-precise positioning operation on the virtual interface.

13. The method according to claim 12, wherein said precise positioning operation includes:

clicking a button on the virtual interface or choosing a specific region on the virtual interface.

14. The method according to claim 12, wherein said non-precise positioning operation includes:

hand waving from right to left, hand waving from left to right, hand waving from up to down, hand waving from down to up, or separating or converging of hands.

15. The method according to claim 12, wherein said projecting the image signal provided by the signal source onto the user'"'"'s retina such that the user visually senses a virtual interface, and the image signal being displayed on the virtual interface comprises:

superimposing said virtual interface on the real environment viewing field formed by the real environment light around the user.

16. The method according to claim 15, further comprising:

by the retina display unit, displaying a spatial virtual pointer element corresponding to user'"'"'s hand on the virtual interface and/or the real environment viewing field;

by the identification-interaction unit, locating and tracking the position and shape information of user'"'"'s hand in real time based on image data of user'"'"'s hand motion, and feeding back the position and shape information of user'"'"'s hand to the signal source, and

by the signal source, outputting the image signal of the spatial virtual pointer element to the retina display unit in real time to achieve that a moving trajectory of the spatial virtual pointer element is in conformity with the user'"'"'s hand motion on the virtual interface, according to the position and shape information of user'"'"'s hand provided by the identification-interaction unit.

17. The method according to claim 16, wherein said spatial virtual pointer element is a transparent profile line typed pointer superimposed on the user'"'"'s hand.