METHOD, SYSTEM AND COMPUTER PROGRAM PRODUCT FOR REAL-TIME TOUCHLESS INTERACTION

• US 20150077340A1
• Filed: 09/18/2013
• Published: 03/19/2015
• Est. Priority Date: 09/18/2013
• Status: Abandoned Application

• Current Assignee: Genius Toy Taiwan Co., Ltd.
• Original Assignee: Genius Toy Taiwan Co., Ltd.
• Inventors: LIN, Wen-Pin
• Application Number:
  US14/029,993
• Publication Number:
  US 20150077340A1
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour

Abstract

The real-time touchless interaction method of the present invention comprises the following steps: characteristic information of an object is acquired; the characteristic information is recognized and used to generate a 3D icon which corresponds to the object; the characteristic information of the object continues to be acquired, and the 3D icon is reconstructed in real-time based on how the object is manipulated; at the same time, the 3D icon is utilized as an pointer to interact with the smart device without requiring any physical manipulation with the touch screen. A system and a computer program product for real-time touchless interaction are disclosed herein. The present invention enables users to interact with a smart device without physically touching the screen, thereby generating a plethora of new interactive applications and greatly increasing the versatility of available operations between the user and the smart device.

Claims

1. A method of real-time touchless interaction, which is applicable to a smart device, said method comprising:

acquiring a characteristic information of an object;

recognizing said characteristic information to generate a 3D icon corresponding to said object; and

successively acquiring said characteristic information of said object, and reconstructing said 3D icon in real time according to the manipulation of said object, wherein said 3D icon is utilized to act as an pointer to interact with said smart device without physically touching said smart device.

2. The method according to claim 1, wherein the method for acquiring said characteristic information includes:

capturing an image of said object and acquiring said characteristic information according to at least one characteristic of said object.

3. The method according to claim 1, wherein said characteristic information comprises a configuration information, an electronic tag information, a patterned tag information, or the combination thereof, and wherein said configuration information is a combination of at least one characteristic of said object, wherein said characteristic may be the color, the shape, or the size of said object.

4. The method according to claim 1, wherein said characteristic information comprises said electronic tag information, the method for acquiring said characteristic information comprises a step:

using a wireless communication technology to communicate with said object to acquire said characteristic information from said object.

5. The method according to claim 1, wherein said 3D icon is a 3D image generated via interpreting said configuration information.

6. The method according to claim 1, wherein said 3D icon is a 3D mirror image which corresponds to said object.

7. The method according to claim 1, wherein at least a portion of graphic information of said 3D icons is stored in said smart device, and wherein after said characteristic information is recognized, said corresponding 3D icon is then retrieved from said graphic information based on a recognition result, and said graphic information is updatable.

8. The method according to claim 1, wherein the method for reconstructing said 3D icon comprises steps:

estimating a tilt angle of said object from said characteristic information which is successively acquired from said object; and

rebuilding the 3D visualization of said 3D icon according to said tilt angle.

9. The method according to claim 1, wherein the method for reconstructing said 3D icon includes steps:

estimating the displacement information of said object from said characteristic information which is successively acquired from said object; and

estimating a position of said 3D icon from said displacement information.

10. The method according to claim 9, wherein said displacement information comprises the displacement magnitude and the displacement direction of said object.

11. A real-time touchless interactive system comprising an object having a characteristic information;

and

a smart device, comprising;

a capture module to acquire said characteristic information of said object;

a recognition module electrically connected with said capture module to recognize said characteristic information; and

an image generation module electrically connected with said capture module and said recognition module to generate a 3D icon which corresponds to said object on the display screen of said smart device, wherein

said capture module successively capturing said characteristic information from said object to rebuild said 3D icon in real time according to the manipulation of said object, wherein said 3D icon is utilized to act as a pointer to interact with said smart device without physically touching the display screen of said smart device.

12. The real-time touchless interactive system according to claim 11, wherein said characteristic information comprises a configuration information, an electronic tag information, a patterned tag information, or the combination thereof, and wherein said configuration information is a combination of at least one characteristic of said object, wherein said characteristic may be the color, the shape, or the size of said object.

13. The real-time touchless interactive system according to claim 12, wherein said characteristic information includes said electronic tag information, and wherein said capture module includes a wireless-signal receiving module to wirelessly communicate with said object, and said recognition module includes a wireless-signal recognition module to recognize said characteristic information retrieved from said object.

14. The real-time touchless interactive system according to claim 11, wherein said capture module is an image capture module utilized to capture an image of said object;

and said recognition module is an image recognition module utilized to recognize said characteristic information retrieved from said object.

15. The real-time touchless interactive system according to claim 11, wherein said smart device comprises a smart phone, a tablet computer, notebook computer, a personal digital assistant or a smart television.

16. The real-time touchless interactive system according to claim 11, wherein said smart device comprises a storage module to store at least a portion of graphic information of said 3D icons.

17. The real-time touchless interactive system according to claim 16, wherein said graphic information is updatable.

18. The real-time touchless interactive system according to claim 11, wherein said recognition module is utilized to estimate a tilt angle of said object from said characteristic information which is successively acquired by said capture module;

and said recognition module is utilized to rebuild the 3D visualization of said 3D icon according to said tilt angle.

19. The real-time touchless interactive system according to claim 11, wherein said recognition module is utilized to estimate the displacement information of said object from said characteristic information which is successively acquired by said capture module and said recognition module is utilized to estimate a position of said 3D icon according to said displacement information.

20. The real-time touchless interactive system according to claim 19, wherein said displacement information comprises the displacement magnitude and the displacement direction of said object.

21. A computer program product, which is loaded into a smart device to execute a method of real-time touchless interaction, said method comprising:

acquiring a characteristic information of an object;

recognizing said characteristic information to generate a 3D icon corresponding to said object; and

successively acquiring said characteristic information of said object, and reconstructing said 3D icon in real time according to the manipulation of said object, wherein said 3D icon is utilized to act as an pointer to interact with said smart device without physically touching said smart device.

22. The computer program product according to claim 21, wherein the method for acquiring said characteristic information includes:

capturing an image of said object and acquiring said characteristic information according to at least one characteristic of said object.

23. The computer program product according to claim 21, wherein said characteristic information comprises a configuration information, an electronic tag information, a patterned tag information, or the combination thereof, and wherein said configuration information is a combination of at least one characteristic of said object, wherein said characteristic may be the color, the shape, or the size of said object.

24. The computer program product according to claim 21, wherein said characteristic information comprises said electronic tag information, the method for acquiring said characteristic information comprises a step:

using a wireless communication technology to communicate with said object to acquire said characteristic information from said object.

25. The computer program product according to claim 21, wherein said 3D icon is a 3D image generated via interpreting said configuration information.

26. The computer program product according to claim 21, wherein said 3D icon is a 3D mirror image which corresponds to said object.

27. The computer program product according to claim 21, wherein at least a portion of graphic information of said 3D icons is stored in said smart device, and wherein after said characteristic information is recognized, said corresponding 3D icon is then retrieved from said graphic information based on a recognition result, and said graphic information is updatable.

28. The computer program product according to claim 21, wherein the method for reconstructing said 3D icon comprises steps:

estimating a tilt angle of said object from said characteristic information which is successively acquired from said object; and

rebuilding the 3D visualization of said 3D icon according to said tilt angle.

29. The computer program product according to claim 21, wherein the method for reconstructing said 3D icon includes steps:

estimating the displacement information of said object from said characteristic information which is successively acquired from said object; and

estimating a position of said 3D icon from said displacement information.

30. The computer program product according to claim 29, wherein said displacement information comprises the displacement magnitude and the displacement direction of said object.