Touch and gesture control system and touch and gesture control method

• US 10,073,529 B2
• Filed: 06/25/2015
• Issued: 09/11/2018
• Est. Priority Date: 11/14/2014
• Status: Active Grant

• Current Assignee: Coretronic Corporation
• Original Assignee: Coretronic Corporation
• Inventors: Liao, Chien-Chung, Chen, Yung-Chih
• Primary Examiner(s): Rosario, Nelson
• Application Number:
  US14/751,116
• Publication Number:
  US 20160139672A1
• Time in Patent Office: 1,174 Days
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A touch and gesture control system including a light curtain module, an invisible light lighting module, an image capturing module and a control unit is provided. The light curtain module is configured to form a touch zone. The invisible light lighting module is configured to form a gesture control zone. The touch zone and the gesture control zone are at least partially overlapped. The control unit is electrically connected to the light curtain module, the image capturing module and the invisible light lighting module. The control unit turns on the light curtain module and controls the image capturing module to capture a first set of images when executing a touch mode, and turns on the invisible light lighting module and controls the image capturing module to capture a second set of images when executing a gesture control mode. A touch and gesture control method is also provided.

Claims

1. A touch and gesture control system, comprising:

a light curtain generating module, disposed at a side of a display area, configured to generate a light curtain parallel to the display area and to form a touch zone;

an invisible light lighting module, configured to provide invisible light, wherein the invisible light is projected toward the display area to form a gesture control zone, and the touch zone and the gesture control zone are at least partially overlapped;

an image capturing module, configured to form a sensing region, wherein the sensing region at least covers the touch zone and the gesture control zone; and

a control unit, respectively electrically connected with the light curtain generating module, the image capturing module, and the invisible light lighting module and configured to execute a touch mode and a gesture control mode, wherein the control unit turns on the light curtain generating module and controls the image capturing module to capture a first set of images when executing the touch mode, and the control unit turns on the invisible light lighting module, and controls the image capturing module to capture a second set of images when executing the gesture control mode,

wherein when the control unit executes the touch mode, the image capturing module detects the sensing region to capture the first set of images, the control unit determines whether there is a touch signal according to the first set of images, and outputs a touch command if there is the touch signal, and executes the gesture control mode if there is no touch signal, and wherein when the control unit executes the gesture control mode, the image capturing module detects the sensing region to capture the second set of images, determines whether there is a gesture signal according to the second set of images, and executes the touch mode again if there is no gesture signal, the control unit determines whether time after outputting the previous touch command exceeds a predetermined value if there is the gesture signal, and executes the touch mode again if the time does not exceed the predetermined value, and the control unit identifies the gesture signal and outputs a gesture command if the time exceeds the predetermined value.

2. The touch and gesture control system as claimed in claim 1, wherein the touch command comprises a touch position in the touch zone computed according to the touch signal.

3. The touch and gesture control system as claimed in claim 1, wherein the control unit defines a plurality of time intervals of odd frames and a plurality of time intervals of even frames, the time intervals of the odd frames and the time intervals of the even frames proceed alternately, the control unit controls the time interval of the odd frames to be synchronized with time that the light curtain generating module generates the light curtain and controls the image capturing module to detect the sensing region during the time intervals of the odd frames to capture the first set of images, and the control unit controls the time interval of the even frames to be synchronized with time that the invisible light lighting module provides the invisible light and controls the image capturing module to detect the sensing region during the time intervals of the even frames to capture the second set of images.

4. The touch and gesture control system as claimed in claim 1, wherein the display area is a white plane or a white reflective layer.

5. The touch and gesture control system as claimed in claim 1, further comprising:

a projection apparatus, facing the display area and configured to project an image beam onto the display area, wherein the light curtain is located between the display area and the projection apparatus, and the invisible light lighting module and the image capturing module are respectively independent components and disposed on the projection apparatus.

6. The touch and gesture control system as claimed in claim 5, wherein the invisible light lighting module comprises:

an invisible light source, providing the invisible light; and

a projection optical module, configured to project the invisible light emitted by the invisible light source toward the display area.

7. The touch and gesture control system as claimed in claim 1, further comprising:

a projection apparatus, facing the display area and configured to project an image beam onto the display area, wherein the light curtain is located between the display area and the projection apparatus, the image capturing module is disposed on the projection apparatus, and the invisible light lighting module is integrated into the projection apparatus.

8. The touch and gesture control system as claimed in claim 7, wherein the invisible light lighting module comprises an invisible light filter.

9. The touch and gesture control system as claimed in claim 8, wherein the projection apparatus further comprises a color wheel, and the invisible light filter is disposed on the color wheel.

10. The touch and gesture control system as claimed in claim 7, wherein the projection apparatus further comprises:

a light source module, configured to provide an illumination beam and the invisible light;

a light combining element, configured to combine the invisible light and the illumination beam; and

a light valve, located on transmitting paths of the invisible light and the illumination beam, wherein the image beam and the invisible light are projected out of the projection apparatus after the illumination beam is converted into the image beam by the light valve.

11. A touch and gesture control method, comprising:

executing a touch mode, wherein steps of the touch mode comprise;

turning on a light curtain generating module, and causing the light curtain generating module generates a light curtain to form a touch zone, wherein the light curtain generating module is disposed at a side of a display area, and the light curtain is parallel to the display area;

causing an image capturing module to detect a sensing region and capture a first set of images; and

determining whether there is a touch signal according to the first set of images, outputting a touch command if there is the touch signal, and executing a gesture control mode if there is no touch signal, wherein steps of the gesture control mode comprise;

turning on an invisible light lighting module, and causing the invisible light lighting module projects invisible light toward the display area to form a gesture control zone;

causing the image capturing module to detect the sensing region and capture a second set of images;

determining whether there is a gesture signal according to the second set of images, performing the steps of the touch mode again if there is no gesture signal, and identifying the gesture signal and outputting a gesture command if there is the gesture signal, wherein the sensing region at least covers the touch zone and the gesture control zone, and the touch zone and the gesture control zone are at least partially overlapped; and

before performing the step of identifying the gesture signal, determining whether time after outputting the previous touch command exceeds a predetermined value, executing the touch mode again if the time does not exceed the predetermined value, and executing the step of identifying the gesture signal if the time exceeds the predetermined value.

12. The touch and gesture control method as claimed in claim 11, wherein the touch command comprises a touch position in the touch zone computed according to the touch signal.

13. The touch and gesture control method as claimed in claim 11, further comprising:

defining a plurality of time intervals of odd frames and a plurality of time intervals of even frames, wherein the time intervals of the odd frames and the time intervals of the even frames proceed alternately;

synchronizing the time interval of the odd frames with time that the light curtain generating module generates the light curtain;

causing the image capturing module to detect the sensing region during the time intervals of the odd frames to capture the first set of images;

synchronizing the time interval of the even frames with time that the invisible light lighting module provides the invisible light; and

causing the image capturing module to detect the sensing region during the time intervals of the even frames to capture the second set of images.

14. The touch and gesture control method as claimed in claim 11, wherein the display area is a white plane or a white reflective layer.

15. The touch and gesture control method as claimed in claim 11, wherein the invisible light lighting module and the image capturing module are respectively independent components and disposed on a projection apparatus facing the display area, the light curtain is located between the display area and the projection apparatus, and the invisible light lighting module comprises:

an invisible light source, providing the invisible light; and

a projection optical module, configured to project the invisible light emitted by the invisible light source toward the display area.

16. The touch and gesture control method as claimed in claim 11, wherein the invisible light lighting module is integrated in a projection apparatus, the projection apparatus faces the display area, the light curtain is located between the display area and the projection apparatus, and the invisible light lighting module comprises an invisible light filter.

17. The touch and gesture control method as claimed in claim 16, wherein the projection apparatus further comprises a color wheel, and the invisible light filter is disposed on the color wheel.

18. The touch and gesture control method as claimed in claim 11, wherein the invisible light lighting module is integrated in a projection apparatus, the projection apparatus faces the display area, the light curtain is located between the display area and the projection apparatus, and the projection apparatus further comprises:

a light source module, configured to provide an illumination beam and the invisible light;

a light combining element, configured to combine the invisible light and the illumination beam; and

a light valve, located on transmitting paths of the invisible light and the illumination beam, wherein an image beam and the invisible light are projected out of the projection apparatus after the illumination beam is converted into the image beam by the light valve.

19. A touch and gesture control system, comprising:

a light curtain generating module, disposed at a side of a display area, configured to generate a light curtain parallel to the display area and to form a touch zone;

an invisible light lighting module, configured to provide invisible light, wherein the invisible light is projected toward the display area to form a gesture control zone, and the touch zone and the gesture control zone are at least partially overlapped;

an image capturing module, configured to form a sensing region, wherein the sensing region at least covers the touch zone and the gesture control zone; and

a control unit, respectively electrically connected with the light curtain generating module, the image capturing module, and the invisible light lighting module and configured to execute a touch mode and a gesture control mode, wherein the control unit turns on the light curtain generating module and controls the image capturing module to capture a first set of images when executing the touch mode, and the control unit turns on the invisible light lighting module, and controls the image capturing module to capture a second set of images when executing the gesture control mode,

wherein the control unit defines a plurality of time intervals of odd frames and a plurality of time intervals of even frames, the time intervals of the odd frames and the time intervals of the even frames proceed alternately, the control unit controls the time interval of the odd frames to be synchronized with time that the light curtain generating module generates the light curtain and controls the image capturing module to detect the sensing region during the time intervals of the odd frames to capture the first set of images, and the control unit controls the time interval of the even frames to be synchronized with time that the invisible light lighting module provides the invisible light and controls the image capturing module to detect the sensing region during the time intervals of the even frames to capture the second set of images.

20. A touch and gesture control method, comprising:

executing a touch mode, wherein steps of the touch mode comprise;

turning on a light curtain generating module, and causing the light curtain generating module generates a light curtain to form a touch zone, wherein the light curtain generating module is disposed at a side of a display area, and the light curtain is parallel to the display area;

causing an image capturing module to detect a sensing region and capture a first set of images; and

determining whether there is a touch signal according to the first set of images, outputting a touch command if there is the touch signal, and executing a gesture control mode if there is no touch signal, wherein steps of the gesture control mode comprise;

turning on an invisible light lighting module, and causing the invisible light lighting module projects invisible light toward the display area to form a gesture control zone;

causing the image capturing module to detect the sensing region and capture a second set of images;

determining whether there is a gesture signal according to the second set of images, performing the steps of the touch mode again if there is no gesture signal, and identifying the gesture signal and outputting a gesture command if there is the gesture signal, wherein the sensing region at least covers the touch zone and the gesture control zone, and the touch zone and the gesture control zone are at least partially overlapped;

defining a plurality of time intervals of odd frames and a plurality of time intervals of even frames, wherein the time intervals of the odd frames and the time intervals of the even frames proceed alternately;

synchronizing the time interval of the odd frames with time that the light curtain generating module generates the light curtain;

causing the image capturing module to detect the sensing region during the time intervals of the odd frames to capture the first set of images;

synchronizing the time interval of the even frames with time that the invisible light lighting module provides the invisible light; and

causing the image capturing module to detect the sensing region during the time intervals of the even frames to capture the second set of images.