User interface system and optical finger mouse system

• US 9,433,382 B2
• Filed: 10/01/2012
• Issued: 09/06/2016
• Est. Priority Date: 11/22/2011
• Status: Active Grant

• Current Assignee: PixArt Imaging Incorporated
• Original Assignee: PixArt Imaging Incorporated
• Inventors: Huang, Sen-Huang, Gu, Ren-Hau, Kao, Ming-Tsan
• Primary Examiner(s): Leiby, Christopher E
• Application Number:
  US13/632,327
• Publication Number:
  US 20130127714A1
• Time in Patent Office: 1,436 Days
• Field of Search: 345/158, 345/175, 600300-301, 600309-310, 600/322
• US Class Current: 1/1
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/14552 Details of sensors specially adapted therefor
• CPC Class Codes: A61B 5/6898 Portable consumer electronic devices, e.g. music players, telephones, tablet computers
• CPC Class Codes: A61B 5/721 using a separate sensor to detect motion or using motion information derived from signals other than the physiological signal to be measured
• CPC Class Codes: G06F 2203/0338 Fingerprint track pad, i.e. fingerprint sensor used as pointing device tracking the fingertip image
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184

Abstract

There is provided a user interface system including a slave device and a master device. The slave device provides light of two different wavelengths to illuminate a finger surface, receives reflected light from the finger surface to generate a plurality of image frames, calculates and outputs an image data associated with a predetermined number of the image frames. The master device calculates a contact status and a displacement of the finger surface and a physiological characteristic of a user according to the image data.

Claims

1. A user interface system configured to detect a displacement of a finger of a user and a physiological characteristic of the user, the user interface system comprising:

a slave device configured to

provide light of two different wavelengths to illuminate the finger,

receive reflected light from the finger to generate a plurality of first image frames and second image frames corresponding to on-states of the light of two different wavelengths,

calculate and output first image data associated with a first predetermined number of the first image frames and second image data associated with a second predetermined number of the second image frames; and

a master device configured to

calculate the displacement of the finger according to the correlation between the first image data and the second image data;

divide each of the first image data into a first image area of the first image data and a second image area of the first image data, respectively calculate an average brightness of each of the first image area and the second image area such that one average brightness is obtained for each of the first and second image areas of the first image data, wherein the second image area is other than the first image area among the first image data;

analyze, using an independent component analysis method, the average brightness of each of the first image area and the second image area of the first image data to obtain (i) a first movement information, which contains a first intensity variation of signal noise caused by movement, and (ii) a first physiology information;

abandon all the first movement information to eliminate the signal noise caused by movement, without calculating the physiological characteristic according to the first movement information;

divide each of the second image data into a third image area of the second image data and a fourth image area of the second image data, respectively calculate an average brightness of each of the third image area and the fourth image area such that one average brightness is obtained for each of the third and fourth image areas of the second image data, wherein the fourth image area is other than the third image area among the second image data;

analyze, using the independent component analysis method, the average brightness of each of the third image area and the fourth image area of the second image data to obtain (a) a second movement information, which contains a second intensity variation of the signal noise caused by movement, and (b) a second physiology information;

abandon all the second movement information to eliminate the signal noise caused by movement, without calculating the physiological characteristic according to the second movement information;

calculate the physiological characteristic according to the first physiology information and the second physiology information when the calculated displacement of the finger does not exceed a displacement threshold; and

not calculate the physiological characteristic according to the first image data and the second image data when the calculated displacement of the finger exceeds the displacement threshold.

2. The user interface system as claimed in claim 1, wherein

the slave device is a mouse, a remote controller, a keyboard or an optical distance measuring device;

and

the master device is a television, a projection device, a game machine system or a computer system.

3. The user interface system as claimed in claim 1, wherein the physiological characteristic comprises a blood oxygenation and a heart rate.

4. The user interface system as claimed in claim 1, wherein the slave device further comprises:

a first light source configured to provide light of a first wavelength to the finger;

a second light source configured to provide light of a second wavelength to the finger, the second wavelength different from the first wavelength;

a light control unit configured to control on-states and off-states of the first light source and the second light source;

an image sensor configured to receive the reflected light from the finger at a sampling frequency to generate the first image frames and the second image frames; and

a first processing unit configured to calculate the first image data of the first predetermined number of the first image frames and the second image data of the second predetermined number of the second image frames.

5. The user interface system as claimed in claim 4, wherein

the first image data is a first image sum, a first image average, a normalized first image sum or a normalized first image average of the first predetermined number of the first image frames;

and

the second image data is a second image sum, a second image average, a normalized second image sum or a normalized second image average of the second predetermined number of the second image frames.

6. The user interface system as claimed in claim 4, wherein

the light control unit is configured to enable the off-states of the first light source and the second light source for predetermined time intervals such that the image sensor captures a plurality of bright first image frames, a plurality of bright second image frames and a plurality of dark image frames;

and

the first processing unit is configured to calculate the first image data and the second image data according to a difference between the bright first image frames and the dark image frames and according to a difference between the bright second image frames and the dark image frames.

7. The user interface system as claimed in claim 4, wherein

the light control unit is configured to alternatively enable the on-states of the first light source and the second light source such that the image sensor receives the reflected light associated with the first light source and the second light source alternatively;

or

the light control unit is configured to simultaneously enable the on-states of the first light source and the second light source such that the image sensor receives the reflected light associated with the first light source and the second light source simultaneously, and the image sensor comprises an optical filter covering at least a part of a sensing surface thereof.

8. The user interface system as claimed in claim 1, wherein the master device further comprises:

a memory unit configured to store the first image data and the second image data received from the slave device;

a checking unit configured to check oversampled data and data loss of the first image data and the second image data; and

a second processing unit configured to calculate the displacement and the physiological characteristic.

9. The user interface system as claimed in claim 8, wherein the second processing unit is further configured to calculate a heart rate according to a comparison result of comparing at least one pulse threshold with at least one of the first physiology information and the second physiology information.

10. The user interface system as claimed in claim 1, wherein

the master device further comprises a response unit configured to respond to the displacement and the physiological characteristic;

and

the response unit is at least one of a display device, a lamp device, a seven-segment display and a sound device.

11. The user interface system as claimed in claim 1, wherein the user interface system is configured to enter a sleep mode after idling for a predetermined time period.

12. The user interface system as claimed in claim 1, wherein the slave device further comprises a memory unit configured to store a system frequency calibration parameter.

13. The user interface system as claimed in claim 1, wherein the master device is further configured to compare a brightness value of the first image data and the second image data with at least one brightness threshold to identify a contact status.

14. The user interface system as claimed in claim 1, wherein the master device is configured to calculate the displacement according to two of the first image data, according to one of the first image data and one of the second image data, and according to two of the second image data.

15. An optical finger mouse system, comprising:

an optical detection device, comprising;

a first light source configured to provide light of a first wavelength to a finger of a user;

a second light source configured to provide light of a second wavelength to the finger, the second wavelength different from the first wavelength;

a light control unit configured to control on-states and off-states of the first light source and the second light source;

an image sensor configured to receive reflected light from the finger at a sampling frequency to generate a plurality of first image frames corresponding to the on-states of the first light source and a plurality of second image frames corresponding to the on-states of the second light source; and

a processing unit configured to calculate and output first image data associated with a first predetermined number of the first image frames and second image data associated with a second predetermined number of the second image frames; and

a host configured to

receive and check the first image data and the second image data;

calculate a displacement of the finger according to the correlation between the first image data and the second image data;

divide each of the first image data into a first image area of the first image data and a second image area of the first image data, respectively calculate an average brightness of each of the first image area and the second image area such that one average brightness is obtained for each of the first and second image areas of the first image data, wherein the second image area is other than the first image area among the first image data;

analyze, using an independent component analysis method, the average brightness of each of the first image area and the second image area of the first image data to obtain a first movement information, which contains a first intensity variation of signal noise caused by movement, and a first physiology information;

abandon all the first movement information to eliminate the signal noise caused by movement, without calculating the physiological characteristic according to the first movement information;

divide each of the second image data into a third image area of the second image data and a fourth image area of the second image data, respectively calculate an average brightness of each of the third image area and the fourth image area such that one average brightness is obtained for each of the third and fourth image areas of the second image data, wherein the fourth image area is other than the third image area among the second image data;

analyze, using the independent component analysis method, the average brightness of each of the third image area and the fourth image area of the second image data to obtain a second movement information, which contains a second intensity variation of the signal noise caused by movement, and a second physiology information;

abandon all the second movement information to eliminate the signal noise caused by movement, without calculating the physiological characteristic according to the second movement information;

calculate a physiological characteristic of the user according to the first physiology information and the second physiology information when the calculated displacement of the finger does not exceed a displacement threshold; and

not calculate the physiological characteristic according to the first image data and the second image data when the calculated displacement of the finger exceeds the displacement threshold.

16. The optical finger mouse system as claimed in claim 15, wherein

when the host detects that the first and second image data have oversampled data, the host is configured to discard the oversampled data;

and

when the host detects that the first and second image data have data loss, the host is configured to perform interpolation.

17. A user interface system, comprising:

an optical finger mouse configured to

provide light of two different wavelengths to illuminate a finger of a user,

receive reflected light from the finger to generate a plurality of first image frames and second image frames corresponding to on-states of the light of two different wavelengths,

calculate and output first image data associated with a first predetermined number of the first image frames and second image data associated with a second predetermined number of the second image frames; and

a host, comprising;

a processing unit configured to

calculate a displacement of the finger according to the correlation between the first image data and the second image data, and

divide each of the first image data into a first image area of the first image data and a second image area of the first image data, respectively calculate an average brightness of each of the first image area and the second image area such that one average brightness is obtained for each of the first and second image areas of the first image data, wherein the second image area is other than the first image area among the first image data;

analyze, using an independent component analysis method, the average brightness of each of the first image area and the second image area of the first image data to obtain a first movement information, which contains a first intensity variation of signal noise caused by movement, and a first physiology information;

abandon all the first movement information to eliminate the signal noise caused by movement, without calculating the physiological characteristic according to the first movement information;

divide each of the second image data into a third image area of the second image data and a fourth image area of the second image data, respectively calculate an average brightness of each of the third image area and the fourth image area such that one average brightness is obtained for each of the third and fourth image areas of the second image data, wherein the fourth image area is other than the third image area among the second image data;

analyze, using the independent component analysis method, the average brightness of each of the third image area and the fourth image area of the second image data to obtain a second movement information, which contains a second intensity variation of the signal noise caused by movement, and a second physiology information;

abandon all the second movement information to eliminate the signal noise caused by movement, without calculating the physiological characteristic according to the second movement information;

calculate a physiological characteristic of the user according to the first physiology information and the second physiology information when the calculated displacement of the finger does not exceed a displacement threshold; and

not calculate the physiological characteristic according to the first image data and the second image data when the calculated displacement of the finger exceeds the displacement threshold; and

a response unit configured to respond to the displacement and the physiological characteristic.