Pointing device having fingerprint image recognition function, fingerprint image recognition and pointing method, and method for providing portable terminal service using thereof

• US 20050249386A1
• Filed: 06/30/2004
• Published: 11/10/2005
• Est. Priority Date: 06/30/2003
• Status: Abandoned Application

• Current Assignee: Mobisol
• Original Assignee: Mobisol
• Inventors: Juh, Sung Chul
• Application Number:
  US10/520,651
• Publication Number:
  US 20050249386A1
• Time in Patent Office: Days
• US Class Current: 382/124
• CPC Class Codes: G06F 2203/0336 Mouse integrated fingerprint sensor
• CPC Class Codes: G06F 2203/0338 Fingerprint track pad, i.e. fingerprint sensor used as pointing device tracking the fingertip image
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06V 40/1324 by using geometrical optics, e.g. using prisms G06V40/1312 takes precedence
• CPC Class Codes: G06V 40/1335 Combining adjacent partial images (e.g. slices) to create a composite input or reference pattern; Tracking a sweeping finger movement

Abstract

A pointing device having a fingerprint recognition function, a fingerprint image recognition and pointing method, and a method for providing a portable terminal service using the same are disclosed. In the pointing device having a fingerprint recognition function and a fingerprint recognition method, fingerprint images having small sizes are mapped to generate a large fingerprint image or a small fingerprint image extracted from the large fingerprint image so that the pointing device performs user recognition and pointer control. As a result, respective sensors for the user recognition and for the pointer control are not comprised in the pointing device, but only one kind of sensors for performing both functions of user recognition and pointer control is comprised in the pointing device according to an embodiment of the present invention. Also, it is possible to easily embody miniaturization of a portable terminal device because the user recognition which requires a large fingerprint can be performed only with a small fingerprint image, thereby reducing manufacturing cost. Additionally, important information in the portable terminal device having the pointing device can be protected by selectively limiting kinds of service usable in the portable terminal device depending on user recognition.

Claims

1. A pointing device having a fingerprint image recognition, comprising:

at least one or more fingerprint acquiring means for acquiring a fingerprint image of a finger surface depending on a predetermined cycle;

a characteristic point extracting means for extracting at least one or more characteristic points from the acquired fingerprint image;

a movement detecting means for calculating displacement data between characteristic points of the extracted fingerprint image to detect movement information of the fingerprint image;

a mapping means for mapping the fingerprint image in an inner virtual image space depending on the movement information;

a recognizing means for comparing a previously registered fingerprint image with the whole mapped fingerprint image when the entire size of the mapped fingerprint image reaches a previously set size and determining recognition on the fingerprint; and

an operating means for receiving the displacement data from the movement detecting means and calculating a direction and a distance where a pointer is to move with the displacement data.

2. The pointing device according to claim 1, further comprising a housing which includes the fingerprint acquiring means, the characteristic point extracting means, the movement detecting means, the mapping means, the recognizing means and the operating means and comprises a transparent member having a plane surface which the finger surface contacts with at a predetermined distance from the fingerprint acquitting means.

3. The pointing device according to claim 1, wherein the fingerprint acquiring means is a CMOS image sensor.

4. The pointing device according to claim 1, wherein the sizes of the acquired fingerprint image and the virtual image space are m×

n pixels and M×

N pixels, respectively, and the m and the n are smaller than the M and the N, respectively.

5. The pointing device according to claim 1, wherein the movement detecting means calculates movement distances and directions of the characteristics points of the fingerprint image acquired in the current cycle from those of the fingerprint image acquired in the previous cycle.

6. The pointing device according to claim 1, wherein the mapping means maps a fingerprint image in the virtual image space so that identical characteristic points are superposed when there are the identical characteristic points between characteristic points of the n−

1^th fingerprint image and the n^th fingerprint image.

7. The pointing device according to claim 1, wherein the recognizing means determines whether the characteristic points of the previously registered fingerprint image are identical with those of the whole mapped fingerprint image by matching the characteristic points of the previously registered fingerprint image with those of the whole mapped fingerprint image, and decides recognition of the fingerprint depending on the determination result.

8. A portable terminal device which comprises a pointing device described in claim 1 and performs a fingerprint recognition of a user and a control of the pointer.

47. The pointing device according to claim 1, wherein the CPU further comprises a function of processing a fingerprint recognition process in a software way for comparing a fingerprint image stored in the storing means with the fingerprint image received from the fingerprint acquiring means.

9. A method for recognizing a fingerprint for user recognition, comprising the steps of:

a fingerprint image acquiring step for acquiring at least one or more fingerprint images with a predetermined fingerprint acquiring sensor depending on a set cycle;

a characteristic point extracting step for extracting at least one or more characteristic points from the acquired fingerprint image;

a first mapping step for mapping a first fingerprint image in a specific location of a virtual image space;

a displacement data calculating step for calculating displacement data between characteristic points of the first fingerprint image and those of a second fingerprint image acquired in the next cycle after the cycle where the first fingerprint image is acquired;

a second mapping step for mapping the second fingerprint image with the displacement data in the virtual image space; and

a fingerprint recognition step for comparing characteristic points of the previously registered fingerprint image with those of the whole mapped fingerprint image when the whole size of the fingerprint images mapped in the virtual image space reaches a previously set size, and determines recognition of the fingerprint.

10. The method according to claim 9, wherein the sizes of the acquired fingerprint image and the virtual image space are m×

n pixels and M×

N pixels, respectively, and the m and the n are smaller than the M and the N, respectively.

11. The method according to claim 9, wherein the displacement data calculating step is to calculate movement distances and directions of the characteristic points of the second fingerprint image from those of the first fingerprint image.

12. The method according to claim 9, wherein the second mapping step is to map the second fingerprint image in a location corresponding the calculated displacement data from the first fingerprint image mapped in the virtual image space.

13. The method according to claim 9, wherein the second mapping step is to map the second fingerprint image in the virtual image space so that identical characteristic points are superposed when there are the identical characteristic points between the characteristic points of the first fingerprint image and the second fingerprint image.

14. The method according to claim 9, wherein the fingerprint image acquiring step is to acquire a plurality of fingerprint images at every time with a plurality of fingerprint acquiring sensors.

15. The method according to claim 14, wherein the plurality of acquired fingerprint images are images of the adjacent fingerprints.

16. The method according to claim 9, wherein the fingerprint recognition step comprises:

the first step of determining whether the size of the whole fingerprint image mapped in the virtual image space reaches the previously set size;

the second step of extracting at least one or more characteristic points from the whole fingerprint images when the whole fingerprint image reaches the previously set size depending on the determination result;

the third step of comparing characteristic points of the previously registered fingerprint image with the extracted characteristic points; and

the fourth step of determining recognition of the fingerprint depending on the comparison result.

17. A pointing device having a fingerprint recognition, comprising:

a fingerprint acquiring means (first operating cycle) for acquiring a required fingerprint image to a finger surface which controls a pointer through only once 2-dimensional image acquisition;

a fingerprint recognizing unit (second operating cycle) for extracting at least one or more characteristic points from the acquired fingerprint image and comparing characteristic points of the previously registered fingerprint image with the extracted characteristic points to recognize a user of the acquired fingerprint image; and

a pointing control unit (third operating cycle) for detecting movement information based on partial data of an image acquired with the first operating cycle and calculating displacement data of the fingerprint image depending on the movement information to calculate movement direction and distance of the pointer.

18. The pointing device according to claim 17, wherein the fingerprint recognizing unit comprises:

a characteristic point extracting means for extracting at least one or more characteristic points from the fingerprint image acquired by the fingerprint acquiring means; and

a recognizing means for comparing characteristic points of the previously registered fingerprint image with those that are extracted by the characteristic point extracting means to determine recognition of a user of the fingerprint image.

19. The pointing device according to claim 18, wherein the recognizing means determines whether the characteristic points of the previously registered fingerprint image are identical with the extracted characteristic points by matching the characteristic points of the previously registered fingerprint image and the extracted characteristic points, and performs a recognition on the user depending on the determination result.

20. The pointing device according to claim 17, wherein the pointing control unit comprises:

a fingerprint image extracting means for extracting a fingerprint image of m×

n pixels (here, m and n are integers) from that of M×

N pixels (here, M and N are integers, m<

M, n<

N) acquired by the fingerprint acquiring means;

a movement detecting means for calculating displacement data of the extracted fingerprint image of m×

n pixels to detect movement information of the respective fingerprint image; and

an operating means for receiving displacement data from the movement detecting means and calculating movement direction and distance of the pointer based on the displacement data.

21. The pointing device according to claim 20, wherein the movement detecting means calculates the movement direction and distance of the characteristic points of the fingerprint image acquired in the current cycle from those of the fingerprint image acquired in the previous cycle to calculate the displacement data of the fingerprint image.

22. The pointing device according to claim 20, wherein the M and the N range from 90 to 400, and the m and the n range from 15 to 80.

23. The pointing device according to claim 17, wherein the fingerprint acquiring means is a CMOS image sensor.

24. The pointing device according to claim 17, wherein the fingerprint acquiring means is an active capacitive sensor.

25. The pointing device according to claim 17, wherein the second operating cycle is 1˜

3 times/second.

26. The pointing device according to claim 17, wherein the third operating cycle is 800˜

1200 times/second.

27. The pointing device according to claim 17, wherein the fingerprint recognizing unit and the pointing control unit are individually operated at the same time depending on the second operating cycle and the third operating cycle.

28. The pointing device according to claim 17, further comprising:

a light emitting means for emitting light toward the finger surface which controls the pointer; and

a light gathering means for condensing the fingerprint image reflected from the finger surface, wherein a fingerprint image condensed by the light gathering means is acquired by the fingerprint acquiring means.

29. The pointing device according to claim 28, wherein the light gathering means is located between the light emitting means and the fingerprint acquiring means, the ratio of the distance between the light emitting means and the light gathering means and the distance between the light gathering means and the fingerprint acquiring means are n:

1, and n is a real number ranging from 1 to 5.

30. The pointing device according to claim 28, wherein the light gathering means is aspherics.

31. A portable terminal device comprising a pointing device described in claim 17 and for simultaneously performing a fingerprint recognition and a pointer control.

32. The portable terminal device according to claim 31, further comprising a display means for displaying previously stored information, wherein the display means displays the whole information or a performing function admitted to the user when a user recognition is performed in the fingerprint recognizing unit, and only displays information and usable functions within a limited range which is previously set when the user recognition is not performed.

33. A pointing method having a fingerprint recognition function, the pointing method of a pointer control device with an image sensor having a smaller size than a predetermined picture image required in a fingerprint recognition, comprising the steps of:

a fingerprint image acquiring step for acquiring at least one or more fingerprint images of M×

N pixels depending on a first operating cycle with a predetermined fingerprint acquiring sensor on a finger surface which controls a movable pointer;

a recognition step for determining recognition of a user of the fingerprint image by extracting characteristic points from the acquired fingerprint image depending on a second operating cycle and comparing the extracted characteristic points with those of the previously register fingerprint image;

a fingerprint image extracting step for extracting a fingerprint image of m×

n pixels from the acquired fingerprint image depending on a third operating cycle;

a movement detecting step for detecting movement information of the respective fingerprint image by calculating displacement data of the respective extracted fingerprint image of M×

N pixels; and

an operating step for calculating and outputting a direction and a distance where the pointer is to move with the displacement data.

34. The pointing method according to claim 33, wherein the recognition step comprises:

outputting a signal to use the whole information previously set when a user recognition is performed; and

outputting a signal to use limited information previously set when a user recognition is not performed.

35. The pointing method according to claim 33, wherein the M and N range from 90 to 400, and the m and the n range from 15 to 80.

36. The pointing method according to claim 33, wherein the third operating cycle is 800˜

1200 times/second.

37. The pointing method according to claim 33, wherein the movement detecting step calculating movement distance and direction of characteristic points of the k^th fingerprint image from the K−

1^th fingerprint image to calculate displacement data of each fingerprint image.

38. The pointing method according to claim 33, wherein the image extracting step, the movement detecting step and the operating step is individually performed from the recognition step depending on the third operating cycle.

39. The pointing method according to claim 33, further comprising the steps of:

a light emitting step for emitting light with a finger surface which controls the pointer; and

a condensing step for condensing a fingerprint image generated by the finger surface with aspherics, wherein the fingerprint acquiring step is to acquire a fingerprint image condensed by the condensing step.

40. The pointing method according to claim 39, wherein the condensing step is to condense the fingerprint image by reducing the size of the fingerprint on the finger surface by 1/n by regulating the aspherisc.

41. The pointing method according to claim 40, wherein the n is a real number ranging from 1 to 5.

42. A pointing device having a fingerprint recognition function, comprising:

at least one or more fingerprint acquiring means for acquiring an image of a finger surface depending on a predetermined cycle or on occasional requirement;

an operating means for calculating displacement data with the acquired fingerprint image to calculate direction and distance where a pointer is to move with the displacement data;

a storing means for mapping a fingerprint image obtained from the fingerprint acquiring means corresponding to the amount of movement in the displacement data received from the operating means; and

a CPU for analyzing and processing data of the operating means and the storing means.

43. The pointing device according to claim 42, wherein the CPU is formed as one package with the operating means and the storing means.

44. The pointing device according to claim 42, wherein the CPU is formed as a structure separate from the operating means and the storing means.

45. The pointing device according to claim 42, further comprising:

a light emitting means for emitting light toward the finger surface; and

a light gathering means for condensing a fingerprint image reflected from the fingerprint surface, wherein the fingerprint image condensed by the light gathering means is obtained by the fingerprint acquiring means.

46. The pointing device according to claim 45, wherein the light gathering means is located between the light emitting means and the fingerprint acquiring means, the ratio of the distance between the light emitting means and the light gathering means and the distance between the light gathering means and the fingerprint acquiring means are n:

1, and n is a real number ranging from 1 to 5.

48. The pointing device according to claim 42, wherein the storing means periodically stores a fingerprint image depending on a predetermined cycle or stores a fingerprint image only when the storing means receives a request of the CPU.

49. A method for providing service of a portable terminal device having a fingerprint recognition function, the method comprising:

the first step of acquiring a fingerprint image of a user by a pointing device having the fingerprint recognition function, and comprising a previously registered fingerprint image with the acquired fingerprint image to determined recognition of the user; and

the second step of classifying usage rights of a user depending on the determination result in the first step to display a menu corresponding to the usage right.

50. The method according to claim 49, further comprising the third step of enabling a user to use a desired function of the menu displayed in the second step.

51. The method according to claim 50, wherein the second step is to display the whole menu when recognition on the user is performed or to display a specific menu admitted to use when the recognition on the user is not performed.

52. The method according to claim 51, wherein when a user having no recognition intends to use other functions than the menu admitted to use, the message that no use is allowed is displayed and the menu admitted to use is displayed again.

53. The method according to claim 49, wherein the first step is performed as a background process to automatically acquire a fingerprint image while a user performs a navigation with a finger by using the pointing device without performing an additional process to acquire a fingerprint of the user.