Vision-based operating method and system

• US 20030190076A1
• Filed: 04/05/2002
• Published: 10/09/2003
• Est. Priority Date: 04/05/2002
• Status: Active Grant

• Current Assignee: Morphotrust USA Incorporated (Advent International Corporation)
• Original Assignee: Identix, Inc. (Advent International Corporation)
• Inventors: DeLean, Bruno
• Granted Patent: US 7,369,685 B2
• Time in Patent Office: Days
• US Class Current: 382/209
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A vision-based operating system is provided in which a computer-based system accepts visual inputs and takes appropriate actions in a wide variety of environments. One environment is a security environment in which the vision-based operating system acquires an image of a face and determines, with an arbitrarily small probability of a false positive, whether the face is of a person who is authorized to enter the environment.

Claims

1. A method of determining whether an acquired image matches a reference image, comprising:

acquiring a digital image for comparison to a reference image;

identifying a group of pixels in the acquired image;

selecting the pixel group in the reference image that is the best match for the pixel group in the acquired image;

repeating the preceding steps for at least one more group of pixels;

comparing the relative locations of the selected pixel groups in the reference images and the pixel groups in the acquired image; and

determining the probability that the relative locations occurred randomly.

2. A method of claim 1, further comprising:

comparing the probability of a random match to a threshold probability for concluding that the images match.

3. A method of claim 2, wherein the threshold probability is set to reduce the probability of a false conclusion to an arbitrarily low number.

4. A method of claim 1,

wherein finding the closest pixel group uses a least squares calculation of the differences between the pixel groups in the reference image and the selected pixel group in the acquired image.

5. A method of claim 1, further comprising:

upon acquiring the reference image and the acquired image, filtering the images prior to completing the additional steps.

6. A method of claim 5, wherein the filtering comprises:

determining the luminance of a pixel based on an average of values of neighboring pixels;

comparing the value of the luminance of the pixel to a threshold value;

setting the filtered value of the pixel to zero if the luminance is below the threshold; and

setting the filtered value of the pixel to one if the luminance is equal to or greater than the threshold value.

7. A method of claim 6, further comprising:

normalizing the acquired image and the reference image by locating known features in a predetermined location.

8. A method of claim 1, wherein acquiring the image is accomplished by a vision processing system having a vision-based operating system.

9. A method of claim 1, further comprising:

comparing the acquired image to more than one reference image.

10. A method of claim 1, wherein comparing the relative positions of the pixel groups of the acquired image and the pixel groups of the reference image comprises:

defining a first vector between the first pixel group of the acquired image and a second pixel group of the acquired image;

defining a second vector between the first pixel group of the reference image and the second pixel group of the reference image; and

calculating an absolute value of the difference between the first vector and the second vector.

11. A method of claim 10, further comprising:

repeating the calculation for additional pixel groups.

12. A method of claim 1, further comprising:

comparing the probability of a random match to a threshold probability for concluding that the images match;

repeating the steps for a different reference image; and

allowing unlimited number of mismatches before allowing a match to one of the reference images.

13. A method of claim 1, further comprising:

upon determining a match, adding the acquired image to the database of reference images to facilitate future matches.

14. A processor-based system having an image based-operating system, comprising:

a camera, located in an environment; and

a computer-based system in data connection with the camera, the computer-based operating system having an operating system that is capable of operating the computer-based system in response to image data acquired by the camera.

15. A system of claim 14, wherein the operating system is capable of operating the computer-based system solely based on the image data.

16. A system of claim 14, further comprising another system, wherein the other system is capable of receiving instructions from the computer-based system in response to actions taken by the operating system.

17. A system of claim 16, wherein the other system is selected from the group consisting of a security system, an alarm system, a communications system, an automated teller system, a banking system, a safe, another camera system, a speaker system, a microphone, a computer, a server, a laptop, a handheld computer, a bluetooth enabled device, an entertainment system, a television, a recorder, an appliance, a tool, an automobile system, a transportation system, a vehicle system, a sensor, an emitter, a transmitter, a transceiver, an antenna, a transponder, a gaming system, a computer network, a home network, a local area network, a wide area network, the Internet, the worldwide web, a satellite system, a cable system, a telecommunications system, a modem, a telephone, and a cellular phone.

18. A system of claim 14, wherein the operating system is capable of identifying a characteristic in an image and taking an action based on the characteristic.

19. A system of claim 18, wherein the characteristic is selected from the group consisting of:

a matching face, a matching code, motion, a biometric, a non-match element, a structure in an environment, an emotion of a face, presence of item in an environment, absence of an item in an environment, movement of an item, appearance of a new item in an image, smoke, fire, water, a leak, damage to an environment, action of a person, action of a pet, action of a child, action of an elderly person, a face, a gesture, positioning of a face in front of a camera, change of an image, detection of a face in the image, speech, lip movement, a finger movement, a hand movement, an arm movement, a leg movement, a movement of the body, a movement of the head, a movement of the neck, a shoulder movement, and a gait.

20. A system of claim 18, wherein the characteristic is matching a face and wherein the action is opening a security system.

21. A system of claim 18 wherein the action is opening a security system and wherein the action is based on a characteristic of the image and at least one other requirement.

22. A system of claim 21, wherein the other requirement is a requirement for at least one of a key, a pass card, a voiceprint, a fingerprint and a biometric.

23. A system of claim 14, wherein the environment is selected from the group consisting of an airport, an airplane, a transportation venue, a bus, a bus station, a train, a train station, a rental car venue, a car, a truck, a van, a workplace, a venue, a ticketed venue, a sports arena, a concert arena, a stadium, a sports venue, a concert venue, a museum, a store, a home, a pool, a gym, a health club, a golf club, a tennis club, a club, a parking lot, a computer, laptop, an electronic commerce environment, an ATM, a storage location, a safe deposit box, a bank, and an office.

24. A system of claim 14, wherein the environment is home and wherein the system is used to control access based on matching a face to a reference face.

25. A system of claim 24, wherein the system requires an additional step selected from the group consisting of providing a key, entering a code, inserting a card, recognizing a voice, recognizing a fingerprint, and recognizing another biometric.

26. A system of claim 14, wherein the system takes an action based on detecting an event.

27. A system of claim 26, wherein the event is intrusion and the action is triggering an alarm.

28. A system of claim 26, wherein the action is triggering an alarm and the event is selected from the group consisting of intrusion, smoke, fire, water, proximity of a child to a dangerous condition, and proximity of an elderly person to a dangerous condition.

29. A system of claim 26, wherein the event is a gesture selected from the group consisting of thumbs up sign, an okay sign, a number of fingers, a hand motion, a head motion, a sign language motion, a lip motion, and an eye motion.

30. A system of claim 26 wherein the event is motion in proximity to a structure selected from the group consisting of a pool, a stove, an oven, an appliance, an electrical equipment item, and a hot item.

31. A system of claim 26 wherein the event is absence of motion.

32. A system of claim 26 wherein the event is proximity of a collision and the action is a warning.

33. A method, comprising:

locating a camera in an environment;

capturing an image of the environment, the image comprising an image of an event of the environment;

providing a vision-based operating system for processing the image;

processing the image to identify a characteristic of the event; and

taking an action based on the characteristic.

34. A method of claim 33 wherein the operating system manages another operating system.

35. A method of claim 33 wherein the operating system triggers an event in a parent operating system.

36. A method of claim 33 wherein all of the commands for the operating system are enabled through the capture of image data.

37. A system for determining whether an acquired image matches a reference image, comprising:

a camera for acquiring a digital image for comparison to a reference image;

an identification module for identifying a group of pixels in the acquired image;

a selection module for selecting the pixel group in the reference image that is the best match for the pixel group in the acquired image;

a comparison module for repeating the preceding steps for at least one more group of pixels and for comparing the relative locations of the selected pixel groups in the reference images and the pixel groups in the acquired image; and

a calculator for determining the probability that the relative locations occurred randomly.

38. A system of claim 37, further comprising:

a match module for comparing the probability of a random match to a threshold probability for concluding that the images match.

39. A system of claim 38, wherein the threshold probability is set to reduce the probability of a false conclusion to an arbitrarily low number.

40. A system of claim 37, wherein finding the closest pixel group uses a least squares calculation of the differences between the pixel groups in the reference image and the selected pixel group in the acquired image.

41. A system of claim 37, further comprising:

a filter for filtering the images prior to completing the additional steps.

42. A system of claim 41, wherein the filtering comprises determining the luminance of a pixel based on an average of values of neighboring pixels, comparing the value of the luminance of the pixel to a threshold value, setting the filtered value of the pixel to zero if the luminance is below the threshold and setting the filtered value of the pixel to one if the luminance is equal to or greater than the threshold value.

43. A system of claim 42, further comprising:

a normalization module for normalizing the acquired image and the reference image by locating known features in a predetermined location.

44. A system of claim 37, wherein acquiring the image is accomplished by a vision processing system having a vision-based operating system.

45. A system of claim 37, further comprising:

a comparison module for comparing the acquired image to more than one reference image.

46. A system of claim 37, wherein comparing the relative positions of the pixel groups of the acquired image and the pixel groups of the reference image comprises:

defining a first vector between the first pixel group of the acquired image and a second pixel group of the acquired image;

defining a second vector between the first pixel group of the reference image and the second pixel group of the reference image; and

calculating an absolute value of the difference between the first vector and the second vector.

47. A system of claim 37, further comprising:

a calculator for the probability of a random match to a threshold probability for concluding that the images match; and

a reference image retrieval module for repeating the steps for a different reference image, wherein the system allows an unlimited number of mismatches before allowing a match to one of the reference images.

48. A system of claim 37, further comprising:

an image storage module for, upon determining a match, adding the acquired image to the database of reference images to facilitate future matches.

49. A method of providing a processor-based system having an image based-operating system, comprising:

locating a camera in an environment; and

providing a computer-based system in data connection with the camera, the computer-based operating system having an operating system that is capable of operating the computer-based system in response to image data acquired by the camera.

50. A method of claim 49, wherein the operating system is capable of operating the computer-based system solely based on the image data.

51. A method of claim 49, further comprising providing another system, wherein the other system is capable of receiving instructions from the computer-based system in response to actions taken by the operating system.

52. A method of claim 51, wherein the other system is selected from the group consisting of a security system, an alarm system, a communications system, an automated teller system, a banking system, a safe, another camera system, a speaker system, a microphone, a computer, a server, a laptop, a handheld computer, a bluetooth enabled device, an entertainment system, a television, a recorder, an appliance, a tool, an automobile system, a transportation system, a vehicle system, a sensor, an emitter, a transmitter, a transceiver, an antenna, a transponder, a gaming system, a computer network, a home network, a local area network, a wide area network, the Internet, the worldwide web, a satellite system, a cable system, a telecommunications system, a modem, a telephone, and a cellular phone.

53. A system of claim 49, wherein the operating system is capable of identifying a characteristic in an image and taking an action based on the characteristic.

54. A method of claim 53, wherein the characteristic is selected from the group consisting of:

a matching face, a matching code, motion, a biometric, a non-match element, a structure in an environment, an emotion of a face, presence of item in an environment, absence of an item in an environment, movement of an item, appearance of a new item in an image, smoke, fire, water, a leak, damage to an environment, action of a person, action of a pet, action of a child, action of an elderly person, a face, a gesture, positioning of a face in front of a camera, change of an image, detection of a face in the image, speech, lip movement, a finger movement, a hand movement, an arm movement, a leg movement, a movement of the body, a movement of the head, a movement of the neck, a shoulder movement, and a gait.

55. A method of claim 54, wherein the characteristic is matching a face and wherein the action is opening a security system.

56. A method of claim 54, wherein the action is opening a security system and wherein the action is based on a characteristic of the image and at least one other requirement.

57. A method of claim 56, wherein the other requirement is a requirement for at least one of a key, a pass card, a voiceprint, a fingerprint and a biometric.

58. A method of claim 49, wherein the environment is selected from the group consisting of an airport, an airplane, a transportation venue, a bus, a bus station, a train, a train station, a rental car venue, a car, a truck, a van, a workplace, a venue, a ticketed venue, a sports arena, a concert arena, a stadium, a sports venue, a concert venue, a museum, a store, a home, a pool, a gym, a health club, a golf club, a tennis club, a club, a parking lot, a computer, laptop, an electronic commerce environment, an ATM, a storage location, a safe deposit box, a bank, and an office.

59. A method of claim 49, wherein the environment is home and wherein the system is used to control access based on matching a face to a reference face.

60. A method of claim 59, wherein the system requires an additional step selected from the group consisting of providing a key, entering a code, inserting a card, recognizing a voice, recognizing a fingerprint, and recognizing another biometric.

61. A method of claim 49, wherein the system takes an action based on detecting an event.

62. A method of claim 61, wherein the event is intrusion and the action is triggering an alarm.

63. A method of claim 61, wherein the action is triggering an alarm and the event is selected from the group consisting of intrusion, smoke, fire, water, proximity of a child to a dangerous condition, and proximity of an elderly person to a dangerous condition.

64. A method of claim 61, wherein the event is a gesture selected from the group consisting of thumbs up sign, an okay sign, a number of fingers, a hand motion, a head motion, a sign language motion, a lip motion, and an eye motion.

65. A method of claim 61, wherein the event is motion in proximity to a structure selected from the group consisting of a pool, a stove, an oven, an appliance, an electrical equipment item, and a hot item.

66. A method of claim 61, wherein the event is absence of motion.

67. A method of claim 61, wherein the event is proximity of a collision and the action is a warning.

68. A vision-based system, comprising:

a camera in an environment for capturing an image of the environment, the image comprising an image of an event of the environment;

a vision-based operating system for processing the image to identify a characteristic of the event; and

an actuator for taking an action based on the characteristic.

69. A system of claim 68, wherein the operating system manages another operating system.

70. A system of claim 68, wherein the operating system triggers an event in a parent operating system.

71. A system of claim 68, wherein all of the commands for the operating system are enabled through the capture of image data.