Method and Apparatus to Infer Object and Agent Properties, Activity Capacities, Behaviors, and Intents from Contact and Pressure Images

• US 20150282766A1
• Filed: 03/18/2015
• Published: 10/08/2015
• Est. Priority Date: 03/19/2014
• Status: Active Grant

• Current Assignee: Tactonic Technologies, LLC
• Original Assignee: Tactonic Technologies, LLC
• Inventors: Seidman, Gerald, Cole, Michael John
• Granted Patent: US 10,335,091 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 5/1038 Measuring plantar pressure during gait
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/112 Gait analysis
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/4023 Evaluating sense of balance
• CPC Class Codes: A61B 5/4088 Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia
• CPC Class Codes: A61B 5/6892 Mats
• CPC Class Codes: A61B 5/7246 using correlation, e.g. template matching or determination of similarity
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G01L 5/16 for measuring several components of force
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

An apparatus for determining a non-apparent attribute of an object having a sensor portion with which the object makes contact and to which the object applies pressure. The apparatus has a computer in communication with the sensor portion that receives signals from the sensor portion corresponding to the contact and pressure applied to the sensor portion, and determines from the signals the non-apparent attribute. The apparatus has an output in communication with the computer that identifies the non-apparent attribute determined by the computer. A method for determining a non-apparent attribute of an object.

Claims

1. A computer-implemented method to learn classifications and properties of one or more objects by processing a sequence of surface contact and/or pressure measurements captured by a plurality of local contact or pressure sensing system at a time and over time, the method comprising the steps of:

receiving a frame or frames of the sequence which includes data for a plurality of contact or pressure measurements included in the frame;

identifying one or more collections of contact and/or pressure measurements in the frame, where each collection represents an object on the surface;

generating models to extract one or more features from the contact and/or pressure measurement collections that are associated with each identified object;

extracting a plurality of features from the collections of contact and/or pressure measurements;

classifying each of the collections of contact and/or pressure measurements using a trained or untrained classifier;

supplying the extracted features and/or the object classifications from one or more frames to a machine learning engine; and

using the machine learning engine to generate values for one or more properties of one or more objects and/or generate semantic representations of the behavior of one or more objects over a plurality of frames, where the machine learning engine is configured to learn properties and behavior patterns observed in the contact and/or surface pressure measurements over the plurality of frames and to identify patterns of behavior by the classified objects.

2. The method of claim 1 further comprising the steps of locating one or more objects on the surface by detecting one or more features using a plurality of feature prediction models applied to the entire surface or to regularly or randomly selected regions of the surface, or by using similarity to an exemplar or to a reference collection of examples or to previously observed objects, applied to the entire surface or to regularly or randomly selected regions of the surface.

3. The method of claim 2 further comprising locating and tracking one or more objects on the surface by detecting their current position in a space-time window and calculating the overlap with previous detected positions or by predicting the next position of the object or objects using learned models from the configured machine learning engine or by applying a search algorithm using the previously detected positions.

4. The method of claim 3 further comprising the reacquisition of one or more objects with some level of confidence when they reenter the surface by comparing directly or with some feature transform their feature signature with stored models of objects accessible by the system.

5. The method of claim 3 further comprising recording the location and monitoring the paths of one or more objects on the surface stored in local memory or remotely.

6. The method of claim 3 further comprising the segmentation of the recorded paths of one or more objects into units using learned models from the configured machine learning engine or by applying heuristic rules or rules learned from observations not based on contact or pressure data.

7. The method of claim 5 further comprising the segmentation of the recorded paths of one or more objects into units using measurements from other systems in real time or from stored data of the same objects or exemplars of the objects.

8. The method of claim 1 further comprising the configuration of the machine learning module to infer properties of objects that are correlated with contact and pressure properties of objects.

9. The method of claim 8 further comprising the identification of people from combinations of inferred properties and contact and/or pressure patterns.

10. The method of claim 8 further comprising the identification of people from combinations of inferred properties and contact and/or pressure patterns and their location.

11. The method of claim 3 further comprising the calculation of gait velocity and balance using learned models from the configured machine learning engine.

12. The method of claim 11 using changes in patterns of gait velocity and balance measurements recorded at disparate times to calculate the future risk of falling using learned models from the configured machine learning engine.

13. The method of claim 11 using changes in patterns of gait velocity and balance measurements recorded at disparate times and correlating them with changes in a reference collection of expert judgments about the future risk of falling based on sequences of gait velocity and balance measurements recorded at disparate times.

14. The method of claim 11 using changes in patterns of gait velocity and balance measurements recorded at disparate times to calculate the rate of cognitive capacity decline using learned models from the configured machine learning engine.

15. The method of claim 11 using changes in patterns of gait velocity and balance measurements recorded at disparate times and correlating them with changes in a reference collection of expert judgments about the rate of cognitive capacity decline based on sequences of gait velocity and balance measurements recorded at disparate times.

16. An apparatus for determining a non-apparent attribute of an object comprising:

a sensor portion with which the object makes contact and to which the object applies pressure;

a computer in communication with the sensor portion that receives signals from the sensor portion corresponding to the contact and pressure applied to the sensor portion, determines from the signals the non-apparent attribute; and

an output in communication with the computer that identifies the non-apparent attribute determined by the computer.

17. The apparatus of claim 16 wherein the object is a person, the non-apparent attribute is cognitive capacity decline, the sensor portion is a plurality of sensor tiles forming a walkway, and the signals are indicative of gait of the person moving on the walkway.

18. A method for determining a non-apparent attribute of an object comprising the steps of:

making contact and applying pressure with an object to a sensor portion;

receiving signals by a computer in communication with the sensor portion from the sensor portion corresponding to the contact and pressure applied to the sensor portion;

determining by the computer from the signals the non-apparent attribute; and

identifying at an output in communication with the computer the non-apparent attribute determined by the computer.

19. The method of claim 18 wherein the object is a person, the non-apparent attribute is cognitive capacity decline, the sensor portion is a plurality of sensor tiles forming a walkway, and the signals are indicative of gait of the person moving on the walkway.