Method and System for Recognizing User Activity Type

• US 20160048738A1
• Filed: 10/28/2015
• Published: 02/18/2016
• Est. Priority Date: 05/29/2013
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: He, Xiuqiang, Zhang, Gong
• Granted Patent: US 9,984,304 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24573 using data annotations, e.g. user-defined metadata
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

The present invention discloses a method and system for recognizing a user activity type, where the method includes: collecting an image of a location in which a user is located; extracting, from the image, characteristic data of an environment in which the user is located and characteristic data of the user; and obtaining, by recognition, an activity type of the user by using an image recognition model related to an activity type or an image library related to an activity type and the characteristic data.

Claims

1. A method for recognizing a user activity type, the method comprising:

collecting an image of a location in which a user is located;

extracting environmental characteristic data that is characteristic an environment in which the user is located and user characteristic data of the user from the image; and

obtaining by recognition a user activity type by using an image recognition model or an image library related to an activity type and the environmental characteristic data and user characteristic data.

2. The method according to claim 1, wherein obtaining the user activity type comprises using the image recognition model related to the activity type.

3. The method according to claim 1, wherein obtaining the user activity type comprises using an image library related to an activity type.

4. The method according to claim 1, wherein the extracting step comprises extracting the environmental characteristic data and the user characteristic data from the image by using an image object recognition method;

and

wherein the obtaining step comprises matching the environmental characteristic data and user characteristic data by using a rule that is pre-learned by using an activity type rule model method or an activity type machine learning method, so as to obtain the user activity type.

5. The method according to claim 1, wherein:

the environmental characteristic data and user characteristic data is a hash sketch value;

the extracting step comprises extracting a hash sketch value of the image by using an image hash characteristic extraction method; and

the obtaining step comprises;

searching, according to a matching algorithm, the image library for an image matching the hash sketch value;

searching for an annotation of the image matching the hash sketch value, wherein the annotation is used to indicate content comprised in the image matching the hash sketch value; and

recognizing the user activity type according to the annotation.

6. The method according to claim 5, wherein the image matching the hash sketch value does not have the annotation, the method further comprising:

extracting the environmental characteristic data and user characteristic data from the image matching the hash sketch value by using an image object recognition method; and

matching the environmental characteristic data and user characteristic data of the image matching the hash sketch value by using a rule that is pre-learned by using an activity type rule model method and an activity type machine learning method, so as to obtain the user activity type.

7. The method according to claim 1, wherein before the obtaining step, the method further comprises collecting auxiliary information of the user environment or auxiliary information of the user;

and

wherein the obtaining step comprises matching the environmental characteristic data and user characteristic data by using the image recognition model related to the user activity type or the image library related to the activity type and the auxiliary information, so as to obtain the user activity type.

8. The method according to claim 7, wherein the auxiliary information comprises position information, acceleration information, light intensity information, and sound information.

9. A system comprising:

a first collecting module, configured to collect an image of a user location;

an extracting module, configured to extract, from the image, environment characteristic data in which the user is located and user characteristic data; and

a recognizing module, configured to obtain, by recognition, a user activity type by using an image recognition model related to an activity type and the environment characteristic data and the user characteristic data or by using an image library related to the activity type and the environment characteristic data and the user characteristic data.

10. The system according to claim 9, wherein the extracting module comprises a first sub-extracting unit, configured to extract, from the image by using an image object recognition method, the environment characteristic data and the user characteristic data;

and

wherein the recognizing module comprises a first sub-recognizing unit, configured to match the environment characteristic data and the user characteristic data by using a rule that is pre-learned by using an activity type rule model method or an activity type machine learning method, so as to obtain the user activity type.

11. The system according to claim 9, wherein

the environment characteristic data and the user characteristic data are a hash sketch value;

the extracting module comprises a second sub-extracting unit, configured to extract a hash sketch value of the image by using an image hash characteristic extraction method;

the recognizing module comprises a matching unit, a searching unit, and a second sub-recognizing unit; and

the matching unit is configured to search, according to a matching algorithm, the image library for an image matching the hash sketch value;

the searching unit is configured to search for an annotation of the image matching the hash sketch value, wherein the annotation is used to indicate content comprised in the image matching the hash sketch value; and

the second sub-recognizing unit is configured to recognize the user activity type according to the annotation.

12. The system according to claim 11, wherein

the recognizing module further comprises a third sub-extracting unit and a third sub-recognizing unit;

the third sub-extracting unit is configured to, when the searching unit does not find the annotation of the image matching the hash sketch value, extract, from the image matching the hash sketch value and by using an image object recognition method, the environment characteristic data and the user characteristic data; and

the third sub-recognizing unit is configured to match, by using a rule that is pre-learned by using an activity type rule model method or an activity type machine learning method the environment characteristic data and the user characteristic data of the image matching the hash sketch value, so as to obtain the user activity type.

13. The system according to claim 9, wherein the system further comprises:

a second collecting module, configured to collect auxiliary information of an environment in which the user is located or auxiliary information of the user;

wherein the recognizing module is specifically configured to match the environment and user characteristic data by using the image recognition model related to the activity type or the image library related to the activity type and the auxiliary information, so as to obtain the user activity type.

14. The system according to claim 13, wherein the auxiliary information comprises position information, acceleration information, light intensity information, and sound information.