Method and system for recognizing user activity type

• US 9,984,304 B2
• Filed: 10/28/2015
• Issued: 05/29/2018
• Est. Priority Date: 05/29/2013
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: He, Xiuqiang, Zhang, Gong
• Primary Examiner(s): Liew, Alex
• Application Number:
  US14/925,368
• Publication Number:
  US 20160048738A1
• Time in Patent Office: 944 Days
• CPC Class Codes: G06F 16/24573 using data annotations, e.g. user-defined metadata
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

The present invention discloses a method and system for recognizing a user activity type, where the method includes: collecting an image of a location in which a user is located; extracting, from the image, characteristic data of an environment in which the user is located and characteristic data of the user; and obtaining, by recognition, an activity type of the user by using an image recognition model related to an activity type or an image library related to an activity type and the characteristic data.

Claims

1. A method for recognizing a user activity type, the method comprising:

collecting an image of a location in which a user is located;

extracting, from the image, environmental characteristic data that is characteristic of an environment in which the user is located, and user characteristic data of the user, wherein the environmental characteristic data include data related to one or more activity-related objects, other than the user; and

obtaining, by recognition, a user activity type by using an image recognition model or an image library related to an activity type and further according to a match between an annotation of an image in the image library and a hash sketch value representing the environmental characteristic data and user characteristic data, wherein the activity type is associated with the one or more activity-related objects.

2. The method according to claim 1, wherein obtaining the user activity type comprises using the image recognition model related to the activity type.

3. The method according to claim 1, wherein obtaining the user activity type comprises using the image library related to an activity type.

4. The method according to claim 1, wherein the extracting the environmental characteristic data and the user characteristic data comprises extracting the environmental characteristic data and the user characteristic data from the image by using an image object recognition method;

and

wherein the obtaining the user activity type comprises matching the environmental characteristic data and user characteristic data by using a rule that is pre-learned by using an activity type rule model method or an activity type machine learning method, so as to obtain the user activity type.

5. The method according to claim 1, wherein the environmental characteristic data and user characteristic data is the hash sketch value;

wherein the extracting the environmental characteristic data and the user characteristic data comprises extracting a hash sketch value of the image by using an image hash characteristic extraction method; and

wherein the obtaining the user activity type comprises;

searching, according to a matching algorithm, the image library for an image matching the hash sketch value;

searching for the annotation of the image matching the hash sketch value, wherein the annotation indicates content comprised in the image matching the hash sketch value; and

recognizing the user activity type according to the annotation.

6. The method according to claim 5, wherein the method further comprises, when the image matching the hash sketch value does not have the annotation:

extracting the environmental characteristic data and user characteristic data from the image matching the hash sketch value by using an image object recognition method; and

matching the environmental characteristic data and user characteristic data of the image matching the hash sketch value by using a rule that is pre-learned by using an activity type rule model method and an activity type machine learning method, so as to obtain the user activity type.

7. The method according to claim 1, wherein the method further comprises collecting, before the obtaining the user activity type, auxiliary information of the user environment or auxiliary information of the user;

and

wherein the obtaining the user activity type comprises matching the environmental characteristic data and user characteristic data by using the image recognition model related to the user activity type or the image library related to the activity type and the auxiliary information, so as to obtain the user activity type.

8. The method according to claim 7, wherein the auxiliary information comprises position information, acceleration information, light intensity information, and sound information.