Method and device for automatically playing expression on virtual image

• US 9,457,265 B2
• Filed: 03/25/2013
• Issued: 10/04/2016
• Est. Priority Date: 04/06/2012
• Status: Active Grant

• Current Assignee: Tencent Technology Shenzhen Company Limited (Tencent Holdings Limited)
• Original Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Inventors: Wang, Dong
• Primary Examiner(s): Ahmed, Masud
• Application Number:
  US14/129,636
• Publication Number:
  US 20150038222A1
• Time in Patent Office: 1,289 Days
• Field of Search: 463/42
• US Class Current: 1/1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/176 Dynamic expression

Abstract

Provided are a method and device for automatically playing an expression on a virtual image. The method includes the steps of: A, capturing a video image related to a game player in a game, and determining whether the captured video image contains a facial image of the game player, and if so, executing a step B; otherwise, returning to the step A; B: extracting facial features from the facial image, and obtaining a motion track of each of the facial features by motion directions of the extracted facial feature that are acquired for N times, where N is greater than or equal to 2; and C, determining a corresponding facial expression according to the motion track of the facial feature which is obtained in step B, and automatically playing the determined facial expression on a virtual image of the game player.

Claims

1. A method for automatically playing an expression on a virtual image, comprising steps of:

A;

capturing, by a machine, a video image related to a game player in a game, and determining, by the machine, whether the captured video image contains a facial image of the game player, and if so, executing a step B;

otherwise, returning to the step A;

B;

extracting, by the machine, facial features from the facial image, and obtaining, by the machine, a motion track of each of the extracted facial features by motion directions of the extracted facial features that are sequentially acquired for N times;

wherein N is greater than or equal to 2; and

C;

determining, by the machine, a corresponding facial expression according to the motion track of the facial feature which is obtained in step B, and automatically playing, by the machine, the determined facial expression on a virtual image of the game player;

and further comprising;

obtaining, by the machine, a set of correspondence relations between facial expressions and common facial motion tracks through pre-training;

wherein, the step C comprises;

C1;

searching a correspondence relation from the set of correspondence relations, wherein the common facial motion tracks of the correspondence relation are related to motion tracks of the extracted facial features; and

if the correspondence relation is searched, then executing a step C2;

C2;

automatically playing the facial expression in the searched correspondence relation on the virtual image of the game player;

wherein, in the step C1, the fact that the common facial motion tracks of the correspondence relation are related to motion tracks of the extracted facial features comprises that;

a match between the motion track of each of the extracted facial features and a respective one of the common facial motion tracks is larger than or equal to a match value corresponding to the extracted facial features.

2. The method of claim 1, wherein, the facial features extracted in the step B at least represent five facial parts of the game player, and each of the facial parts is represented by a plurality of feature vectors;

obtaining a motion track of each of the extracted facial features by motion directions of the extracted facial feature that are sequentially acquired for N times in the step B comprises;

with respect to each of the extracted facial parts, sequentially acquiring feature vectors representing the extracted facial part for N times, and determining the motion track of the facial part according to the feature vectors representing the facial part that are acquired for N times.

3. The method of claim 1, wherein, in the step C1, when the correspondence relation is not present in set of correspondence relations, the method further comprises:

proceeding to the capturing in the step A.

7. A non-transitory machine-readable storage medium on which an instruction set is stored, wherein the instruction set, when executed, is configured to instruct a machine to execute a method according to claim 1, the steps of the method comprising:

A;

capturing a video image related to a game player in a game, and determining whether the captured video image contains a facial image of the game player, and if so, executing a step B;

otherwise, returning to the step A;

B;

extracting facial features from the facial image, and obtaining a motion track of each of the extracted facial features by motion directions of the extracted facial features that are sequentially acquired for N times;

wherein N is greater than or equal to 2; and

C;

determining a corresponding facial expression according to the motion track of the facial feature which is obtained in step B, and automatically playing the determined facial expression on a virtual image of the game player,

and further comprising;

obtaining, by the machine, a set of correspondence relations between facial expressions and common facial motion tracks through pre-training;

wherein, the step C comprises;

C1;

searching a correspondence relation from the set of correspondence relations, wherein the common facial motion tracks of the correspondence relation are related to motion tracks of the extracted facial features; and

if the correspondence relation is searched, then executing a step C2;

C2;

automatically playing the facial expression in the searched correspondence relation on the virtual image of the game player;

wherein, in the step C1, the fact that the common facial motion tracks of the correspondence relation are related to motion tracks of the extracted facial features comprises that;

a match between the motion track of each of the extracted facial features and a respective one of the common facial motion tracks is larger than or equal to a match value corresponding to the extracted facial features.

4. A device for automatically playing an expression on a virtual image, comprising:

a capture unit, a track determination unit and an expression play unit, wherein,

the capture unit is configured to capture a video image related to a game player in a game, and determine whether the captured video image contains a facial image of the game player, and if so, send a notification to the track determination unit;

otherwise, continue to capture the video image related to the game player in the game;

the track determination unit is configured to extract facial features from the facial image after receiving the notification sent by the capture unit, and obtain a motion track of each of the extracted facial features by motion directions of the extracted facial feature that are sequentially acquired for N times;

where N is greater than or equal to 2; and

the expression play unit is configured to determine a corresponding facial expression according to the motion track of the facial feature which is obtained by the track determination unit, and automatically play the determined facial expression on the virtual image of the game player,

wherein, the expression play unit comprises;

a match module, which is configured to search a correspondence relation from the set of correspondence relations, wherein the common facial motion tracks of the correspondence relation are related to the motion tracks of the extracted facial features; and

if the correspondence relation is searched, then send a notification to an expression play module;

wherein the set of correspondence relations are obtained through pre-training; and

the expression play module, which is configured to play the facial expression in the searched correspondence relation on the virtual image of the game player;

wherein, the fact that the common facial motion tracks of the correspondence relation are related to motion tracks of the extracted facial features comprises that;

a match between the motion track of each of the extracted facial features and a respective one of the common facial motion tracks is larger than or equal to a match value corresponding to the extracted facial features.

5. The device of claim 4, wherein, the track determination unit comprises:

a feature extraction module, which is configured to extract the facial features from the facial image captured by the capture unit, wherein the extracted facial features at least represent five facial parts of the game player, and each of the facial parts is represented by a plurality of feature vectors;

a track determination module, which is used to;

with respect to each of the facial parts extracted by the feature extraction module, sequentially acquire feature vectors representing the facial part for N times; and

determine the motion track of the facial part according to the feature vectors representing the facial part that are sequentially acquired for N times.

6. The device of claim 4, wherein, the match module is further configured to trigger the capture unit to continue to capture the video image related to the game player in the game when the correspondence relation, is not present in set of correspondence relations.