METHOD AND DEVICE FOR AUTOMATICALLY PLAYING EXPRESSION ON VIRTUAL IMAGE

• US 20150038222A1
• Filed: 03/25/2013
• Published: 02/05/2015
• Est. Priority Date: 04/06/2012
• Status: Active Grant

• Current Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Original Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Inventors: Wang, Dong
• Granted Patent: US 9,457,265 B2
• Time in Patent Office: Days
• US Class Current: 463/31
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/176 Dynamic expression

Abstract

Provided are a method and device for automatically playing an expression on a virtual image. The method includes the steps of: A, capturing a video image related to a game player in a game, and determining whether the captured video image contains a facial image of the game player, and if so, executing a step B; otherwise, returning to the step A; B: extracting facial features from the facial image, and obtaining a motion track of each of the facial features by motion directions of the extracted facial feature that are acquired for N times, where N is greater than or equal to 2; and C, determining a corresponding facial expression according to the motion track of the facial feature which is obtained in step B, and automatically playing the determined facial expression on a virtual image of the game player.

Claims

1. A method for automatically playing an expression on a virtual image, comprising steps of:

A;

capturing a video image related to a game player in a game, and determining whether the captured video image contains a facial image of the game player, and if so, executing a step B;

otherwise, returning to the step A;

B;

extracting facial features from the facial image, and obtaining a motion track of each of the extracted facial features by motion directions of the extracted facial features that are sequentially acquired for N times;

wherein N is greater than or equal to 2; and

C;

determining a corresponding facial expression according to the motion track of the facial feature which is obtained in step B, and automatically playing the determined facial expression on a virtual image of the game player.

2. The method of claim 1, wherein, the facial features extracted in the step B at least represent five facial parts of the game player, and each of the facial parts is represented by a plurality of feature vectors;

obtaining a motion track of each of the extracted facial features by motion directions of the extracted facial feature that are sequentially acquired for N times in the step B comprises;

with respect to each of the extracted facial parts, sequentially acquiring feature vectors representing the extracted facial part for N times, and determining the motion track of the facial part according to the feature vectors representing the facial part that are acquired for N times.

3. The method of claim 2, wherein, the step C comprises steps of:

C1;

matching the motion track of each of the extracted facial parts among a set of correspondence relations between common facial motion tracks obtained from pre-train and facial expressions; and

if a correspondence relation, the common facial motion tracks of which contain a motion track related to the motion track of each of the extracted facial parts, exists in the set of correspondence relations, then executing a step C2;

C2;

automatically playing the facial expression in the correspondence relation, the common facial motion tracks of which contain a motion track related to the motion track of each of the extracted facial parts, on the virtual image of the game player.

4. The method of claim 3, wherein, in the step C1, the fact that the common facial motion tracks of a correspondence relation contain a motion track related to the motion track of each of the extracted facial parts comprises that:

with respect to the motion track of each of the extracted facial parts, the common facial motion tracks of the correspondence relation contain a facial motion track, a match rate of which with the motion track of the extracted facial part is larger than or equal to a match value corresponding to the extracted facial part.

5. The method of claim 3, wherein, in the step C1, when the correspondence relation, the common facial motion tracks of which contain a motion track related to the motion track of each of the extracted facial parts, is not present in set of correspondence relations, the method further comprises:

proceeding to the capturing in the step A.

11. A non-transitory machine-readable storage medium on which an instruction set is stored, wherein the instruction set, when executed, is configured to instruct a machine to execute a method according to claim 1, the steps of the method comprising:

A;

capturing a video image related to a game player in a game, and determining whether the captured video image contains a facial image of the game player, and if so, executing a step B;

otherwise, returning to the step A;

B;

extracting facial features from the facial image, and obtaining a motion track of each of the extracted facial features by motion directions of the extracted facial features that are sequentially acquired for N times;

wherein N is greater than or equal to 2; and

C;

determining a corresponding facial expression according to the motion track of the facial feature which is obtained in step B, and automatically playing the determined facial expression on a virtual image of the game player.

6. A device for automatically playing an expression on a virtual image, comprising:

a capture unit, a track determination unit and an expression play unit, wherein,

the capture unit is configured to capture a video image related to a game player in a game, and determine whether the captured video image contains a facial image of the game player, and if so, send a notification to the track determination unit;

otherwise, continue to capture the video image related to the game player in the game;

the track determination unit is configured to extract facial features from the facial image after receiving the notification sent by the capture unit, and obtain a motion track of each of the extracted facial features by motion directions of the extracted facial feature that are sequentially acquired for N times;

where N is greater than or equal to 2; and

the expression play unit is configured to determine a corresponding facial expression according to the motion track of the facial feature which is obtained by the track determination unit, and automatically play the determined facial expression on the virtual image of the game player.

7. The device of claim 6, wherein, the track determination unit comprises:

a feature extraction module, which is configured to extract the facial features from the facial image captured by the capture unit, wherein the extracted facial features at least represent five facial parts of the game player, and each of the facial parts is represented by a plurality of feature vectors;

a track determination module, which is used to;

with respect to each of the facial parts extracted by the feature extraction module, sequentially acquire feature vectors representing the facial part for N times; and

determine the motion track of the facial part according to the feature vectors representing the facial part that are sequentially acquired for N times.

8. The device of claim 7, wherein, the expression play unit comprises:

a match module, which is configured to match the motion track of each of the extracted facial parts among a set of correspondence relations between common facial motion tracks obtained from pre-train and facial expressions; and

if a correspondence relation, the common facial motion tracks of which contain a motion track related to the motion track of each of the extracted facial parts, exists in the set of correspondence relations, then send the notification to the expression play module; and

an expression play module, which is configured to automatically play the facial expression in the correspondence relation, the common facial motion tracks of which contain a motion track related to the motion track of each of the extracted facial parts, on the virtual image of the game player after receiving the notification.

9. The device of claim 8, wherein, the fact that the common facial motion tracks of a correspondence relation contain a motion track related to the motion track of each of the extracted facial parts comprises that:

with respect to the motion track of each of the extracted facial parts, the common facial motion tracks of the correspondence relation contain a facial motion track, a match rate of which with the motion track of the extracted facial part is larger than or equal to a match value corresponding to the extracted facial part.

10. The device of claim 8, wherein, the match module is further configured to trigger the capture unit to continue to capture the video image related to the game player in the game when the correspondence relation, the common facial motion tracks of which contain a motion track related to the motion track of each of the extracted facial parts, is not present in set of correspondence relations.