Wearable apparatus and information processing method and device thereof

• US 10,325,144 B2
• Filed: 02/05/2016
• Issued: 06/18/2019
• Est. Priority Date: 09/22/2015
• Status: Active Grant

• Current Assignee: BOE Technology Group Co., Ltd.
• Original Assignee: BOE Technology Group Co., Ltd.
• Inventors: Zhang, Jingyu
• Primary Examiner(s): Vu, Kim Y
• Assistant Examiner(s): Delaney, Molly
• Application Number:
  US15/326,114
• Publication Number:
  US 20170262696A1
• Time in Patent Office: 1,229 Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A wearable apparatus, including: an image collector (11), configured to acquire facial images and body images of an interaction partner who is interacting with the user of the apparatus; a controller (12) connected with the image collector and configured to analyze the facial images and the body images to achieve analysis results; and an output device (13) connected with the controller and configured to output the analysis results. The wearable apparatus is portable and of low cost, and capable of determining the psychological situation of the current interaction partner according to the images information of the interaction partner based on simple operations. An information processing method and an information processing apparatus of the wearable apparatus are also provided.

Claims

1. A wearable apparatus comprising:

a camera configured to acquire facial images and body images of an interaction partner who is interacting with the user of the apparatus;

a central processing unit connected with the camera and configured to analyze the facial images and the body images, and determine a truthfulness or an interest of the interaction partner with respect to the interacting with the user to achieve analysis results; and

a display connected with the central processing unit and configured to output the analysis results;

wherein the wearable apparatus further comprises a selecting button, which is connected with the central processing unit and configured to receive a selecting instruction from the user and send it to the central processing unit so that the central processing unit may obtain analysis results corresponding to the selecting instruction,

wherein the analysis results comprise an analysis result of behavior obtained through analyzing the body images as well as an analysis result of heart rate and an analysis result of pupil obtained by analyzing the facial images.

2. The apparatus of claim 1, wherein the camera is a first camera to acquire facial images and body images of the interaction partner;

or

the camera comprises a second camera to acquire body images of the interaction partner and a receiver to receive facial images of the interaction partner.

3. The apparatus of claim 1, wherein the first camera and/or the second camera are/is binocular camera(s).

4. The apparatus of claim 1, wherein the display is a display screen for display of analysis results;

or the display is a voice displayer for playback of analysis results in voice.

5. The apparatus of claim 1, wherein the wearable apparatus is intelligent glasses.

6. The apparatus of claim 5, wherein the intelligent glasses further comprise a glasses body and the camera, central processing unit and display are all disposed on the glasses body.

7. The apparatus of claim 1, wherein the selecting device comprises a first selecting unit to receive an analysis instruction for interest from the user and a second selecting unit to receive an analysis instruction for credibility from the user.

8. The apparatus of claim 1, wherein the wearable apparatus further comprises:

a wireless device connected with the central processing unit and operable to communicate with network equipments through a wireless network; and

/or

a GPS device connected with the central processing unit and operable to locate the wearable apparatus; and

/or

a camera connected with the central processing unit and operable to acquire facial images of the user of the wearable apparatus and a transmitter connected with the central processing unit and operable to transmit the images acquired by the camera.

9. The apparatus of claim 1, wherein the wearable apparatus comprises a head-mounted wearable apparatus.

10. The apparatus of claim 1, wherein the interaction partner comprises a conversation partner.

11. An information processing method for a wearable apparatus, comprising:

acquiring image information of the interaction partner who is interacting with the user of the wearable apparatus, the information comprising facial images and body images of the interaction partner;

analyzing the body images to achieve an analysis result of behavior and analyzing the facial images to achieve an analysis result of heart rate and an analysis result of pupil;

determining an output result of the conversation contents given by the interaction partner in accordance with the analysis result of behavior, analysis result of heart rate and analysis result of pupil, wherein the output result is about a truthfulness or an interest of the interaction partner with respect to the interacting with the user; and

outputting the output result of the conversation contents to the user of the wearable apparatus;

wherein before the body images are analyzed, the method further comprises;

receiving a selecting instruction from a selecting button of the wearable apparatus;

determining the analysis result of behavior, the analysis result of heart rate and the analysis result of pupil according to the selecting instruction,

wherein if the received selecting instruction is an analysis instruction for interest, the analysis results of behavior, heart rate and pupil are respectively analysis results of behavior, heart rate and pupil indicating whether there is interest in the interaction contents; and

if the received selecting instruction is an analysis instruction for credibility, the analysis results of behavior, heart rate and pupil are respectively analysis results of behavior, heart rate and pupil indicating the credibility rate of the interaction contents.

12. The method of claim 11, wherein analyzing the body images to achieve an analysis result of behavior comprises:

comparing the body image at the current point in time with the body image at the first point in time; and

determining an analysis result of behavior indicating that the conversation partner is interested in the current interaction contents if the body image at the current point in time is nearer to the user relative to the body image at the first point in time;

wherein the first point in time is prior to the current point in time, and there is a preset interval between the two points.

13. The method of claim 11, wherein analyzing the facial images to achieve analysis results of heart rate and pupil comprises:

analyzing the facial images using contactless pulse oximetry to achieve the analysis result of heart rate;

obtaining the pupil area in the facial image at the current point in time and the pupil area in the facial image at the second point in time; and

comparing the pupil area at the current point in time with the pupil area at the second point in time to achieve the analysis result of pupil;

wherein the second point in time is prior to the current point in time, and there is a preset interval between the two points.

14. The method of claim 11, wherein determining an output result of the interaction contents given by the interaction partner in accordance with the analysis result of behavior, analysis result of heart rate and analysis result of pupil comprises:

determining the output result of the interaction contents of the interaction partner to be positive when at least two of the analysis result of behavior, analysis result of heart rate and analysis result of pupil are positive;

or

in accordance with predetermined weighting factors of individual analysis results, obtaining the output result of the interaction contents of the interaction partner by multiplying the analysis results with their own weighting factors and adding the products together;

or

obtaining the average value of all the analysis results and taking it as the output result of the interaction contents of the interaction partner.

15. The method of claim 11, wherein the wearable apparatus comprises a head-mounted wearable apparatus.

16. The method of claim 11, wherein the interaction partner comprises a conversation partner.

17. An information processing device for a wearable apparatus, the device comprising:

a camera configured to acquire image information of the interaction partner who is interacting with the user of the wearable apparatus, the information comprising facial images and body images of the interaction partner;

a central processing unit configured to;

analyze the body images to achieve an analysis result of behavior and analyze the facial images to achieve analysis results of heart rate and pupil; and

determine an output result of the interaction contents given by the interaction partner in accordance with the analysis result of behavior, analysis result of heart rate and analysis result of pupil, wherein the output result is about a truthfulness or an interest of the interaction partner with respect to the interacting with the user;

a display configured to output the output result of the interaction contents to the user of the wearable apparatus; and

a receiver operable to receive a selecting instruction of the user from a selecting button of the wearable apparatus before the body images and the facial images are analyzed by the central processing unit;

wherein the central processing unit is further configured to determine the analysis result of behavior, analysis result of heart rate and analysis result of pupil according to the selecting instruction,

wherein the processing unit is further configured to

analyze the facial images using contactless pulse oximetry to achieve an analysis result of heart rate;

obtain the pupil area in the facial image at the current point in time and the pupil area in the facial image at a second point in time; and

compare the pupil area at the current point in time with the pupil area at the second point in time to achieve an analysis result of pupil;

wherein the second point in time is prior to the current point in time, and there is a preset interval between the two points.