WEARABLE APPARATUS AND INFORMATION PROCESSING METHOD AND DEVICE THEREOF

• US 20170262696A1
• Filed: 02/05/2016
• Published: 09/14/2017
• Est. Priority Date: 09/22/2015
• Status: Active Grant

• Current Assignee: BOE Technology Group Co., Ltd.
• Original Assignee: BOE Technology Group Co., Ltd.
• Inventors: ZHANG, Jingyu
• Granted Patent: US 10,325,144 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A wearable apparatus, including: an image collector (11), configured to acquire facial images and body images of an interaction partner who is interacting with the user of the apparatus; a controller (12) connected with the image collector and configured to analyze the facial images and the body images to achieve analysis results; and an output device (13) connected with the controller and configured to output the analysis results. The wearable apparatus is portable and of low cost, and capable of determining the psychological situation of the current interaction partner according to the images information of the interaction partner based on simple operations. An information processing method and an information processing apparatus of the wearable apparatus are also provided.

Claims

1. A wearable apparatus comprising:

an image collector configured to acquire facial images and body images of an interaction partner who is interacting with the user of the apparatus;

a controller connected with the image collector and configured to analyze the facial images and the body images to achieve analysis results; and

an output device connected with the controller and configured to output the analysis results.

2. The apparatus of claim 1, wherein the image collector is a first camera to acquire facial images and body images of the interaction partner;

or

the image collector comprises a second camera to acquire body images of the interaction partner and a receiver to receive facial images of the interaction partner.

3. The apparatus of claim 1, wherein the first camera and/or the second camera are/is binocular camera(s).

4. The apparatus of claim 1, wherein the output device is a display screen for display of analysis results;

or the output device is a voice displayer for playback of analysis results in voice.

5. The apparatus of claim 1, wherein the wearable apparatus is intelligent glasses.

6. The apparatus of claim 5, wherein the intelligent glasses further comprise a glasses body and the image collector, controller and output device are all disposed on the glasses body.

7. The apparatus of claim 1, wherein the analysis results comprise an analysis result of behavior obtained through analyzing the body images as well as an analysis result of heart rate and an analysis result of pupil obtained by analyzing the facial images.

8. The apparatus of claim 1, wherein the wearable apparatus further comprises a selecting device, which is connected with the controller and configured to receive a selecting instruction from the user and send it to the controller so that the controller may obtain analysis results corresponding to the selecting instruction.

9. The apparatus of claim 8, wherein the selecting device comprises a first selecting unit to receive an analysis instruction for interest from the user and a second selecting unit to receive an analysis instruction for credibility from the user.

10. The apparatus of claim 1, wherein the wearable apparatus further comprises:

a wireless module connected with the controller and operable to communicate with network equipments through a wireless network; and

/or

a GPS module connected with the controller and operable to locate the wearable apparatus; and

/or

an image collector connected with the controller and operable to acquire facial images of the user of the wearable apparatus and a transmitter connected with the controller and operable to transmit the images acquired by the image collector.

11. The apparatus of claim 1, wherein the wearable apparatus comprises a head-mounted wearable apparatus.

12. The apparatus of claim 1, wherein the interaction partner comprises a conversation partner.

13. An information processing method for a wearable apparatus, comprising:

acquiring image information of the interaction partner who is interacting with the user of the wearable apparatus, the information comprising facial images and body images of the interaction partner;

analyzing the body images to achieve an analysis result of behavior and analyzing the facial images to achieve an analysis result of heart rate and an analysis result of pupil;

determining an output result of the conversation contents given by the interaction partner in accordance with the analysis result of behavior, analysis result of heart rate and analysis result of pupil; and

outputting the output result of the conversation contents to the user of the wearable apparatus.

14. The method of claim 13, wherein, before the body images are analyzed, the method further comprises the step of receiving a selecting instruction;

if the received selecting instruction is an analysis instruction for interest, the analysis results of behavior, heart rate and pupil are respectively analysis results of behavior, heart rate and pupil indicating whether there is interest in the interaction contents; and

if the received selecting instruction is an analysis instruction for credibility, the analysis results of behavior, heart rate and pupil are respectively analysis results of behavior, heart rate and pupil indicating the credibility rate of the interaction contents.

15. The method of claim 13, wherein the step of analyzing the body images to achieve an analysis result of behavior comprises:

comparing the body image at the current point in time with the body image at the first point in time; and

determining an analysis result of behavior indicating that the conversation partner is interested in the current interaction contents if the body image at the current point in time is nearer to the user relative to the body image at the first point in time;

wherein the first point in time is prior to the current point in time, and there is a preset interval between the two points.

16. The method of claim 13, wherein the step of analyzing the facial images to achieve analysis results of heart rate and pupil comprises:

analyzing the facial images using contactless pulse oximetry to achieve the analysis result of heart rate;

obtaining the pupil area in the facial image at the current point in time and the pupil area in the facial image at the second point in time; and

comparing the pupil area at the current point in time with the pupil area at the second point in time to achieve the analysis result of pupil;

wherein the second point in time is prior to the current point in time, and there is a preset interval between the two points.

17. The method of claim 13, wherein the step of determining an output result of the interaction contents given by the interaction partner in accordance with the analysis result of behavior, analysis result of heart rate and analysis result of pupil comprises:

determining the output result of the interaction contents of the interaction partner to be positive when at least two of the analysis result of behavior, analysis result of heart rate and analysis result of pupil are positive;

or

in accordance with predetermined weighting factors of individual analysis results, obtaining the output result of the interaction contents of the interaction partner by multiplying the analysis results with their own weighting factors and adding the products together;

or

obtaining the average value of all the analysis results and taking it as the output result of the interaction contents of the interaction partner.

18. The method of claim 13, wherein the wearable apparatus comprises a head-mounted wearable apparatus.

19. The method of claim 13, wherein the interaction partner comprises a conversation partner.

20. An information processing device for a wearable apparatus, the device comprising:

an acquiring unit configured to acquire image information of the interaction partner who is interacting with the user of the wearable apparatus, the information comprising facial images and body images of the interaction partner;

a processing unit configured to analyze the body images to achieve an analysis result of behavior and analyze the facial images to achieve analysis results of heart rate and pupil;

an analyzing unit configured to determine an output result of the interaction contents given by the interaction partner in accordance with the analysis result of behavior, analysis result of heart rate and analysis result of pupil; and

an output unit configured to output the output result of the interaction contents to the user of the wearable apparatus.

21. (canceled)

22. (canceled)

23. (canceled)

24. (canceled)

25. (canceled)

26. (canceled)