Wearable device and vehicle diagnosis apparatus including the same

• US 10,360,452 B2
• Filed: 11/22/2016
• Issued: 07/23/2019
• Est. Priority Date: 04/26/2016
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group), Kia Motors Corporation (Kia Corp.)
• Original Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Inventors: Kim, Hyuk Lae, Choi, Jae Hun, Kim, Sechang, Yoo, Ho Jong, Kim, Dong Il, Cho, Yong Sun
• Primary Examiner(s): Nguyen, Kevin M
• Application Number:
  US15/358,931
• Publication Number:
  US 20170308751A1
• Time in Patent Office: 973 Days
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G07C 2205/02 using a vehicle scan tool
• CPC Class Codes: G07C 5/0808 Diagnosing performance data testing of vehicles G01M17/00; testing of electrical installation on vehicles G01R31/005
• CPC Class Codes: G07C 5/0825 using optical means
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

This disclosure provides a wearable device providing a user with information required for diagnosis and repair of a vehicle using an augmented reality image and receiving a control instruction of the user by a gesture, eye gaze, or speech and a vehicle diagnosis apparatus including the same. The vehicle diagnosis apparatus includes a wearable device configured to acquire a forward image of a user; a controller configured to recognize at least one selected from the group consisting of a gesture and an eye gaze of the user from the forward image and determine a part of a vehicle corresponding to a position of the recognized gesture or the recognized eye gaze; and a communication device configured to transmit a diagnostic trouble code (DTC) request message about the determined part to the vehicle and receive a DTC about the determined part from the vehicle.

Claims

1. A vehicle diagnosis apparatus comprising:

a wearable device configured to acquire a forward image of a user;

a controller configured to recognize at least one selected from the group consisting of a gesture and an eye gaze of the user from the forward image and determine a part of a vehicle corresponding to a position of the recognized gesture or the recognized eye gaze; and

a communication device configured to transmit a diagnostic trouble code (DTC) request message about the determined part to the vehicle and receive a DTC about the determined part from the vehicle,

wherein the wearable device outputs an augmented reality image to the user,

wherein the controller determines a part located at a position corresponding to the recognized gesture or the recognized eye gaze as a target part of malfunction diagnosis,

wherein the communication device transmits the DTC request message about the target part of malfunction diagnosis to the vehicle and receives the DTC about the target part of malfunction diagnosis from the vehicle,

wherein the controller determines the target part of malfunction diagnosis based on the DTC, and

wherein the wearable device outputs and displays a position corresponding to each of DTCs.

2. The vehicle diagnosis apparatus according to claim 1, wherein the controller recognizes one from a plurality of parts comprised in the vehicle from the forward image and transmits information about the recognized part to the wearable device.

3. The vehicle diagnosis apparatus according to claim 2, wherein the wearable device outputs the augmented reality image comprising the information about the recognized part.

4. The vehicle diagnosis apparatus according to claim 3, wherein the wearable device outputs the augmented reality image overlaying information about the recognized part on a position of the recognized part.

5. The vehicle diagnosis apparatus according to claim 1, further comprising a storage configured to store a plurality of single parts comprised in the vehicle in groups classified according to a plurality of systems.

6. The vehicle diagnosis apparatus according to claim 5, wherein the controller recognizes an eye gaze of the user from the forward image, determines a system of the vehicle corresponding to a position of the recognized eye gaze, recognizes a gesture of the user from the forward image, and determines a single part corresponding to a position of the recognized gesture among single parts comprised in the determined system.

7. The vehicle diagnosis apparatus according to claim 5, wherein the controller determines a single part located at a position corresponding to the recognized gesture or the recognized eye gaze and determines a system comprising the determined single part as a target of malfunction diagnosis.

8. The vehicle diagnosis apparatus according to claim 1, wherein the wearable device receives an input speech of the user and converts the input speech into an electric speech signal, and

the controller recognizes a command comprised in the speech signal and determines a part corresponding to the recognized command as the target part of malfunction diagnosis.

9. The vehicle diagnosis apparatus according to claim 1, wherein the wearable device outputs information to guide a repair of the part indicated by the DTC as an augmented reality image.

10. The vehicle diagnosis apparatus according to claim 9, wherein the wearable device outputs an augmented reality image overlaying information to guide the repair of the part indicated by the DTC on a position of the part.

11. The vehicle diagnosis apparatus according to claim 9, wherein when the information to guide the repair of the part comprises a plurality of stages, the controller recognizes a gesture of the user from the forward image and transmits a control signal to output information of a next stage among the plurality of stages to the wearable device when the recognized gesture is a prestored gesture.

12. The vehicle diagnosis apparatus according to claim 1, wherein the wearable device outputs the augmented reality image overlaying a predetermined color on a positon of the part indicated by the DTC.

13. The vehicle diagnosis apparatus according to claim 1, wherein the controller recognizes a gesture of the user from the forward image and transmits a control signal to remove the augmented reality image to the wearable device when the recognized gesture is a prestored gesture.

14. A wearable device comprising:

an imaging device configured to acquire a forward image of a user;

a controller configured to recognize at least one selected from the group consisting of a gesture and an eye gaze of the user from the forward image and determine a part of a vehicle corresponding to a position of the recognized gesture or the recognized eye gaze; and

a communication device configured to transmit a diagnostic trouble code (DTC) request message about the determined part to the vehicle and receive a DTC about the determined part from the vehicle,

wherein the wearable device further comprises a display configured to output an augmented reality image overlaying a virtual image on a space at which the user gazes,

wherein the controller determines a part located at a position corresponding to the recognized gesture or the recognized eye gaze as a target part of malfunction diagnosis,

wherein the controller determines the target part of malfunction diagnosis based on the DTC, and

wherein the wearable device outputs and displays a position corresponding to each of DTCs.

15. The wearable device according to claim 14, further comprising a storage configured to store a plurality of single parts comprised in the vehicle in groups classified according to a plurality of systems.

16. The wearable device according to claim 15, wherein the controller recognizes an eye gaze of the user from the forward image, determines a system of the vehicle corresponding to a position of the recognized eye gaze, recognizes a gesture of the user from the forward image, and determines a single part corresponding to a position of the recognized gesture among single parts comprised in the determined system.

17. The wearable device according to claim 15, wherein the controller determines a single part located at a position corresponding to the recognized gesture or the recognized eye gaze and determines a system comprising the determined single part as a target of malfunction diagnosis.

18. The wearable device according to claim 14, further comprising a microphone configured to receive an input speech of the user and convert the input speech into an electric speech signal,

wherein the controller recognizes a command comprised in the speech signal and determines a part corresponding to the recognized command as a target part of malfunction diagnosis.

19. The wearable device according to claim 14, wherein the display outputs information to guide a repair of the part indicated by the DTC as the augmented reality image.

20. The wearable device according to claim 19, wherein the display outputs the augmented reality image overlaying information to guide the repair of the part indicated by the DTC on a position of the part.

21. The wearable device according to claim 19, wherein when the information to guide the repair of the part comprises a plurality of stages, the controller recognizes a gesture of the user from the forward image and transmits a control signal to output information of a next stage among the plurality of stages to the wearable device when the recognized gesture is a prestored gesture.

22. The wearable device according to claim 14, wherein the display outputs an augmented reality image overlaying a predetermined color on a positon of the part indicated by the DTC.

23. The wearable device according to claim 14, wherein the controller recognizes a gesture of the user from the forward image and transmits a control signal to remove the augmented reality image to the wearable device when the recognized gesture is a prestored gesture.