System for controlling an in-vehicle device using augmented reality and method thereof

• US 8,761,962 B2
• Filed: 11/29/2010
• Issued: 06/24/2014
• Est. Priority Date: 09/13/2010
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Original Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Inventors: Seok, Dong Cheol
• Primary Examiner(s): BEHNCKE, CHRISTINE M
• Application Number:
  US12/955,267
• Publication Number:
  US 20120065814A1
• Time in Patent Office: 1,303 Days
• Field of Search: 345/156, 382/104, 382/181, 382/190, 701/2, 701/1, 701/36, 701/48, 340/426.13, 340/426.14, 340/426.17, 340/426.16, 340/425.13, 700/245, 700/264, 715/771
• US Class Current: 701/2
• CPC Class Codes: B60K 2360/11 Instrument graphical user interfaces or menu aspects
• CPC Class Codes: B60K 2360/55 Remote control arrangements
• CPC Class Codes: B60K 2360/589 Wireless data transfers
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/213 Virtual instruments
• CPC Class Codes: B60K 35/80 Arrangements for controlling instruments
• CPC Class Codes: B60K 35/81 for controlling displays
• CPC Class Codes: B60K 35/85 Arrangements for transferring vehicle- or driver-related data
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

Provided are a system and method of controlling an in-vehicle device using augmented reality. A system for controlling an in-vehicle device using augmented reality includes a mobile device configured to identify a vehicle object unit as an image and receive a vehicle control command through implementation of the augmented reality of the image, and a driving control unit configured to transmit a vehicle type information to the mobile device and, upon receiving a command signal from the mobile device, to control the in-vehicle device that corresponds to the command signal. Accordingly, by remotely controlling an in-vehicle device by using augmented reality of a mobile device, user convenience may be improved.

Claims

1. A system for controlling an in-vehicle device using augmented reality, the system comprising:

a mobile device configured to identify a vehicle object unit as an image and receive a vehicle control command through implementation of the augmented reality of the image; and

a driving control unit configured to transmit vehicle type information to the mobile device and, upon receiving a command signal from the mobile device, control an in-vehicle device that corresponds to the command signal,

wherein the mobile device includes,

a camera configured to capture an image of the vehicle object unit;

a storage unit configured to store image pattern information of the vehicle object unit, and the vehicle type information; and

a control unit configured to, if the image pattern information stored in the storage unit matches pattern information of the captured image, enter into a vehicle remote control mode, provide an icon for an input of a command, and, if the vehicle control command is inputted through the icon, control transmission of the vehicle control command to the driving control unit,

wherein, when the image of the vehicle object unit is captured, the control unit is configured to divide the captured image of the vehicle object unit into a reference area and a non-reference area according to the vehicle type information, and to compare the image pattern information stored in the storage unit with pattern information of the reference area of the captured image, and

wherein, when the vehicle object unit is a center fascia of a vehicle, the control unit is configured to determine an air conditioner grill area as the reference area.

2. The system according to claim 1, wherein the in-vehicle device is an air conditioner, an audio, a radio, a digital multimedia broadcasting (DMB), a navigation unit, an interior lighting device or a window.

3. The system according to claim 1, wherein, when the mobile device sends a request for the vehicle type information to a vehicle, the driving control unit is configured to transmit the vehicle type information of the vehicle to the mobile device.

4. The system according to claim 1, wherein, when the mobile device approaches within a predetermined distance from the vehicle, the driving control unit is configured to transmit the vehicle type information of the vehicle to the mobile device.

5. A system for controlling an in-vehicle device using augmented reality, the system comprising:

a mobile device configured to identify a vehicle object unit as an image and receive a vehicle control command through implementation of the augmented reality of the image; and

a driving control unit configured to transmit vehicle type information to the mobile device and, upon receiving a command signal from the mobile device, control an in-vehicle device that corresponds to the command signal,

wherein the mobile device includes,

a camera configured to capture an image of the vehicle object unit;

a storage unit configured to store image pattern information corresponding to the vehicle object unit and coordinate information of an operation button positioned on the image; and

a control unit configured to, if the image pattern information stored in the storage unit matches pattern information of the captured image, enter into a vehicle remote control mode, and, if the operation button on the captured image is touched, to identify the vehicle control command by comparing coordinate information of a touched position with coordinate information stored in the storage unit so as to control transmission of the vehicle control command to the driving control unit.

6. A method of controlling an in-vehicle device using augmented reality, the method comprising:

pre-storing in a mobile device image pattern information of a center fascia by a vehicle type;

receiving, by the mobile device, vehicle type information from the vehicle;

capturing an image of a vehicle object unit that is placed on a screen of the mobile device;

comparing pattern information of the image captured by the mobile device that varies according to the vehicle type information with the image pattern information pre-stored in the storage unit;

entering into a vehicle remote control mode with the mobile device when the pattern information of the captured image matches the pre-stored image pattern information; and

displaying on the screen of the mobile device a device control command icon of the vehicle.

7. The method according to claim 6, wherein the step of receiving the vehicle type information includes transmitting, by the vehicle, the vehicle type information of the vehicle to the mobile device when the mobile device is within a predetermined distance from the vehicle.

8. The method according to claim 6, wherein the step of receiving the vehicle type information includes receiving, by the mobile device, the vehicle type information of the vehicle upon sending a request for the vehicle type information to the vehicle.

9. The method according to claim 6, wherein, the step of comparing includes,

dividing the captured image into a reference area and a non-reference area based on the vehicle type information;

and

comparing pattern information of the reference area of the captured image with the pre-stored image pattern information.

10. The method according to claim 9, wherein, if the vehicle object unit is the center fascia, an air conditioner grill unit is the reference area and a central portion is the non-reference area.

11. The method according to claim 9, wherein the in-vehicle device is an air conditioner, an audio, a radio, a digital multimedia broadcasting (DMB), a navigation unit, an interior lighting device or a window.

12. The method according to claim 9, wherein the step of receiving the in-vehicle device control command includes comparing, when the image of the vehicle object unit displayed on the screen of the mobile device is touched, coordinate information of a touched position with the pre-stored coordinate information of the image of the vehicle object unit, to identify a vehicle control command.

13. The method according to claim 6, further comprising:

wirelessly transmitting and receiving, when the device control command icon of the vehicle is selected, a command signal corresponding to the icon to the driving control unit; and

controlling the in-vehicle device according to the command signal received by the driving control unit.

14. The method according to claim 6, wherein, the step of comparing pattern information includes,

dividing the captured image into a reference area and a non-reference area based on the vehicle type information;

and

comparing pattern information of the reference area of the captured image with the pre-stored image pattern information.

15. A method of controlling an in-vehicle device using augmented reality, the method comprising:

pre-storing in a mobile device image pattern information of a vehicle object unit by a vehicle type, and coordinate information of an image of the vehicle object unit;

receiving, by the mobile device, the vehicle type information from the vehicle;

capturing the image of the vehicle object unit that is placed on a screen of the mobile device;

comparing pattern information of the image captured by the mobile device that varies according to the vehicle type with the pre-stored image pattern information;

entering, by the mobile device, into a vehicle remote control mode when the pattern information of the captured image matches the pre-stored image pattern information;

receiving an in-vehicle device control command through a touch on the image displayed on the screen of the mobile device; and

transmitting the in-vehicle device control command to the vehicle.

16. The method according to claim 15, wherein the step of receiving the vehicle type information includes transmitting, by the vehicle, the vehicle type information of the vehicle to the mobile device when the mobile device is within a predetermined distance from the vehicle.

17. The method according to claim 15, wherein the step of receiving the vehicle type information includes receiving, by the mobile device, the vehicle type information of the vehicle upon sending a request for the vehicle type information to the vehicle.