Vehicle operating system using motion capture

• US 9,547,373 B2
• Filed: 10/15/2015
• Issued: 01/17/2017
• Est. Priority Date: 03/16/2015
• Status: Expired due to Fees

• Current Assignee: Thunder Power New Energy Vehicle Development Company Limited
• Original Assignee: Thunder Power Hong Kong Limited (Thunder Power Holdings Ltd.)
• Inventors: Hsiao, Jen-Chieh, Chen, Yong-Syuan
• Primary Examiner(s): Li, Tracy Y
• Assistant Examiner(s): Rahman, Mohammad J
• Application Number:
  US14/883,621
• Publication Number:
  US 20160274668A1
• Time in Patent Office: 460 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: B60K 2360/1438 Touch screens
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/207 using multi-purpose displays, e.g. camera image and navigation or video on same display
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/8006 for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Vehicle operating systems for operating a vehicle having a driving seat for a vehicle driver and at least one passenger seat for passengers are described. The vehicle operating system may include one or more camera devices for shooting images of hand actions of the driver or images of hand actions of a passenger, and a storage device for storing operating signals corresponding to hand actions. A processing device may be configured to select the driver or the passengers as a gesture command operator, and to control the camera device to shoot hand action images of the selected gesture command operator. The command system may also be configured to convert the shot hand action images into corresponding operating signals according to hand action indicia stored in the storage device. The operating signals may be sent to execution devices, that execute the corresponding operations.

Claims

1. A vehicle operating system for operating a vehicle including a driving seat for a vehicle driver and at least one passenger seat for passengers, the vehicle operating system comprising:

camera devices for capturing at least one of images of gestures of the driver and images of gestures of a passenger;

a storage device for storing operating signals corresponding to gesture actions;

a processing device configured to;

control the camera devices to capture gesture action images of the driver and passenger simultaneously or substantially simultaneously,

convert the captured gesture action images into corresponding operating signals according to the operating signals corresponding to the gesture actions stored in the storage device,

determine a first operational signal is from a gesture action image for the driver and determine a second operational signal is from a gesture action image for the passenger, the first and second operational signals both for operating a same component of the vehicle,

determine whether the first operation signal is consistent with the second operation signal,

select the first operational signal as the operating signal and discard the second operational signal by virtue of the first operational signal being from the driver in response to the determination that first operation signal is not consistent with the second operation signal, and

send out the operating signals; and

execution devices configured to receive the operating signals sent by the processing device, and to execute operations corresponding to the operating signals.

2. The vehicle operating system of claim 1, wherein the same component being operated by first and second operational signals is not an in-vehicle camera.

3. The vehicle operating system of claim 2, wherein the same component being operated by first and second operational signals is not an in-vehicle camera includes an in vehicle air-conditioning system.

4. The vehicle operating system of claim 3, wherein the first operational signal is for navigating one or more options for the component.

5. The vehicle operating system of claim 1, wherein the first operational signal is for selecting an operational mode for the component.

6. The vehicle operating system of claim 1, wherein the camera devices are configured to capture an image from a body locating point of a corresponding gesture command operator to a hand of the gesture command operator, and wherein the processing device is configured to check whether the gesture image and the body locating point image are image-continuous, and if so, the gesture actions are converted into the operating signals.

7. The vehicle operating system of claim 6, wherein the body locating point comprises an arm, head or breast of the gesture command operator.

8. The vehicle operating system of claim 1, wherein each of the camera devices comprises a camera and a driving device for the camera, and the driving device configured for driving the camera to move or rotate.

9. The vehicle operating system of claim 8, wherein the driving device comprises a motor and a gear set.

10. The vehicle operating system of claim 1, wherein one of the camera devices is arranged in front of the driving seat.

11. The vehicle operating system of claim 1, wherein one of the camera devices is arranged at a different location providing a view of the back seat.

12. The vehicle operating system of claim 1, wherein the execution devices comprise at least one of a multimedia system, a door switch, a refill opening switch, Bluetooth equipment, a light switch and an air-conditioning system.

13. The vehicle operating system of claim 1, wherein,

the execution devices comprise a center console provided with at least one touch screen, the touch screen at least extends from the front part between the driving seat and the front passenger seat to the front part of the front passenger seat,

a touch screen operating interface is displayed on the touch screen, and

the center console can control the touch screen operating interface to move on the touch screen according to an operating signal.

14. The vehicle operating system of claim 1, wherein,

each operating signal corresponds to a two-step hand action,

a first step of the hand action is used for selecting an execution device, and

a second step of the hand action is used for instructing the selected execution device to carry out a specific operation.

15. An electric vehicle, comprising:

a driving seat for a vehicle driver;

at least one passenger seat for passengers; and

a vehicle operating system including;

camera devices for capturing gesture action images of the driver and passenger simultaneously or substantially simultaneously;

a storage device for storing operating signals corresponding to gesture actions;

a processing device configured to;

control the camera devices to capture gesture action images of the driver and passenger simultaneously or substantially simultaneously,

convert the captured gesture action images into corresponding operating signals based at least in part on a comparison between the gesture action images and parameters associated with the gesture actions,

determine a first operational signal is from a gesture action image for the driver and determine a second operational signal is from a gesture action image for the passenger, the first and second operational signals both for operating a same component of the vehicle,

determine whether the first operation signal is consistent with the second operation signal,

select the first operational signal as the operating signal and discard the second operational signal by virtue of the first operational signal being from the driver in response to the determination that first operation signal is not consistent with the second operation signal, and

send the operating signals; and

an execution device configured to receive the operating signals sent by the processing device, and to execute operations corresponding to the operating signals.

16. A method for operating a vehicle including a driving seat for a vehicle driver and at least one passenger seat for passengers, the method comprising:

storing operating signals corresponding to gesture actions;

controlling the camera devices to capture gesture action images of the driver and passenger simultaneously or substantially simultaneously;

converting the captured gesture action images into corresponding operating signals according to the operating signals corresponding to the gesture actions stored in the storage device;

determining a first operational signal is from a gesture action image for the driver and determine a second operational signal is from a gesture action image for the passenger, the first and second operational signals both for operating a same component of the vehicle;

determining whether the first operation signal is consistent with the second operation signal; and

selecting the first operational signal as the operating signal and discard the second operational signal by virtue of the first operational signal being from the driver in response to the determination that first operation signal is not consistent with the second operation signal.

17. The method of claim 16, wherein the same component being operated by first and second operational signals is not an in-vehicle camera.

18. The method of claim 16, wherein the same component being operated by first and second operational signals is not an in-vehicle camera includes an in vehicle air-conditioning system.

19. The method of claim 16, wherein the first operational signal is for navigating one or more options for the component.

20. The method of claim 16, wherein the first operational signal is for selecting an operational mode for the component.