VEHICLE OPERATING SYSTEM USING MOTION CAPTURE

• US 20170235371A1
• Filed: 11/21/2016
• Published: 08/17/2017
• Est. Priority Date: 03/16/2015
• Status: Active Grant

• Current Assignee: Thunder Power New Energy Vehicle Development Company Limited
• Original Assignee: Thunder Power Hong Kong Limited (Thunder Power Holdings Ltd.)
• Inventors: Hsiao, Jen-Chieh, Chen, Yong-Syuan
• Granted Patent: US 10,281,989 B2
• Time in Patent Office: Days
• CPC Class Codes: B60K 2360/1438 Touch screens
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2370/1438 Touch screens
• CPC Class Codes: B60K 2370/146 Input by gesture
• CPC Class Codes: B60K 2370/152 Displays
• CPC Class Codes: B60K 2370/21 using cameras
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/207 using multi-purpose displays, e.g. camera image and navigation or video on same display
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/8006 for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Vehicle operating systems for operating a vehicle having a driving seat for a vehicle driver and at least one passenger seat for passengers are described. The vehicle operating system may include one or more camera devices for shooting images of hand actions of the driver or images of hand actions of a passenger, and a storage device for storing operating signals corresponding to hand actions. A processing device may be configured to select the driver or the passengers as a gesture command operator, and to control the camera device to shoot hand action images of the selected gesture command operator. The command system may also be configured to convert the shot hand action images into corresponding operating signals according to hand action indicia stored in the storage device. The operating signals may be sent to execution devices, that execute the corresponding operations.

Claims

1. A vehicle operating system for operating a vehicle having multiple users inside the vehicle, the vehicle operating system comprising:

a camera device for capturing at least one image for each of the multiple users;

a storage device for storing operating signals corresponding to gesture actions;

a processing device configured to

control the camera device to capture gesture action images for the multiple users simultaneously or substantially simultaneously,

convert the captured gesture action images into corresponding operating signals according to the operating signals corresponding to the gesture actions stored in the storage device

select a first operational signal from a first user from the operating signals, and

send out the first operating signal; and

execution devices configured to receive the operating signals sent by the processing device, and to execute a first operation corresponding to the first operating signal.

2. The vehicle operating system of claim 1, wherein the processing device switches the gesture command operator from the driver to a passenger or from a passenger to the driver according to a switch instruction.

3. The vehicle operating system of claim 2, wherein the switch instruction is a hand action of the gesture command operator.

4. The vehicle operating system of claim 1, wherein the camera device captures an image from a body locating point of the gesture command operator to his hand, the processing device checks whether the gesture image and the body locating point image are image-continuous, and if so, the gesture actions are converted into the operating signals.

5. The vehicle operating system of claim 4, wherein the body locating point comprises an arm, head or breast of the gesture command operator.

6. The vehicle operating system of claim 1, wherein the camera device comprises a camera and a driving device for the camera, and the driving device configured for driving the camera to move or rotate.

7. The vehicle operating system of claim 6, wherein the driving device comprises a motor and a gear set.

8. The vehicle operating system of claim 1, wherein the camera device is arranged in front of the driving seat.

9. The vehicle operating system of claim 1, including a plurality of camera devices, at least one of the plurality of camera devices being arranged in front of the driving seat, and at least one camera device being arranged at a different location providing a view of the back seat.

10. The vehicle operating system of claim 1, wherein the execution devices comprise at least one of a multimedia system, a door switch, a refill opening switch, Bluetooth equipment, a light switch and an air-conditioning system.

11. The vehicle operating system of claim 1, wherein,

the execution devices comprise a center console provided with at least one touch screen, the touch screen at least extends from the front part between the driving seat and the front passenger seat to the front part of the front passenger seat,

a touch screen operating interface is displayed on the touch screen, and

the center console can control the touch screen operating interface to move on the touch screen according to an operating signal.

12. The vehicle operating system of claim 1, wherein,

each operating signal corresponds to a two-step hand action,

a first step of the hand action is used for selecting an execution device, and

a second step of the hand action is used for instructing the selected execution device to carry out an specific operation.

13. An electric vehicle, comprising:

a driving seat for a vehicle driver;

at least one passenger seat for passengers; and

a vehicle operating system including;

one or more camera devices for capturing at least one of images of gestures of the driver or images of gestures of a passenger;

a storage device for storing operating signals corresponding to gesture actions;

a processing device configured to

control the camera device to capture gesture action images for the multiple users simultaneously or substantially simultaneously,

convert the captured gesture action images into corresponding operating signals according to the operating signals corresponding to the gesture actions stored in the storage device

select a first operational signal from a first user from the operating signals, and

send out the first operating signal; and

an execution device configured to receive the operating signals sent by the processing device, and to execute a first operation corresponding to the first operating signal.

14. A vehicle operating method, comprising:

storing operating signals corresponding to hand actions in a storage device;

selecting one of multiple users in the vehicle as a gesture command operator;

shooting an image of a hand action of the selected gesture command operator;

converting the shot hand action image into a corresponding operating signal according to the stored operating signals corresponding to the hand actions;

select a first operational signal from a first user from the operating signals; and

executing, by an execution device, the corresponding operation according to the converted operating signal such that a first operation corresponding to the first operating signal is executed.

15. The vehicle operating method of claim 14, further comprising switching the gesture command operator from the driver to a passenger or from a passenger to the driver according to a switch instruction.

16. The vehicle operating method of claim 15, wherein the switch instruction is a hand action of the gesture command operator.

17. The vehicle operating method of claim 14, further comprising:

shooting an image from a body locating point of the gesture command operator to his hand, checking whether the shot hand image and the body locating point image are image-continuous, and if so, converting the hand action into the operating signal.

18. The vehicle operating system of claim 14, wherein,

each operating signal corresponds to a two-step hand action,

a first step of the hand action is used for selecting an execution device, and

a second step of the hand action is used for instructing the selected execution device to carry out an specific operation.