Universal autonomous robotic driving system

• US 10,613,544 B2
• Filed: 05/02/2016
• Issued: 04/07/2020
• Est. Priority Date: 05/05/2015
• Status: Active Grant

• Current Assignee: B.G. Negev Technologies And Applications Ltd (Ben Gurion University of The Negev)
• Original Assignee: B.G. Negev Technologies And Applications Ltd (Ben Gurion University of The Negev)
• Inventors: Guterman, Hugo, Yechiel, Oded
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Huynh, Luat T
• Application Number:
  US15/571,331
• Publication Number:
  US 20180284801A1
• Time in Patent Office: 1,436 Days
• Field of Search: 701 1
• CPC Class Codes: B25J 11/00 Manipulators not otherwise provided for
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G01C 21/34 Route searching; Route guidance
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0268 using internal positioning means

Abstract

A robotic system for driving a vehicle to a destination is configured for placement inside the vehicle, and it utilizes an imaging system for acquiring image data of at least a part of the interior of the vehicle and image data of external surroundings of the vehicle, and a control system for processing the image data acquired by the imaging system. The control system is configured to identify in the image data of at least part of the interior of the vehicle image data of the vehicle state indicators, and determine therefrom at least vehicle status data. The control system is adapted to determine from the image data of external surroundings road status data, and analyze the vehicle status data and the road status data for generating control instructions for driving the vehicle along the road.

Claims

1. A robotic system for driving a vehicle to a destination, the robotic system comprising:

a support configured for easily and quickly securing said robotic system to a driver seat of the vehicle;

an imaging system configured and operable to acquire image data of at least a part of the interior of the vehicle, and image data of external surroundings of the vehicle; and

a control system configured and operable to

process and analyze the image data acquired by the imaging system, to identify in the image data of the at least apart of the interior of the vehicle, image data of the vehicle state indicators and determine therefrom at least vehicle status data of said vehicle,

determine from the image data of external surroundings, road status data, and

analyze said at least vehicle status data and said road status data for generating control instructions for operating actuators of the vehicle and driving the vehicle along said road accordingly.

2. The system of claim 1, further comprising an actuators manipulating system configured and operable to couple to the actuators of the vehicle and to controllably change operational states thereof according to the control instructions generated by the control system.

3. The system of claim 2, wherein the support comprises a base assembly configured to accommodate the actuators manipulating system and to be secured to the driver seat of the vehicle.

4. The system of claim 3, wherein the actuators manipulating system comprises at least a brake pedal manipulating system, an accelerator pedal manipulating system, and a steering wheel manipulating system.

5. The system of claim 4, wherein the steering wheel manipulating system is elastically coupled to the support to permit movement of the steering wheel manipulating system in at least one of its translational axes.

6. The system of claim 4, wherein the actuators manipulating system further comprises at least one lever manipulating system configured and operable to actuate at least a gear system of the vehicle.

7. The system of claim 1 wherein the control system is configured and operable to receive location data indicative of a current location of the vehicle and road data, and determine a route for driving the vehicle to the destination based at least in part on said location and said road data.

8. The system of claim 7 comprising a positioning unit configured and operable to determine the current location of the vehicle.

9. The system of claim 7 comprising a wireless communication unit configured and operable for data communication with a remote computer system for receiving at least the road data via said wireless communication unit.

10. The system of claim 1, comprising a user interface unit for data communication with a user.

11. The system of claim 10 wherein the user interface unit comprises a data communication device configured and operable for communicating data with a user device, for receiving at least one of the location data and the road data via the data communication device.

12. The system of claim 10, comprising an audio input device for receiving audible instructions from a user via the user interface unit, said audible instructions being indicative of at least one of the destination and vehicle operating instructions.

13. The system of claim 10 comprising an audio output device for outputting audible information to the user via the user interface unit.

14. The system of claim 1, wherein the imaging system is configured for acquiring image data indicative of the state indicators on a control panel of the vehicle, and acquiring front image data indicative of front side external surroundings of the vehicle.

15. The system of claim 14, wherein the imaging system comprises at least two imager units, comprising:

a panel imager unit for acquiring said image data indicative of the state indicators on the control panel of the vehicle, and a front view imager for acquiring said front image data indicative of front side external surroundings of the vehicle.

16. The system of claim 14, wherein the imaging system comprises a back imager unit for acquiring rear-view image data of the external surroundings of the vehicle.

17. The system of claim 14, wherein the imaging system is further configured for acquiring image data indicative of at least one of left-view of the external surroundings of the vehicle and left-side rear-view reflected by a left-side mirror of the vehicle, and for acquiring image data indicative of at least one of right-view of the external surroundings of the vehicle and right-side rear-view reflected by a right-side mirror of the vehicle.

18. The system of claim 17, wherein the imaging system comprises a left imager unit for acquiring said image data indicative of at least one of left-view of the external surroundings of the vehicle and left-side rear-view reflected by the left-side mirror of the vehicle, and a right imager unit for acquiring said image data indicative of at least one of right-view of the external surroundings of the vehicle and right-side rear-view reflected by the right-side mirror of the vehicle.

19. The system of claim 1 wherein the support comprises a vertical support assembly configured and operable for accommodating the imaging and control systems and for securing the same to a back support of the driver seat of the vehicle.

20. A computer-implemented method of autonomously driving a vehicle to a destination, the method comprising:

securing a support of a robotic system to a driver seat of the vehicle;

using one or more processors and memories of said robotic system to carry out the following;

receiving image data from an imaging system of the robotic system, said imaging data being indicative of at least the state indicators of the vehicle and of external surroundings of the vehicle;

identifying in the received image data at least the states of said state indicators and objects in the external surroundings of the vehicle, and respectively generating vehicle status data and road status data based on the identified indicators states and the external objects; and

generating control instructions based at least in part on said vehicle status data and said road status data for operating actuators of the vehicle and driving the vehicle to said destination.

21. The method of claim 20 comprising receiving location data indicative of a current location of the vehicle and determining a route to the destination based on said location data.

22. The method of claim 20 wherein the identifying comprises detecting at least one of road lanes and road curves in the image data and generating lane data indicative thereof usable for navigating the vehicle on the driven road.

23. The method of claim 20 comprising tracking at least one of the identified objects and generating trajectory data indicative of a trajectory thereof, and using said trajectory data in at least one of the route determining and the generating of the control instructions.

24. A robotic system for driving a vehicle to a destination, said robotic system comprising a support configured and operable for easily and quickly securing said robotic system to a driver seat of the vehicle, said support comprising:

an imaging system configured and operable to acquire image data of external surroundings of the vehicle; and

a control system is configured and operable to process and analyze the image data and determine therefrom road status data, analyze said road status data, and generate control instructions based thereon for operating actuators of the vehicle and driving the vehicle to the destination.

25. The system of claim 24 comprising a positioning unit configured and operable to determine the current location of the vehicle.

26. The system of claim 25 comprising a wireless communication unit configured and operable for data communication with a remote computer system for receiving at least road data via said wireless communication unit.

27. The system of claim 24 comprising a wireless communication unit configured and operable for data communication with a remote computer system for receiving at least the road data via said wireless communication unit.

28. The system of claim 24, wherein the imaging system is configured for acquiring image data indicative of state indicators on a control panel of the vehicle, and wherein the control system is configured and operable to process the image data indicative of the vehicle state indicators and determine therefrom at least vehicle status data of said vehicle.

29. The system of claim 24 wherein the imaging system is configured and operable to acquire front-view and rear-view image data of the external surroundings of the vehicle.

30. The system of claim 24 wherein the imaging system is configured for acquiring image data indicative of at least one of left-view of the external surroundings of the vehicle and left-side rear-view reflected by a left-side mirror of the vehicle, and for acquiring image data indicative of at least one of right-view of the external surroundings of the vehicle and right-side rear-view reflected by a right-side mirror of the vehicle.