SYSTEMS AND METHODS FOR DRIVER ASSISTANCE

• US 20180308275A1
• Filed: 04/18/2018
• Published: 10/25/2018
• Est. Priority Date: 04/21/2017
• Status: Active Grant

• Current Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Inventors: Fortmann, Gustavo Adolfo
• Granted Patent: US 10,614,617 B2
• Time in Patent Office: Days
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/586 of parking space

Abstract

Embodiments are disclosed for generating a virtual view that is at least partially decoupled from a view of one or more physical cameras. An example method includes capturing images from at least one physical camera, generating a virtual image of a virtual camera from the captured images of at least one physical camera, the virtual image having either a different viewpoint, a different orientation, or a different viewpoint and orientation than the images captured by at least one physical camera, and automatically maintaining the virtual position of the virtual camera, relative to a reference point, when the at least one physical camera moves relative to the reference point.

Claims

1. A method, comprising

capturing images from at least one physical camera coupled to a movable object;

generating a virtual image of a virtual camera from the captured images of the at least one physical camera, the virtual image having either a different viewpoint, a different orientation, or a different viewpoint and orientation than the images captured by the at least one physical camera; and

automatically maintaining a virtual position of the virtual camera, relative to a reference point, when the at least one physical camera moves relative to the reference point.

2. The method of claim 1, wherein the movement of the at least one physical camera relative to the reference point is, directly or indirectly, measured in order to maintain the virtual position of the virtual camera relative to the reference point.

3. The method of claim 1, wherein an object or part of the object perceived by the at least one physical camera is used in order to maintain the virtual position of the virtual camera relative to the reference point.

4. The method of claim 1, wherein the at least one physical camera is fixedly mounted to the movable object, and wherein maintaining the virtual position of the virtual camera further comprises maintaining an orientation of the virtual camera.

5. The method of claim 1, wherein automatically maintaining the virtual position of the virtual camera, relative to the reference point, when the at least one physical camera moves relative to the reference point comprises automatically maintaining the virtual position of the virtual camera fixed in 3-D grounded space relative to earth while the at least one physical camera and movable object move together relative to the earth.

6. The method of claim 1, further comprising detecting a trigger for decoupling the virtual camera from a vehicle-locked position, and generating the virtual image responsive to detecting the trigger.

7. The method of claim 6, wherein the trigger is a first trigger, the method further comprising detecting a second trigger, and, responsive to detecting the second trigger, returning the virtual camera to a vehicle-locked position and displaying the virtual view of the vehicle from the virtual perspective of the virtual camera in the vehicle-locked position.

8. The method of claim 7, wherein the second trigger comprises determining that the world-locked position is no longer able to be generated from the images captured by the physical camera due to movement of the vehicle away from the world-locked position.

9. The method of claim 7, wherein the second trigger comprises determining that a speed of the vehicle exceeds a threshold.

10. The method of claim 6, wherein the trigger includes detecting a recognized object associated with a difficult vehicle maneuver.

11. The method of claim 6, wherein the trigger includes detecting an upcoming turn maneuver in a direction of travel of the vehicle.

12. The method of claim 6, wherein the trigger includes detecting a change in movement of a vehicle in a selected degree of freedom that is above a threshold.

13. The method of claim 6, wherein the trigger comprises user input.

14. A virtual view system comprising:

at least one physical camera positioned on an object;

a display device;

an image processing unit coupled to the at least one physical camera; and

a storage device storing instructions executable by the image processing unit to;

capture, via the at least one physical camera, image data from an associated physical perspective relative to the object;

generate, via the image processing unit using the image data from the at least one physical camera, a view of the object from a virtual position of a virtual camera, the virtual camera capturing the view from a virtual perspective that extends from the virtual position in a direction of interest, and the virtual camera being decoupled from the at least one physical camera under a first condition; and

display, via the display device, the view from the virtual perspective, the view appearing static as the at least one physical camera moves while under the first condition.

15. The virtual view system of claim 14, wherein the virtual camera is coupled to the at least one physical camera under a second condition, the view appearing to move with the object under the second condition.

16. The virtual view system of claim 14, wherein the at least one physical camera is a first physical camera of a plurality of physical cameras positioned on the object, each physical camera of the plurality of physical cameras having a field of view that at least partially overlaps with another physical camera of the plurality of physical cameras.

17. A virtual view system comprising:

a physical camera mounted on an object;

a display device;

a processor; and

a storage device storing instructions executable by the processor to;

capture, via the physical camera, image data from a plurality of perspectives relative to the object;

receive, from an image processing unit using the image data from the physical camera, a virtual view of the object from a virtual perspective of a virtual camera, the virtual camera having a virtual position from which the virtual perspective is captured, the virtual position remaining static relative to movement of the physical camera under a first condition and the virtual position moving in a coordinated manner with the plurality of physical cameras under a second condition; and

display, via the display device, the virtual view of the object, the virtual view appearing static in at least a first degree of freedom under the first condition and the virtual view appearing to move in at least the first degree of freedom with the object under the second condition.

18. The virtual view system of claim 17, wherein the image processing unit is included in a cloud computing system and wherein the virtual view is generated remotely from the virtual view system.

19. The virtual view system of claim 17, wherein the image processing unit is included in the object and wherein the virtual view is generated locally to the virtual view system.

20. The virtual view system of claim 17, wherein the object is a first part of a vehicle, said first part able to move independently from at least a second part of the vehicle, wherein the physical camera is mounted on the first part of the vehicle and the virtual camera remains static with regards to the second part of the vehicle.