METHOD AND APPARTUS FOR INFORMING PEDETRIAN AND OCCUPANT OF VEHICLE OF EACH OTHER'S APPROACH

• US 20180126902A1
• Filed: 07/11/2017
• Published: 05/10/2018
• Est. Priority Date: 11/09/2016
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: SEO, YoungWan, CHO, Baek Hwan, JEON, Paul Barom
• Granted Patent: US 10,604,070 B2
• Time in Patent Office: Days
• CPC Class Codes: B60Q 1/525 automatically indicating risk of collision between vehicles in traffic or with pedestrians, e.g. after risk assessment using the vehicle sensor data
• CPC Class Codes: B60Q 5/006 indicating risk of collision between vehicles or with pedestrians
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/307 virtually distinguishing relevant parts of a scene from the background of the scene

Abstract

Disclosed is a method and apparatus of detecting an object approaching a vehicle, the method includes a user from the image, extracting feature points associated with the device or a portion of the user'"'"'s body from the image of the user, detecting the object using a device based on the feature point, and issuing an alert of the approaching object.

Claims

1. A method of informing of an approaching vehicle, the method comprising:

detecting an object using a device from an image; and

informing the object that the vehicle is approaching.

2. The method of claim 1, wherein the detecting of the object comprises:

detecting a user from the image;

extracting a feature point associated with the device from the image of the user;

determining whether the user is using the device based on the extracted feature point; and

detecting the user as the object, in response to determining that the user is using the device.

3. The method of claim 2, wherein the extracting of the feature point comprises any one or any combination of:

extracting a static feature point comprising an exterior feature of the user; and

extracting dynamic feature points comprising any one or any combination of features of consecutive motions of the user and a feature of a distinctive movement in an ambient environment around the user.

4. The method of claim 3, wherein the extracting of the static feature point comprises:

extracting the static feature point comprising the exterior feature of the user using a first classifier trained in advance.

5. The method of claim 3, wherein the extracting of the dynamic feature points comprises:

extracting the dynamic feature points using at least one of an optical flow-based object tracking method or a second classifier trained in advance.

6. The method of claim 3, wherein the determining of whether the user is using the device comprises:

determining whether the user is using the device based on at least one of the static feature point or the dynamic feature points.

7. The method of claim 1, further comprising:

detecting a position of a portion of a body of the object.

8. The method of claim 7, wherein the detecting of the object comprises:

detecting a user from the image; and

extracting at least one of a feature point associated with the device or a feature point associated with the portion of the body of the object from the image of the user.

9. The method of claim 7, wherein the portion of the body of the object comprises at least one of a head, an ear, or a hand of the object.

10. The method of claim 1, wherein the informing of the object that the vehicle is approaching comprises any one or any combination of:

informing the object that the vehicle is approaching by providing auditory information using a directional speaker;

informing the object that the vehicle is approaching by providing visual information using a headlight of the vehicle; and

informing the device of the object that the vehicle is approaching through wireless communication between the device and the vehicle.

11. The method of claim 1, further comprising:

Informing an occupant of the vehicle that the object is approaching, in response to a determination that the object is using the device.

12. The method of claim 11, wherein the informing of the occupant of the vehicle that the object is approaching comprises:

informing the passenger in the vehicle that the object is approaching based on any one or any combination of auditory information using an audio apparatus in the vehicle, visual information on a display in the vehicle, tactile information delivered by a steering of the vehicle, and a vibration of a seat.

13. The method of claim 1, wherein the detecting of the object comprises:

detecting the object currently using the device from the image using a classifier trained based on images of users using the device.

14. The method of claim 1, wherein the device comprises at least one of a headset, an earphone, cell phone, smart phone, mobile device, music player, or a portable terminal.

15. The method of claim 2, wherein the user comprises any one of a pedestrian, a wheelchair rider, or a bicycle rider.

16. The method of claim 3, wherein the static feature point comprises any one or any combination of points indicating an angle of a head or a neck pointing downward, a gaze of the user pointing downwards, an angle of a hand pointing upward, a shape of a hand manipulating the device, and points on an ear of the user wearing a headphone.

17. The method of claim 3, wherein the dynamic feature points comprises any one or any combination of the user shaking a body, dancing motions of the user, the user walking at a different pace when compared to other in the ambient environment, and the user moving with a different gesture when compared to the others in the ambient environment.

18. The method of claim 3, wherein the static feature point and the dynamic feature points are extracted simultaneously.

19. The method of claim 3, wherein the static feature point and the dynamic feature points are extracted separately.

20. A non-transitory computer-readable medium storing instructions that, when executed by a processor, causes the processor to perform the method of claim 1.

21. An apparatus for informing an occupant of a vehicle of an approaching object, the apparatus comprising:

a processor configured to

detect an object using a device from an image, and

inform the occupant that the object is approaching.

22. The apparatus of claim 21, wherein the processor is further configured to detect a user from the image, to determine whether the user is using the user based on a feature point associated with the device extracted from the image of the user, and to detect the user as the object, in response to determining that the user is using the device.

23. The apparatus of claim 22,

wherein the processor is further configured to:

extract a static feature point comprising an exterior feature of the user using a first classifier trained in advance; and

extract dynamic feature points comprising any one or any combination of features of consecutive motions of the user and a feature of a distinctive movement in an ambient environment around the user using at least one of an optical flow-based object tracking method or a second classifier trained in advance.

24. The apparatus of claim 21, further comprising:

a vision sensor configured to capture the image.

25. The apparatus of claim 21, wherein the processor is further configured to detect a user from the image, to detect a position of a body portion of the user, and to extract at least one of a feature point associated with the device or a feature point associated with the body portion of the user from the image.

26. The apparatus of claim 21, wherein the processor is further configured to inform the occupant in the vehicle that the object is approaching based on any one or any combination of auditory information using an audio apparatus in the vehicle, visual information on a display in the vehicle, tactile information delivered by a steering of the vehicle, or a vibration of a seat.

27. The apparatus of claim 22, wherein the processor is further configured to inform the object that the vehicle is approaching, in response to the determination that the object is using the device.

28. The apparatus of claim 27, further comprising:

a directional speaker configured to face a direction of the object,

wherein the processor is further configured to inform the object that the vehicle is approaching by providing auditory information using the directional speaker.

29. The apparatus of claim 22, further comprising:

a communication interface configured to perform wireless communication between the vehicle and the device,

wherein the processor is further configured to inform the device used by the object that the vehicle is approaching through the wireless communication.

30. An apparatus for detecting an object approaching a vehicle, the apparatus comprising:

a sensor configured to capture an image;

a processor configured to detect a user from the image, to extract a feature point associated with the device or a feature point associated with a portion of the user'"'"'s body from the image of the user, to detect the object using a device based on the feature point, and to issue an alert of the approaching object; and

a communicator configured to output the alert.