Passenger-related item loss mitigation

• US 10,311,704 B1
• Filed: 03/29/2018
• Issued: 06/04/2019
• Est. Priority Date: 03/29/2018
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Futurewei Technologies Incorporated (Huawei Investment & Holding Co., Ltd.)
• Inventors: Xu, Qijie, Porikli, Fatih, Shen, Junqiang
• Primary Examiner(s): Yacob, Sisay
• Application Number:
  US15/939,984
• Time in Patent Office: 432 Days
• CPC Class Codes: B60R 2300/8006 for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 18/24143 Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 50/40 Business processes related to the transportation industry shipping G06Q10/083
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G08B 13/19647 Systems specially adapted for intrusion detection in or around a vehicle
• CPC Class Codes: G08B 21/24 Reminder alarms, e.g. anti-loss alarms
• CPC Class Codes: G08B 29/186 Fuzzy logic; neural networks
• CPC Class Codes: H04L 12/1895 for short real-time information, e.g. alarms, notifications, alerts, updates
• CPC Class Codes: H04L 51/046 Interoperability with other network applications or services
• CPC Class Codes: H04L 51/58 Message adaptation for wireless communication

Abstract

A method and device for mitigating passenger-related item loss in a travel-for-hire context are disclosed. An item within a vehicle passenger region is identified, and associated the identified item with a passenger identified by a passenger identifier. A vehicle cabin inventory database is updated with an entry based on the identified item associated with the passenger identifier. A presence of the identified item is detected within the vehicle cabin passenger region when a passenger exit event occurs, and message data is generated including the entry that relates to the identified item based on the presence of the identified item within the vehicle cabin passenger region.

Claims

1. A method of mitigating passenger-related item loss in a travel-for-hire context, the method comprising:

identifying an item within a vehicle cabin passenger region;

associating the identified item with a passenger identified by a passenger identifier;

updating a vehicle cabin inventory database with a database entry based on the identified item associated with the passenger identifier;

in response to an occurrence of a passenger exit event, detecting a presence of the identified item within the vehicle cabin passenger region; and

generating message data including the entry that relates to the identified item based on the presence of the identified item within the vehicle cabin passenger region.

2. The method of claim 1, wherein the identifying the item comprises:

capturing an image of the vehicle cabin passenger region;

identifying a region-of-interest of the image, wherein the region-of-interest is based on a detection confidence metric relating to the vehicle cabin passenger region;

upsampling the region-of-interest to produce an upsampled region-of-interest, the upsampled region-of-interest to promote image recognition; and

performing image recognition of the upsampled region-of-interest of the image to identify the item within the vehicle cabin passenger region.

3. The method of claim 2, wherein the capturing the image of the vehicle cabin passenger region occurs upon a passenger entry event.

4. The method of claim 1, wherein the identifying the item with the vehicle cabin passenger region includes verifying the identified item as a passenger-related item.

5. The method of claim 1, further comprising verbally announcing the message data via an announcement device.

6. The method of claim 1, further comprising announcing the message data.

7. The method of claim 6, wherein the announcing the message data comprises at least one of:

transmitting a text message based on the passenger identifier for receipt by a handheld mobile device;

transmitting the message data based on the passenger identifier for receipt by a client application of the handheld mobile device;

or

transmitting the message data based on the passenger identifier as a universal resource locator (URL) link for receipt by a user device.

8. The method of claim 1, wherein the passenger exit event is based on at least one of:

camera device data;

or

vehicle door sensor data.

9. A vehicle control unit for in-vehicle tracking of an item, comprising:

a memory containing instructions; and

a processor coupled to the memory, the processor configured to execute the instructions stored in the memory to cause the processor to;

identify an item within a plurality of vehicle cabin passenger regions;

associate the identified item with a passenger identified by a passenger identifier;

update a vehicle cabin inventory database with an entry including the identified item associated with the passenger identifier;

in response to an occurrence of a passenger exit event, detect a presence of the identified item within the plurality of vehicle cabin passenger regions; and

generate message data with the entry relating to the identified item based on the presence of the identified item within the plurality of vehicle cabin passenger regions.

10. The vehicle control unit of claim 9, comprising further instructions stored in memory to cause the processor to identify the item by:

capturing an image of a vehicle cabin passenger region of the plurality of vehicle cabin passenger regions;

identifying a region-of-interest of the image, wherein the region-of-interest is based on a detection confidence metric relating to the vehicle cabin passenger region of the plurality of vehicle cabin passenger regions;

upsampling the region-of-interest to produce an upsampled region-of-interest, the upsampled region-of-interest to promote image recognition; and

performing image recognition of the upsampled region-of-interest of the image to identify the item within the vehicle cabin passenger region.

11. The vehicle control unit of claim 9, wherein the identifying the item within the plurality of vehicle cabin passenger regions includes verifying the identified item as a passenger-related item.

12. The vehicle control unit of claim 9, comprising further instructions stored in memory to cause the processor to verbally announce the message data via a user interface.

13. The vehicle control unit of claim 9, comprising further instructions stored in the memory to cause the processor to transmit the message data by at least one of:

a text message based on the passenger identifier for receipt by a handheld mobile device;

a client application based on the passenger identifier for receipt by a client application of the handheld mobile device;

or

a universal resource locator (URL) link for receipt by a user device.

14. The vehicle control unit of claim 9, wherein the passenger exit event is based on at least one of:

camera device data;

or

vehicle door sensor data.

15. The vehicle control unit of claim 9, wherein the passenger identifier comprises at least one of:

a username;

an international mobile equipment identity (IMEI) number;

or

an electronic serial number (ESN).

16. A method of mitigating passenger-related item loss, the method comprising:

identifying, via image recognition, an item within vehicle cabin passenger regions as a passenger-related item;

associating the item with a passenger identified by a passenger identifier when the item is a passenger-related item;

updating a vehicle cabin inventory database entry with the passenger identifier and the passenger-related item;

detecting, in response to an occurrence of a passenger exit event, a presence of the passenger-related item within the vehicle cabin passenger regions;

generating message data based on the presence of the passenger-related item within the vehicle cabin passenger regions and based on the vehicle cabin inventory database entry; and

transmitting the message data for alerting the passenger identified by the passenger identifier.

17. The method of claim 16, wherein the identifying the item comprises:

capturing an image of the vehicle cabin passenger regions;

identifying a region-of-interest of the image, wherein the region-of-interest is based on a detection confidence metric relating to within the vehicle cabin passenger regions;

upsampling the region-of-interest to produce an upsampled region-of-interest, wherein the upsampled region-of-interest provides an enhanced focus to promote image recognition; and

performing the image recognition of the upsampled region-of-interest of the image to identify the item within the vehicle cabin passenger regions.

18. The method of claim 16, wherein the transmitting the message data comprises at least one of:

transmitting a text message based on the passenger identifier for receipt by a handheld mobile device;

transmitting the message data based on the passenger identifier for receipt by a client application of the handheld mobile device;

or

transmitting the message data based on the passenger identifier as a universal resource locator (URL) link for receipt by a user device.

19. The method of claim 16, wherein the passenger exit event is based on at least one of:

camera device data;

or

vehicle door sensor data.

20. The method of claim 16, wherein the passenger identifier comprises at least one of:

a username;

an international mobile equipment identity (IMEI) number;

or

an electronic serial number (ESN).