SYSTEM AND METHOD FOR PRESENTING MEDIA CONTENTS IN AUTONOMOUS VEHICLES

• US 20170309072A1
• Filed: 04/26/2016
• Published: 10/26/2017
• Est. Priority Date: 04/26/2016
• Status: Active Grant

• Current Assignee: Baidu USA LLC (Baidu Incorporated)
• Original Assignee: Baidu USA LLC (Baidu Incorporated)
• Inventors: Li, Liyun, Liu, Shaoshan, Wu, Shuang, Peng, James
• Granted Patent: US 10,323,952 B2
• Time in Patent Office: Days
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04N 21/2143 located in a single building, e.g. hotel, hospital or museum arrangements specially adapted for plural spots in a confined site in broadcast systems H04H20/63; adaptations for transmission by electric cable for domestic distribution in television systems H04N7/106
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/816 involving special video data, e.g 3D video
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

In one embodiment, a first image of a physical object external to an autonomous vehicle is received, where the first image is captured by an image sensor attached to the autonomous vehicle. An image recognition is performed on the first image to derive one or more keywords related to the physical object. A list of one or more content items are identified based on the one or more keywords. A first content item selected from the list of content items is augmented onto the first image to generate a second image. The second image is displayed on a display device within the autonomous vehicle.

Claims

1. A computer-implemented method for presenting content in an autonomous vehicle, the method comprising:

receiving a first image of a physical object external to an autonomous vehicle, wherein the first image is captured by an image sensor attached to the autonomous vehicle;

performing an image recognition on the first image to derive one or more keywords related to the physical object;

identifying a list of one or more content items based on the one or more keywords;

augmenting a first content item selected from the list of content items onto the first image to generate a second image; and

displaying the second image on a display device within the autonomous vehicle.

2. The method of claim 1, wherein the second image includes the first image of the physical object and the first content item superimposed thereon.

3. The method of claim 1, wherein the display device is configured and positioned to appear as a virtual window of the autonomous vehicle between a user and the physical object.

4. The method of claim 3, wherein the second image is displayed in a virtual reality manner as if the user directly views the physical object through the virtual window.

5. The method of claim 4, wherein the physical object is displayed on the display device as a virtual object posed from a point of view of the user.

6. The method of claim 1, further comprising ranking the list of content items based on a user profile of the user, wherein the first content item is selected based on the rankings.

7. The method of claim 6, wherein the display device comprises a touch screen to receive user interaction with the second image, and wherein the user interaction is utilized to update the user profile.

8. The method of claim 1, further comprising:

obtaining location and route information along a route in which the autonomous vehicle travels; and

obtaining a point of interest (POI) based on the location and route information, wherein the list of content items is identified based on the POI.

9. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations of presenting content in an autonomous vehicle, the operations comprising:

receiving a first image of a physical object external to an autonomous vehicle, wherein the first image is captured by an image sensor attached to the autonomous vehicle;

performing an image recognition on the first image to derive one or more keywords related to the physical object;

identifying a list of one or more content items based on the one or more keywords;

augmenting a first content item selected from the list of content items onto the first image to generate a second image; and

displaying the second image on a display device within the autonomous vehicle.

10. The non-transitory machine-readable medium of claim 9, wherein the second image includes the first image of the physical object and the first content item superimposed thereon.

11. The non-transitory machine-readable medium of claim 9, wherein the display device is configured and positioned to appear as a virtual window of the autonomous vehicle between a user and the physical object.

12. The non-transitory machine-readable medium of claim 11, wherein the second image is displayed in a virtual reality manner as if the user directly views the physical object through the virtual window.

13. The non-transitory machine-readable medium of claim 12, wherein the physical object is displayed on the display device as a virtual object posed from a point of view of the user.

14. The non-transitory machine-readable medium of claim 9, wherein the operations further comprise ranking the list of content items based on a user profile of the user, wherein the first content item is selected based on the rankings.

15. The non-transitory machine-readable medium of claim 14, wherein the display device comprises a touch screen to receive user interaction with the second image, and wherein the user interaction is utilized to update the user profile.

16. The non-transitory machine-readable medium of claim 9, wherein the operations further comprise:

obtaining location and route information along a route in which the autonomous vehicle travels; and

obtaining a point of interest (POI) based on the location and route information, wherein the list of content items is identified based on the POI.

17. A data processing system, comprising:

a processor; and

a memory coupled to the processor for storing instructions, which when executed by the processor, cause the processor to perform operations of presenting content in an autonomous vehicle, the operations including

receiving a first image of a physical object external to the autonomous vehicle, wherein the first image is captured by an image sensor attached to the autonomous vehicle,

performing an image recognition on the first image to derive one or more keywords related to the physical object,

identifying a list of one or more content items based on the one or more keywords,

augmenting a first content item selected from the list of content items onto the first image to generate a second image, and

displaying the second image on a display device within the autonomous vehicle.

18. The system of claim 17, wherein the second image includes the first image of the physical object and the first content item superimposed thereon.

19. The system of claim 17, wherein the display device is configured and positioned to appear as a virtual window of the autonomous vehicle between a user and the physical object.

20. The system of claim 19, wherein the second image is displayed in a virtual reality manner as if the user directly views the physical object through the virtual window.

21. The system of claim 20, wherein the physical object is displayed on the display device as a virtual object posed from a point of view of the user.

22. The system of claim 17, wherein the operations further comprise ranking the list of content items based on a user profile of the user, wherein the first content item is selected based on the rankings.

23. The system of claim 22, wherein the display device comprises a touch screen to receive user interaction with the second image, and wherein the user interaction is utilized to update the user profile.

24. The system of claim 17, wherein the operations further comprise:

obtaining location and route information along a route in which the autonomous vehicle travels; and

obtaining a point of interest (POI) based on the location and route information, wherein the list of content items is identified based on the POI.