Adapted user interface for surfacing contextual analysis of content

• US 10,402,647 B2
• Filed: 05/17/2017
• Issued: 09/03/2019
• Est. Priority Date: 05/17/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Schuh, Eric
• Primary Examiner(s): Li, Ruiping
• Application Number:
  US15/597,905
• Publication Number:
  US 20180336411A1
• Time in Patent Office: 839 Days
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules

Abstract

Non-limiting examples of the present disclosure describe surfacing of contextual suggestions for image content within an applications/service, a non-limiting example of which is a camera application. Image content is captured through a camera application executing on a computing device. A determination of user intent is generated based on an analysis of a result of execution of image recognition processing on the image content and a result of an analysis of context for data associated with the image content. As an example, processing operations related to determining user intent may be executed by an intelligent personal assistant service that is executing on the computing device. One or more contextual suggestions for the image content are generated based on the determination of the user intent. The contextual suggestions for the image content are surfaced, for example, within the camera application or an accessory device connected with the computing device, among other examples.

Claims

1. A method comprising:

accessing, via an application executing on a computing device, stored image content;

receiving a user selection of the stored image content for display;

displaying, based on the received user selection, the stored image content in a user interface of the application;

determining user intent for the stored image content based on;

executing image recognition processing on the stored image content,

analyzing context for data associated with the stored image content including executing geofencing processing for the stored image content, and

generating a determination of the user intent based on an analysis of a result of the executed image recognition processing and a result of an analysis of the context for the data associated with the stored image content;

generating one or more contextual suggestions for the stored image content based on the determination of the user intent; and

surfacing, as part of the user interface of the application, the one or more contextual suggestions along with the stored image content.

2. The method of claim 1, wherein the user intent for the stored image content is further determined based on executing processing of an audio signal that is captured, from a microphone of the computing device, proximate with the displaying of the stored image content.

3. The method of claim 1, wherein the surfacing presents the stored image content and the one or more contextual suggestions using an accessory device connected with the computing device.

4. The method of claim 3, wherein the accessory device is an augmented reality (AR) hardware component that generates a virtual representation of the one or more contextual suggestions overlaying the stored image content.

10. The method of claim 1, wherein an analysis of the context for data comprises comparing a geo-fencing result for the stored image content with a geo-fencing result of other image content, wherein the context comprises a designation for an object in the stored image content based on a result of the comparing, and wherein the one or more contextual suggestions comprise a suggested action to execute based on the designation of the object.

11. The method of claim 1, wherein an analysis of the context for data comprises comparing a geo-fencing result for the stored image content with a geo-fencing result of other image content, wherein the context comprises a determination that an object in the stored image content has changed locations, and wherein the one or more contextual suggestions comprise a suggested action to execute based on the determination that the object has changed locations.

12. The method of claim 1, wherein an analysis of the context for data comprises correlating an item in the stored image content with a search query being executed through the computing device, and wherein the one or more contextual suggestions comprise one or more of:

a suggested action to execute based on a correlation of the item with the search query, and a search result based on the correlation of the item with the search query.

13. The method of claim 1, wherein the accessing of the stored image content comprises a selection, through the computing device, of the stored image content from a user-specific distributed storage.

5. A system comprising:

at least one processor; and

a memory, operatively connected with the at least one processor, storing computer-executable instructions that, when executed by the at least one processor, causes the at least one processor to execute a method that comprises;

accessing, via an application executing on the system, stored image content;

receiving a user selection of the stored image content for display;

displaying, based on the received user selection, the stored image content in a user interface of the application;

determining user intent for the stored image content based on;

executing image recognition processing on the stored image content,

analyzing context for data associated with the stored image content including executing geofencing processing for the stored image content, and

generating a determination of the user intent based on an analysis of a result of the executed image recognition processing and a result of an analysis of the context for the data associated with the stored image content;

generating one or more contextual suggestions for the stored image content based on the determination of the user intent; and

surfacing, as part of the user interface of the application, the one or more contextual suggestions along with the stored image content.

6. The system of claim 5, wherein the user intent for the stored image content is further determined based on executing processing of an audio signal that is captured, from a microphone of the computing device, proximate with the displaying of the stored image content.

7. The system of claim 5, wherein the surfacing presents the stored image content and the one or more contextual suggestions using an accessory device connected with the computing device.

8. The system of claim 7, wherein the accessory device is an augmented reality (AR) hardware component that generates a virtual representation of the one or more contextual suggestions overlaying the stored image content.

14. The system of claim 5, wherein an analysis of the context for data comprises comparing a geo-fencing result for the stored image content with a geo-fencing result of other image content, wherein the context comprises a designation for an object in the stored image content based on a result of the comparing, and wherein the one or more contextual suggestions comprise a suggested action to execute based on the designation of the object.

15. The system of claim 5, wherein an analysis of the context for data comprises comparing a geo-fencing result for the stored image content with a geo-fencing result of other image content, wherein the context comprises a determination that an object in the stored image content has changed locations, and wherein the one or more contextual suggestions comprise a suggested action to execute based on the determination that the object has changed locations.

16. The system of claim 5, wherein an analysis of the context for data comprises correlating an item in the stored image content with a search query being executed through the computing device, and wherein the one or more contextual suggestions comprise one or more of:

a suggested action to execute based on a correlation of the item with the search query, and a search result based on the correlation of the item with the search query.

17. The system of claim 5, wherein the accessing of the stored image content comprises a selection, through the computing device, of the stored image content from a user-specific distributed storage.

9. A computer-readable medium storing computer-executable instructions that, when executed by at least one processor, causes the at least one processor to execute a method comprising:

accessing, via an application executing on a computing device, stored image content;

receiving a user selection of the stored image content for display;

displaying, based on the received user selection, the stored image content in a user interface of the application;

determining user intent for the stored image content based on;

executing image recognition processing on the stored image content,

analyzing context for data associated with the stored image content including executing geofencing processing for the stored image content, and

generating a determination of the user intent based on an analysis of a result of the executed image recognition processing and a result of an analysis of the context for the data associated with the stored image content;

generating one or more contextual suggestions for the stored image content based on the determination of the user intent; and

surfacing, as part of the user interface of the application, the one or more contextual suggestions along with the stored image content.

18. The computer-readable medium of claim 9, wherein an analysis of the context for data comprises comparing a geo-fencing result for the stored image content with a geo-fencing result of other image content, wherein the context comprises a designation for an object in the stored image content based on a result of the comparing, and wherein the one or more contextual suggestions comprise a suggested action to execute based on the designation of the object.

19. The computer-readable medium of claim 9, wherein an analysis of the context for data comprises comparing a geo-fencing result for the stored image content with a geo-fencing result of other image content, wherein the context comprises a determination that an object in the stored image content has changed locations, and wherein the one or more contextual suggestions comprise a suggested action to execute based on the determination that the object has changed locations.

20. The computer-readable medium of claim 9, wherein an analysis of the context for data comprises correlating an item in the stored image content with a search query being executed through the computing device, and wherein the one or more contextual suggestions comprise one or more of:

a suggested action to execute based on a correlation of the item with the search query, and a search result based on the correlation of the item with the search query.