Method, apparatus and computer program product for disambiguation of points of-interest in a field of view

• US 10,552,680 B2
• Filed: 08/08/2017
• Issued: 02/04/2020
• Est. Priority Date: 08/08/2017
• Status: Active Grant

• Current Assignee: HERE Global B.V. (HERE Holding Corporation)
• Original Assignee: HERE Global B.V. (HERE Holding Corporation)
• Inventors: Hentz, Augusto Henrique, Vellasques, Eduardo, {hacek over (S)}tajner, Tadej, Kovalova, Yuliia
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US15/671,324
• Publication Number:
  US 20190050645A1
• Time in Patent Office: 910 Days
• CPC Class Codes: G01C 21/3476 using point of interest [POI] information, e.g. a route passing visible POIs
• CPC Class Codes: G01C 21/3608 using speech input, e.g. using speech recognition
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A method, apparatus, and computer program product are therefore provided for disambiguating non-deterministic spoken dialog commands as they relate to points-of-interest and information regarding points-of-interest within or near the field of view of a user. An apparatus may include a processor and a memory including program code instructions. The program code instructions are configured to, when executed, cause the apparatus to at least: receive an indication of a user location; receive an indication of a field of view of the user at the location; receive a natural language utterance; generate, from the natural language utterance, a query, where the query includes at least one request determined from the natural language utterance, the location, and the field of view of the user; determine a point-of-interest associated with the query by conducting a search of a database using the generated query; and provide information associated with the point-of-interest via a user interface.

Claims

1. An apparatus comprising at least one processor and at least one non-transitory memory including computer program code instructions, the computer program code instructions configured to, when executed, cause the apparatus to at least:

receive an indication of a user location;

receive an indication of a field of view of the user at the location, wherein the field of view of the user is established, at least in part, by a gaze tracking sensor;

receive a natural language utterance;

generate, from the natural language utterance, a query, wherein the query includes at least one request determined from the natural language utterance, the location, and the field of view of the user;

determine a point-of-interest associated with the query by conducting a search of a database using the generated query; and

provide information associated with the point-of-interest via a user interface.

2. The apparatus of claim 1, wherein the utterance comprises a request for information about a point-of-interest including one or more of:

hours of operation, a name of the point-of-interest, services provided at the point-of-interest, contact information for the point-of-interest, or user reviews of the point-of-interest.

3. The apparatus of claim 1, wherein the apparatus is further caused to:

generate a natural language sentence comprising the information associated with the point-of-interest, wherein causing the apparatus to provide information associated with the point-of-interest via a user interface comprises causing the apparatus to read aloud the natural language sentence.

4. The apparatus of claim 1, wherein the apparatus is further caused to establish a plurality of points-of-interest within the field of view of the user by causing the apparatus to:

establish, from the user location, a circle sector of the field of view, and a radius of the field of view; and

retrieve a plurality of points-of-interest having a location within the circle sector and radius of the user location.

5. The apparatus of claim 4, wherein causing the apparatus to establish a plurality of points-of-interest within the field of view further comprises causing the apparatus to:

retrieve a three-dimensional model of geometries of objects within the circle sector and radius of the user location;

determine if each of the plurality of points-of-interest having a location within the circle sector and radius of the user location are visible to the user from the user location; and

discard any points-of-interest determined to not be visible to the user.

6. The apparatus of claim 5, wherein the apparatus is further caused to rank the plurality of points-of-interest not discarded by causing the apparatus to, for each non-discarded point-of-interest:

determine a distance from the user location;

determine a position of the point-of-interest within the field of view, wherein the position comprises an angle relative to a center of the field of view;

calculate a score for the point-of-interest based, at least in part, on the distance and the position; and

rank each of the plurality of points-of-interest according to their respective scores.

7. The apparatus of claim 1, wherein causing the apparatus to determine a point-of-interest associated with the query by conducting a search of a database using the generated query comprises causing the apparatus to:

determine at least two points-of-interest associated with the query;

determine at least one distinguishing feature that disambiguates the at least two points-of-interest;

generate a natural language question based on the at least one distinguishing feature; and

determine one of the at least two points-of-interest in response to a received reply to the natural language question.

8. The apparatus of claim 7, wherein the at least one distinguishing feature comprises one or more of:

point-of-interest type, building feature, building construction, building material, position relative to a distinct location, building color, building size, business name, or business type.

9. The apparatus of claim 1, wherein causing the apparatus to determine a point-of-interest associated with the query comprises filtering a plurality of points-of-interest according to a ranking of the points-of-interest based on a determined relevance to the query, wherein relevance is established based on a location relative to the user, a position within the field of view of the user, and a relevance of the point-of-interest to the at least one request for each respective point-of-interest.

10. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions to:

receive an indication of a user location;

receive an indication of a field of view of the user at the location, wherein the field of view of the user is established, at least in part, by a gaze tracking sensor;

receive a natural language utterance;

generate, from the natural language utterance, a query, wherein the query includes at least one request determined from the natural language utterance, the location, and the field of view of the user;

determine a point-of-interest associated with the query by conducting a search of a database using the generated query; and

provide information associated with the point-of-interest via a user interface.

11. The computer program product of claim 10, wherein the utterance comprises a request for information about a point-of-interest including one or more of:

hours of operation, a name of the point-of-interest, services provided at the point-of-interest, contact information for the point-of-interest, or user reviews of the point-of-interest.

12. The computer program product of claim 10, further comprising program code instructions to:

generate a natural language sentence comprising the information associated with the point-of-interest, wherein the program code instructions to provide information associated with the point-of-interest via a user interface comprises program code instructions to read aloud the natural language sentence.

13. The computer program product of claim 10, further comprising program code instructions to establish a plurality of points-of-interest within the field of view of the user using program code instructions to:

establish, from the user location, a circle sector of the field of view, and a radius of the field of view; and

retrieve a plurality of points-of-interest having a location within the circle sector and radius of the user location.

14. The computer program product of claim 13, wherein the program code instructions to establish a plurality of points-of-interest within the field of view further comprises program code instructions to:

retrieve a three-dimensional model of geometries of objects within the circle sector and radius of the user location;

determine if each of the plurality of points-of-interest having a location within the circle sector and radius of the user location are visible to the user from the user location; and

discard any points-of-interest determined to not be visible to the user.

15. The computer program product of claim 14, further comprising program code instructions to rank the plurality of points-of-interest not discarded using program code instructions to, for each non-discarded point-of-interest:

determine a distance from the user location;

determine a position of the point-of-interest within the field of view, wherein the position comprises an angle relative to a center of the field of view;

calculate a score for the point-of-interest based, at least in part, on the distance and the position; and

rank each of the plurality of points-of-interest according to their respective scores.

16. The computer program product of claim 10, wherein the program code instructions to determine a point-of-interest associated with the query by conducting a search of a database using the generated query comprises program code instructions to:

determine at least two points-of-interest associated with the query;

determine at least one distinguishing feature that disambiguates the at least two points-of-interest;

generate a natural language question based on the at least one distinguishing feature; and

determine one of the at least two points-of-interest in response to a received reply to the natural language question.

17. The computer program product of claim 16, wherein the at least one distinguishing feature comprises one or more of:

point-of-interest type, building feature, building construction, building material, position relative to a distinct location, building color, building size, business name, or business type.

18. The computer program product of claim 10, wherein the program code instructions to determine a point-of-interest associated with the query comprises program code instructions to filter a plurality of points-of-interest according to a ranking of the points-of-interest based on a determined relevance to the query, wherein relevance is established based on a location relative to the user, a position within the field of view of the user, and a relevance of the point-of-interest to the at least one request for each respective point-of-interest.

19. A mapping system comprising:

a memory configured to store map data and point-of-interest information associated with a plurality of points-of-interest; and

processing circuitry configured to;

receive an indication of a user location;

receive an indication of a field of view of the user at the location, wherein the field of view of the user is established, at least in part, by a gaze tracking sensor;

receive a natural language utterance;

generate, from the natural language utterance, a query, wherein the query includes at least one request determined from the natural language utterance, the location, and the field of view of the user;

determine a point-of-interest associated with the query by conducting a search of the memory using the generated query; and

provide information associated with the point-of-interest via a user interface.

20. The mapping system of claim 19, wherein the processing circuitry configured to determine a point-of-interest associated with the query comprises processing circuitry configured to filter a plurality of points-of-interest according to a ranking of the points-of-interest based on a determined relevance to the query, wherein relevance is established based on a location relative to the user, a position within the field of view of the user, and a relevance of the point-of-interest to the at least one request for each respective point-of-interest.