Augmented reality personalization

• US 9,858,723 B2
• Filed: 12/04/2015
• Issued: 01/02/2018
• Est. Priority Date: 12/13/2011
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: HERE Global B.V. (Nokia Corporation)
• Inventors: Stroila, Matei
• Primary Examiner(s): Tseng, Charles
• Application Number:
  US14/959,620
• Publication Number:
  US 20160086384A1
• Time in Patent Office: 760 Days
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H05K 999/99 dummy group

Abstract

A method is provided, such as for mobile augmented reality personalization. A front-facing camera of the mobile device acquires a first view of a user of the mobile device. A personal characteristic of the user of the mobile device is identified from the first view. A location of the mobile device may be determined. A back-facing camera of the mobile device may acquire a second view of a region at the location. Augmented reality information is selected as a function of the personal characteristic. A second view is displayed with the augmented reality information.

Claims

1. A method comprising:

capturing, by a first camera, a first view;

selecting a face of a user in the first view;

performing an analysis of the face of the user;

determining a facial expression characteristic of the user from the analysis of the face of the user;

receiving location data associated with the user;

capturing, by a second camera, a second view;

performing a first query for the location data that returns a plurality of points of interest;

performing a second query for the facial expression characteristic of the user that selects data for a subset of the plurality of points of interest;

displaying the second view with the data for the subset of the plurality of points of interest.

2. The method of claim 1, further comprising:

sending information for the point of interest to a mobile device, wherein the information for the point of interest is combined with the second view from the second camera.

3. The method of claim 1, further comprising:

performing feature extraction on the face of the user.

4. The method of claim 1, wherein the analysis determines an identity of the user, wherein the point of interest is selected based on the identity of the user.

5. The method of claim 1, wherein the analysis determines a permanent feature of the user, wherein the point of interest is associated with the permanent feature of the user.

6. The method of claim 1, wherein the analysis determines a temporary feature of the user, wherein the point of interest is associated with the temporary feature of the user.

7. The method of claim 1, wherein the location data includes position coordinates and an orientation, wherein the point of interest is selected based on the position coordinates and the orientation.

8. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs;

the at least one memory configured to store the computer program code configured to, with the at least one processor, cause the apparatus to at least perform;

identifying a first view of a user collected by a first camera of a mobile device;

performing an analysis of the first view of the user from the first camera;

determining a facial expression characteristic of the user from the analysis of the first view;

receiving, from position circuitry, location data associated with the mobile device;

receiving a second view including a point of interest from a second camera of the mobile device;

performing a first query for the location data that returns a plurality of points of interest;

performing a second query for the facial expression characteristic of the user that selects information for a subset of the plurality of points of interest; and

providing the information with the second view.

9. The apparatus of claim 8, the at least one memory configured to store the computer program code configured to, with the at least one processor, cause the apparatus to at least perform:

sending the information for the point of interest to the mobile device, wherein the information for the point of interest is combined with the second view from the second camera of the mobile device.

10. The apparatus of claim 8, the at least one memory configured to store the computer program code configured to, with the at least one processor, cause the apparatus to at least perform:

combining the information for the point of interest with the second view from the second camera of the mobile device.

11. The apparatus of claim 8, wherein the location data includes position coordinates and an orientation, wherein the point of interest is selected based on the position coordinates and the orientation.

12. A non-transitory computer readable medium configured to store instructions, executable by the processor to perform:

receiving, at the processor, a first view of a user collected by a first camera of a mobile device;

selecting, at the processor, a face of the user in the first view;

receiving, at the processor, a second view collected by a second camera of the mobile device;

identifying, at the processor, a facial expression characteristic of the user from the face of the user, wherein the facial expression characteristic of the user does not include an identity of the user;

receiving, at the processor, location data for the mobile device;

performing, at the processor, a first query for the location data that returns a plurality of points of interest;

performing, at the processor, a second query for the facial expression characteristic of the user that selects information for a subset of the plurality of points of interest; and

providing, at the processor, the information with the second view.