User location determination based on augmented reality

• US 10,552,681 B2
• Filed: 03/07/2019
• Issued: 02/04/2020
• Est. Priority Date: 10/26/2016
• Status: Active Grant

• Current Assignee: Advanced New Technologies Company Limited (Ant Group Co., Ltd.)
• Original Assignee: Alibaba Group Holding Ltd.
• Inventors: Zeng, Xiaodong
• Primary Examiner(s): Aziz, Adnan
• Application Number:
  US16/295,840
• Publication Number:
  US 20190205648A1
• Time in Patent Office: 334 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds

Abstract

An augmented reality (AR) client terminal identifies initial location information for a location of a user. A scene picture associated with the location of the user is captured. Image matching is performed between the captured scene picture and scene pictures in a scene picture set. The scene picture set is generated based on the initial location information. Each scene picture in the scene picture set is associated with detailed location information. In response to the captured scene picture matching a particular scene picture in the scene picture set, refined location information for the location of the user is determined based on the detailed location information of the particular scene picture.

Claims

1. A computer-implemented method, comprising:

capturing a scene picture viewed by a user at a location indicated by initial location information;

determining a threshold distance from the location indicated by the initial location;

determining that the threshold distance is smaller than a positioning error associated with a system used to indicate the initial location information;

in response to determining that the threshold distance is smaller than the positioning error, adjusting the threshold distance;

performing image matching between the captured scene picture and scene pictures of a scene picture set, each scene picture of the scene picture set being associated with detailed location information that satisfies the threshold distance;

determining, in response to the image matching, refined location information of the user based upon the detailed location information of a particular one of the scene pictures of the scene picture set;

generating virtual data associated with a refined location indicated by the refined location information; and

overlaying the virtual data with a real-time scene picture within a field of view of the user at the vantage point of the user at the refined location.

2. The computer-implemented method of claim 1, wherein the initial location information includes a street-level location of a multi-floor building and the refined location information includes a particular floor location of the multi-floor building.

3. The computer-implemented method of claim 1, further comprising:

transmitting the refined location information to a mobile device in proximity with the location of the user; and

sharing the refined location information with one or more electronic devices associated with the mobile device.

4. The computer-implemented method of claim 1, wherein the detailed location information comprises location information of a particular floor of a multi-floored building.

5. The computer-implemented method of claim 1, wherein performing the image matching between the captured scene picture and the scene pictures of a scene picture set comprises image recognition identifying image contents of the captured scene picture and image contents of the scene pictures of a scene picture set.

6. The computer-implemented method of claim 1, further comprising:

determining, in response to the image matching, elevation information associated with the refined location information based upon the detailed location information of the particular one of the scene pictures of the scene picture set.

7. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising:

capturing a scene picture viewed by a user at a location indicated by initial location information;

determining a threshold distance from the location indicated by the initial location;

determining that the threshold distance is smaller than a positioning error associated with a system used to indicate the initial location information;

in response to determining that the threshold distance is smaller than the positioning error, adjusting the threshold distance;

performing image matching between the captured scene picture and scene pictures of a scene picture set, each scene picture of the scene picture set being associated with detailed location information that satisfies the threshold distance;

determining, in response to the image matching, refined location information of the user based upon the detailed location information of a particular one of the scene pictures of the scene picture set;

generating virtual data associated with a refined location indicated by the refined location information; and

overlaying the virtual data with a real-time scene picture within a field of view of the user at the vantage point of the user at the refined location.

8. The non-transitory, computer-readable medium of claim 7, wherein the initial location information includes a street-level location of a multi-floor building and the refined location information includes a particular floor location of the multi-floor building.

9. The non-transitory, computer-readable medium of claim 7, wherein the operations further comprise:

transmitting the refined location information to a mobile device in proximity with the location of the user; and

sharing the refined location information with one or more electronic devices associated with the mobile device.

10. The non-transitory, computer-readable medium of claim 7, wherein the detailed location information comprises location information of a particular floor of a multi-floored building.

11. The non-transitory, computer-readable medium of claim 7, wherein performing the image matching between the captured scene picture and the scene pictures of a scene picture set comprises an operation for image recognition to identify image contents of the captured scene picture and image contents of the scene pictures of a scene picture set.

12. The non-transitory, computer-readable medium of claim 7, wherein the operations further comprise:

determining, in response to the image matching, elevation information associated with the refined location information based upon the detailed location information of the particular one of the scene pictures of the scene picture set.

13. A computer-implemented system, comprising:

one or more computers; and

one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations comprising;

capturing a scene picture viewed by a user at a location indicated by initial location information;

determining a threshold distance from the location indicated by the initial location;

determining that the threshold distance is smaller than a positioning error associated with a system used to indicate the initial location information;

in response to determining that the threshold distance is smaller than the positioning error, adjusting the threshold distance;

performing image matching between the captured scene picture and scene pictures of a scene picture set, each scene picture of the scene picture set being associated with detailed location information that satisfies the threshold distance;

determining, in response to the image matching, refined location information of the user based upon the detailed location information of a particular one of the scene pictures of the scene picture set;

generating virtual data associated with a refined location indicated by the refined location information; and

overlaying the virtual data with a real-time scene picture within a field of view of the user at the vantage point of the user at the refined location.

14. The computer-implemented system of claim 13, wherein the initial location information includes a street-level location of a multi-floor building and the refined location information includes a particular floor location of the multi-floor building.

15. The computer-implemented system of claim 13, wherein the one or more operations further comprise:

transmitting the refined location information to a mobile device in proximity with the location of the user; and

sharing the refined location information with one or more electronic devices associated with the mobile device.

16. The computer-implemented system of claim 13, wherein performing the image matching between the captured scene picture and the scene pictures of a scene picture set comprises one or more operations to perform image recognition identifying image contents of the captured scene picture and image contents of the scene pictures of a scene picture set.

17. The computer-implemented system of claim 13, wherein the one or more operations further comprise:

determining, in response to the image matching, elevation information associated with the refined location information based upon the detailed location information of the particular one of the scene pictures of the scene picture set.