Coded vision system

• US 10,242,477 B1
• Filed: 08/02/2017
• Issued: 03/26/2019
• Est. Priority Date: 01/16/2017
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Charlton, Ebony James, Cansizoglu, Omer, Ouimet, Kirk, Boyd, Nathan Kenneth
• Primary Examiner(s): Yang, Ryan R
• Application Number:
  US15/667,345
• Time in Patent Office: 601 Days
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/352 involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
• CPC Class Codes: A63F 13/58 by computing conditions of game characters, e.g. stamina, strength, motivation or energy level
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/79 involving player-related data, e.g. identities, accounts, preferences or play histories
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/52 for supporting social networking services
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal

Abstract

A system and method for presentation of computer vision (e.g., augmented reality, virtual reality) using user data and a user code is disclosed. A client device can detect an image feature (e.g., scannable code) in one or more images. The image feature is determined to be linked to a user account. User data from the user account can then be used to generate one or more augmented reality display elements that can be anchored to the image feature in the one or more images.

Claims

1. A method comprising:

receiving a user action through a display device displaying one or more images comprising an image feature, the display device being a display of a client device of a first user that has a first user account on a network platform, the image feature being an identifier of a second user account on the network platform, the second user account belonging to a second user;

determining whether the second user account on the network platform has avatar data of the second user;

in response to determining that the second user account has avatar data from the second user, receiving, from a server of the network platform, a response comprising the avatar data of the second user;

generating a combined avatar that displays the avatar data of the second user with avatar data of the first user, the combined avatar generated from a combined avatar template that arranges the avatar data of the first user and the avatar data of the second user into a specified configuration; and

displaying, on the display device of the client device, the combined avatar on the one or more images.

2. The method of claim 1, wherein the combined avatar is displayed at a fixed position in the one or more images with respect to the image feature by tracking the image feature in the one or more images.

3. The method of claim 1, wherein the combined avatar is displayed with a user interface button configured to link the first user to the second user on a social media graph of the network platform.

4. The method of claim 1, further comprising:

determining that the image feature is no longer in the one or more images; and

in response to determining that the image feature is no longer in the one or more images, removing the combined avatar from the display device of the client device.

5. The method of claim 1, wherein determining whether the second user account on the network platform has avatar data of the second user comprises:

generating a code from the image feature; and

transmitting the code to the server of the network platform.

6. The method of claim 1, wherein the response comprises lens metadata configured to modify the one or more images using one or more visual effects.

7. The method of claim 6, further comprising:

applying the one or more visual effects to the one or more images using the lens metadata to generate one or more modified images; and

displaying the one or more modified images on the display device of the client device.

8. The method of claim 7, wherein applying the one or more visual effects to the one or more images comprises:

detecting a display of a face of a user in the one or more images; and

generating the one or more modified images by modifying the display of the face using the lens metadata.

9. The method of claim 1, wherein features in the one or more images are processed in a scan priority with the image feature ranked higher in priority than a display of a face.

10. A system comprising:

one or more processors of a machine; and

a memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising;

receive a user action through a display device displaying one or more images comprising an image feature, the display device being a display of a client device of a first user that has a first user account on a network platform, the image feature being an identifier of a second user account on the network platform, the second user account belonging to a second user;

determine whether the second user account on the network platform has avatar data of the second user;

in response to a determination that the second user account has avatar data from the second user, receive, from a server of the network platform, a response comprising the avatar data of the second user;

generate a combined avatar that displays the avatar data of the second user with avatar data of the first user, the combined avatar generated from a combined avatar template that arranges the avatar data of the first user and the avatar data of the second user into a specified configuration; and

display, on the display device of the client device, the combined avatar on the one or more images.

11. The system of claim 10, wherein the combined avatar is displayed at a fixed position in the one or more images with respect to the image feature by tracking the image feature in the one or more images.

12. The system of claim 10, wherein the combined avatar template is stored locally on the client device.

13. The system of claim 10, the operations further comprising:

determining that the image feature is no longer in the one or more images; and

in response to determining that the image feature is no longer in the one or more images, removing the combined avatar from the display device of the client device.

14. The system of claim 10, wherein determining whether the second user account on the network platform has avatar data of the second user includes operations comprising:

generate a code from the image feature; and

transmit the code to the server of the network platform.

15. The system of claim 10, wherein the response comprises lens metadata configured to modify the one or more images using one or more visual effects.

16. The system of claim 15, the operations further comprising:

apply the one or more visual effects to the one or more images using the lens metadata to generate one or more modified images; and

display the one or more modified images on the display device of the client device.

17. The system of claim 16, wherein applying the one or more visual effects to the one or more images includes operations comprising:

detect a display of a face of a user in the one or more images; and

generate the one or more modified images by modifying the display of the face using the lens metadata.

18. The system of claim 10, wherein features in the one or more images are processed in a scan priority with the image feature ranked higher in priority than a display of a face.

19. A machine-readable storage medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:

receive a user action through a display device displaying one or more images comprising an image feature, the display device being a display of a client device of a first user that has a first user account on a network platform, the image feature being an identifier of a second user account on the network platform, the second user account belonging to a second user;

determine whether the second user account on the network platform has avatar data of the second user;

in response to a determination that the second user account has avatar data from the second user, receive, from a server of the network platform, a response comprising the avatar data of the second user;

generate a combined avatar that displays the avatar data of the second user with avatar data of the first user, the combined avatar generated from a combined avatar template that arranges the avatar data of the first user and the avatar data of the second user into a specified configuration; and

display, on the display device of the client device, the combined avatar on the one or more images.

20. The machine-readable storage medium of claim 19, wherein the combined avatar is displayed at a fixed position in the one or more images with respect to the image feature by tracking the image feature in the one or more images.