Coded vision system

• US 10,559,107 B1
• Filed: 02/15/2019
• Issued: 02/11/2020
• Est. Priority Date: 01/16/2017
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Charlton, Ebony James, Cansizoglu, Omer, Ouimet, Kirk, Boyd, Nathan Kenneth
• Primary Examiner(s): Yang, Ryan R
• Application Number:
  US16/277,684
• Time in Patent Office: 361 Days
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/352 involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
• CPC Class Codes: A63F 13/58 by computing conditions of game characters, e.g. stamina, strength, motivation or energy level
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/79 involving player-related data, e.g. identities, accounts, preferences or play histories
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/52 for supporting social networking services
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal

Abstract

A system and method for presentation of computer vision (e.g., augmented reality, virtual reality) using user data and a user code is disclosed. A client device can detect an image feature (e.g., scannable code) in one or more images. The image feature is determined to be linked to a user account. User data from the user account can then be used to generate one or more augmented reality display elements that can be anchored to the image feature in the one or more images.

Claims

1. A method comprising:

generating, on a first client device of a first user, an electronic message for transmission to a second client device of a second user, the first user having a first user account on a network platform and the second user having a second user account on the network platform;

determining whether the second user account on the network platform has avatar data of the second user;

in response to determining that the second user account has avatar data of the second user, receiving, from a server of the network platform, a response comprising the avatar data of the second user;

generating a combined avatar that displays the avatar data of the second user with avatar data of the first user, the combined avatar generated from a combined avatar template that arranges the avatar data of the first user and the avatar data of the second user into a specified configuration;

displaying the combined avatar on a display device of the first client device; and

transmitting, to the second client device, the electronic message including the generated combined avatar.

2. The method of claim 1, wherein the combined avatar is an animation.

3. The method of claim 1, wherein the electronic message is an ephemeral message accessible through the network platform.

4. The method of claim 3, wherein the ephemeral message automatically expires after a specified time duration such that the second client device cannot access the ephemeral message through the network platform after expiration.

5. The method of claim 1, further comprising:

generating one or more images using an image sensor of the first client device;

displaying the one or more images on the display device of the first client device; and

receiving a user action on the one or more images, wherein determining whether the second user account has avatar data is in response to the user action on the one or more images on the first client device.

6. The method of claim 5, wherein the one or more images displays an image feature that is an identifier of the second user account on the network platform, and wherein the user action is a selection of the identifier in the one or more images.

7. The method of claim 6, wherein determining whether the second user account on the network platform has avatar data of the second user comprises:

generating a code from the image feature; and

transmitting the code to the server of the network platform.

8. The method of claim 5, further comprising:

displaying, on the display device, the combined avatar on the one or more images.

9. The method of claim 8, wherein the electronic message comprises a presentation of the combined avatar overlaid on the one or more images.

10. The method of claim 5, wherein the response comprises lens metadata configured to modify the one or more images using one or more visual effects.

11. The method of claim 10, further comprising:

applying the one or more visual effects to the one or more images using the lens metadata to generate one or more modified images; and

displaying the one or more modified images on the display device of the first client device.

12. The method of claim 11, wherein applying the one or more visual effects to the one or more images comprises:

detecting a display of a face of a user in the one or more images; and

generating the one or more modified images by modifying the display of the face using the lens metadata.

13. A client device comprising:

one or more processors of a machine; and

a display device;

a memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising;

generating, on the client device of a first user, an electronic message for transmission to another client device of a second user, the first user having a first user account on a network platform and the second user having a second user account on the network platform;

determining whether the second user account on the network platform has avatar data of the second user;

in response to determining that the second user account has avatar data from the second user, receiving, from a server of the network platform, a response comprising the avatar data of the second user;

generating a combined avatar that displays the avatar data of the second user with avatar data of the first user, the combined avatar generated from a combined avatar template that arranges the avatar data of the first user and the avatar data of the second user into a specified configuration;

displaying the combined avatar on the display device of the client device; and

transmitting, to the another client device, the electronic message with the generated combined avatar.

14. The client device of claim 13, wherein the combined avatar is an animation.

15. The client device of claim 13, wherein the electronic message is an ephemeral message accessible through the network platform.

16. The client device of claim 15, wherein the ephemeral message automatically expires after a specified time duration such that the another client device cannot access the ephemeral message through the network platform after expiration.

17. The client device of claim 13, the operations further comprising:

generating one or more images using an image sensor of the client device;

displaying the one or more images on the display device of the client device; and

receiving a user action on the one or more images, wherein determining whether the second user account has avatar data is in response to the user action on the one or more images.

18. A machine-readable storage device embodying instructions that, when executed by a client device, cause the client device to perform operations comprising:

generating, on the client device of a first user, an electronic message for transmission to another client device of a second user, the first user having a first user account on a network platform and the second user having a second user account on the network platform;

determining whether the second user account on the network platform has avatar data of the second user;

in response to determining that the second user account has avatar data from the second user, receiving, from a server of the network platform, a response comprising the avatar data of the second user;

generating a combined avatar that displays the avatar data of the second user with avatar data of the first user, the combined avatar generated from a combined avatar template that arranges the avatar data of the first user and the avatar data of the second user into a specified configuration;

displaying the combined avatar on a display device of the client device; and

transmitting, to the another client device, the electronic message with the generated combined avatar.

19. The machine-readable storage device of claim 18, wherein the combined avatar is an animation.

20. The machine-readable storage device of claim 18, the operations further comprising:

generating one or more images using an image sensor of the client device;

displaying the one or more images on the display device of the client device; and

receiving a user action on the one or more images, wherein determining whether the second user account has avatar data is in response to the user action on the one or more images on the client device.