Augmented reality typography personalization system

• US 10,387,730 B1
• Filed: 04/20/2017
• Issued: 08/20/2019
• Est. Priority Date: 04/20/2017
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Cowburn, Piers, Pan, Qi, Pilipski, Eitan
• Primary Examiner(s): Yeh, Euengnan
• Application Number:
  US15/492,089
• Time in Patent Office: 852 Days
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

Disclosed are augmented reality (AR) personalization systems to enable a user to edit and personalize presentations of real-world typography in real-time. The AR personalization system captures an image depicting a physical location via a camera coupled to a client device. For example, the client device may include a mobile device that includes a camera configured to record and display images (e.g., photos, videos) in real-time. The AR personalization system causes display of the image at the client device, and scans the image to detect occurrences of typography within the image (e.g., signs, billboards, posters, graffiti).

Claims

1. A method comprising:

capturing an image at a client device, the image depicting a physical location;

analyzing the image depicting the physical location to identify an occurrence of typography at a location within the image depicting the physical location, the occurrence of the typography comprising at least a text string, and a background pattern;

detecting properties of the text string, the properties of the text string including a character-set, a length, and a height;

identifying a first character and a second character that make up the character-set, the first character at a first position in the text string, and the second character at a second position in the text string;

receiving a request to remove the first character from the text string;

generating an updated text string based on the request to remove the first character from the text string, the updated text string including at least the second character; and

generating a presentation of the image at the client device, the presentation of the image including a display of the updated text string at the location of the occurrence of typography within the image depicting the physical location, the display of the updated text string comprising the background pattern at the first position of the text string previously occupied by the first character, and the second character at the second position of the text string.

2. The method of claim 1, wherein the properties of the text string further include a typeface, and the method further comprises:

receiving a request to add a third character to the text string at a third position in the text string; and

updating the display of the updated text string in the presentation of the image at the client device to include the third character at the third position.

3. The method of claim 1, wherein the properties of the text string further include a typeface, and the generating the presentation of the image at the client device includes:

comparing the text string against a catalogue of characters based on the typeface of the text string;

identifying a distortion of the text string based on the comparing;

determining a shape of a surface of the text string based on the distortion; and

wherein the generating the presentation of the image is based on the shape of the surface.

4. The method of claim 1, wherein the client device is a first client device, and the method further comprises:

retrieving location data of the physical location depicted by the image;

geo-tagging the request to remove the first character from the text string based on the location data;

detecting a second client device at the physical location;

causing display of the presentation of the image at the second client device in response to the detecting the second client device at the physical location, wherein the presentation of the image includes the background pattern at the first position of the text string in the presentation of the image, and the second character at the second position in the text string of the presentation of the image.

5. The method of claim 1, wherein the character-set is a first character-set, and the method further comprises:

receiving a request to replace the first character-set with a second character-set;

removing the first character-set from the presentation of the image in response to the receiving the request to replace the first character-set with the second character-set; and

updating the presentation of the image to include the second character-set.

6. The method of claim 1, wherein the text string is a first text string, the character-set is a first character-set, and the receiving the request to remove the first character from the text string includes:

detecting the client device at the physical location;

accessing a user profile associated with a user of the client device in response to the detecting the client device at the physical location;

retrieving a second character-set from the user profile of the user, the second character-set including a username of the user;

generating a second text string based on the second character set and the properties of the first text string; and

wherein the updating the presentation of the image includes replacing the first text string at the location within the image with the second text string.

7. The method of claim 1, wherein the method further comprises:

overlaying a text field over the text string within the portion of the presentation, the text field having dimensions based on at least the properties of the text string; and

wherein the receiving the request to remove the first character from the text string includes receiving a user input to delete the first character from the text string into the text field.

8. The method of claim 1, wherein the client device is a first client device, the character-set is a first character-set, and the method further comprises:

receiving a second character-set from a second client device, the second character-set to replace the first character-set in the presentation of the image;

assigning the second character-set location coordinates of the physical location; and

wherein the causing display of the presentation of the image at the client device includes;

detecting the first client device at the physical location;

retrieving the second character-set in response to the detecting the first client device at the physical location; and

causing display of the presentation of the image at the first client device, wherein the presentation of the image includes the second character-set.

9. The method of claim 1, wherein the client device is a first client device, and the method further comprises:

generating a message that includes the updated presentation of the image; and

transmitting the message to a second client device.

10. The method of claim 1, wherein the image is a first image, the physical location is a first physical location, the client device is a first client device, and the method further comprises:

assigning the request to remove the first character from the text string to the text string;

capturing a second image at a second client device, the second image depicting a second physical location, wherein the second physical location includes an occurrence of the text string;

identifying the text string within the second image depicting the second physical location in response to the capturing the second image;

retrieving the request to remove the first character from the text string in response to the identifying the text string; and

causing display of a presentation of the second image at the second client device based on the request to remove the first character from the text string.

11. A system comprising:

a memory; and

at least one hardware processor couple to the memory and comprising instructions that cause the system to perform operations comprising;

capturing an image at a client device, the image depicting a physical location;

analyzing the image depicting the physical location to identify an occurrence of typography at a location within the image depicting the physical location, the occurrence of the typography comprising at least a text string, and a background pattern;

detecting properties of the text string, the properties of the text string including a character-set, a length, and a height;

identifying a first character and a second character that make up the character-set, the first character at a first position in the text string, and the second character at a second position in the text string;

receiving a request to remove the first character from the text string;

generating an updated text string based on the request to remove the first character from the text string, the updated text string including at least the second character; and

generating a presentation of the image at the client device, the presentation of the image including a display of the updated text string at the location of the occurrence of typography within the image depicting the physical location, the display of the updated text string comprising the background pattern at the first position of the text string previously occupied by the first character, and the second character at the second position of the text string.

12. The system of claim 11, wherein the properties of the text string further include a typeface, and the instructions cause the system to perform operations further comprising:

receiving a request to add a third character to the text string at a third position in the text string; and

updating the display of the updated text string in the presentation of the image at the client device to include the third character at the third position.

13. The system of claim 11, wherein the client device is a first client device, and the instructions cause the system to perform operations further comprising:

retrieving location data of the physical location depicted by the image;

geo-tagging the request to remove the first character from the text string based on the location data;

detecting a second client device at the physical location;

causing display of the presentation of the image at the second client device in response to the detecting the second client device at the physical location, wherein the presentation of the image includes the background pattern at the first position of the text string and the second character at the second position of the text string.

14. The system of claim 11, wherein the character-set is a first character-set, and the instructions cause the system to perform operations further comprising:

receiving a request to replace the first character-set with a second character-set;

removing the first character-set from the presentation of the image in response to the receiving the request to replace the first character-set with the second character-set; and

updating the presentation of the image to include the second character-set.

15. The system of claim 11, wherein the text string is a first text string, the character-set is a first character-set, and the receiving the request to remove the first character from the text string includes:

detecting the client device at the physical location;

accessing a user profile associated with a user of the client device in response to the detecting the client device at the physical location;

retrieving a second character-set from the user profile of the user, the second character-set including a username of the user;

generating a second text string based on the second character set and the properties of the first text string; and

wherein the updating the presentation of the image includes replacing the first text string with the second text string.

16. The system of claim 11, wherein the instructions cause the system to perform operations further comprising:

overlaying a text field over the text string within the portion of the presentation, the text field having dimensions based on at least the properties of the text string; and

wherein the receiving the request to remove the first character from the text string includes receiving a user input to delete the first character from the text string into the text field.

17. The system of claim 11, wherein the client device is a first client device, the character-set is a first character-set, and the instructions cause the system to perform operations further comprising:

receiving a second character-set from a second client device, the second character-set to replace the first character-set in the presentation of the image;

assigning the second character-set location coordinates of the physical location; and

wherein the causing display of the presentation of the image at the client device includes;

detecting the first client device at the physical location;

retrieving the second character-set in response to the detecting the first client device at the physical location; and

causing display of the presentation of the image at the first client device, wherein the presentation of the image includes the second character-set.

18. The system of claim 11, wherein the client device is a first client device, and the instructions cause the system to perform operations further comprising:

generating a message that includes the updated presentation of the image; and

transmitting the message to a second client device.

19. The system of claim 11, wherein the image is a first image, the physical location is a first physical location, the client device is a first client device, and the instructions cause the system to perform operations further comprising:

assigning the request to remove the first character from the text string to the text string;

capturing a second image at a second client device, the second image depicting a second physical location, wherein the second physical location includes an occurrence of the text string;

identifying the text string within the second image depicting the second physical location in response to the capturing the second image;

retrieving the request to remove the first character from the text string in response to the identifying the text string; and

causing display of a presentation of the second image at the second client device based on the request to remove the first character from the text string.

20. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations including:

capturing an image at a client device, the image depicting a physical location;

analyzing the image depicting the physical location to identify an occurrence of typography at a location within the image depicting the physical location the occurrence of the typography comprising at least a text string, and a background pattern;

detecting properties of the text string, the properties of the text string including a character-set, a length, and a height;

identifying a first character and a second character that make up the character-set, the first character at a first position in the text string, and the second character at a second position in the text string;

receiving a request to remove the first character from the text string;

generating an updated text string based on the request to remove the first character from the text string, the updated text string including at least the second character; and

generating a presentation of the image at the client device, the presentation of the image including a display of the updated text string at the location of the occurrence of typography within the image depicting the physical location, the display of the updated text string comprising the background pattern at the first position of the text string previously occupied by the first character, and the second character at the second position of the text string.