Augmented reality system

• US 10,740,974 B1
• Filed: 09/15/2017
• Issued: 08/11/2020
• Est. Priority Date: 09/15/2017
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Cowburn, Piers, Pan, Qi, Sandvik, Isac Andreas Muller
• Primary Examiner(s): Sheng, Xin
• Application Number:
  US15/706,074
• Time in Patent Office: 1,061 Days
• Field of Search: None
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06T 1/0021 Image watermarking
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/18 Commands or executable codes
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal

Abstract

An augmented reality system to generate and cause display of a presentation of a space at a first client device, receive one or more selections of points within the presentation of the space at the first client device, and render graphical elements at the one or more points within the presentation of the space at the first client device. The augmented reality system is further configured to receive a display request to display the space at a second client device, and in response, may render a second presentation of the space at the second client device, wherein the second presentation of the space includes the graphical elements at the one or more points.

Claims

1. A system comprising:

a memory; and

at least one hardware processor coupled to the memory and comprising instructions that causes the system to perform operations comprising;

causing display of a first presentation of a space at a first client device;

receiving, at the first client device;

a selection of a point within the space, the selection of the point within the space comprising a user input that identifies the point within the first presentation of the space displayed at the first client device;

causing display of an interface that includes a text field at the first client device in response to the selection of the point within the space;

receiving an input that comprises a text string via the text field presented within the interface;

generating a graphical element that based on the text string in response to the input, the graphical element including a user specified duration parameter that defines a temporal period;

determining a perspective of the first client device relative to the point based on a location of the first client device;

detecting an obstruction based on the perspective of the first client device, the obstruction comprising properties; and

rendering, by one or more processors, at least a portion of the graphical element at the point within the first presentation of the space at the first client device for the temporal period defined by the duration parameter, based on the properties of the obstruction, in response to the receiving the selection of the point at the first client device.

2. The system of claim 1, wherein the instructions cause the system to perform operations further comprising:

detecting an edge of the obstruction; and

rendering the graphical element at the point based on the perspective of the first client device relative to the point, the edge of the obstruction, and the properties of the obstruction.

3. The system of claim 1, wherein the instructions cause the system to perform operations further comprising:

receiving a request to display a second presentation of the space from a second client device;

accessing location data from the second client device in response to the receiving the request to display the second presentation of the space;

determining a perspective of the second client device relative to the point based on the location data; and

causing display of the second presentation of the space that includes the graphical element at the point within the second presentation of the space at the second client device based on the perspective of the second client device relative to the point.

4. The system of claim 1, wherein the causing display of the second presentation of the space includes:

identifying the location of the first client device within the second presentation of the space; and

causing display of an indication of the location of the first client device within the second presentation of the space.

5. The system of claim 4, wherein the causing display of the indication of the location of the first client device within the space includes:

accessing a user profile associated with a user of the first client device in response to the causing display of the second presentation of the space at the second client device;

retrieving personalization data of the user from the user profile;

causing display of the indication of the location of the first client device based on the personalization data.

6. The system of claim 1, wherein the user input is a first user input, and the instructions cause the system to perform operations further comprising:

causing display of a set of graphical elements at the first client device in response to the receiving the selection of the point within the presentation of the space, the set of graphical elements including the graphical element;

receiving a second user input that selects the graphical element from among the set of graphical elements; and

rendering the graphical element at the point within the first presentation of the space in response to the receiving the second user input that selects the graphical element from among the of graphical elements.

7. The system of claim 1, wherein the graphical element is an ephemeral element having a display duration, and the instructions cause the system to perform operations further comprising:

detecting an expiration of the display duration of the ephemeral element; and

removing the graphical element from the first presentation of the space at the first client device and the second presentation of the space at the second client device, in response to the expiration of the display duration.

8. The system of claim 1, wherein the rendering, by the one or more processors, the graphical element at the point within the first presentation of the space includes:

detecting a visual obstruction in the first presentation of the space;

modifying the graphical element based on the visual obstruction; and

rendering the modified graphical element at the point within the first presentation.

9. The system of claim 1, wherein the instructions cause the system to perform operations further comprising:

embedding the graphical element with coded data;

detecting the coded data at the second client device, the detecting the coded data in response to the causing display of the second presentation of the space that includes the graphical element at the second client device;

retrieving media content associated with the coded data in response to detecting the coded data; and

presenting the media content at the second client device.

10. The system of claim 1, wherein the instructions cause the system to perform operations further comprising:

detecting an interaction with the graphical element at the second client device; and

altering the graphical element at the first presentation of the space at the first client device based on the interaction from the second client device.

11. A method comprising:

causing display of a first presentation of a space at a first client device;

receiving, at the first client device, a selection of a point within the space, the selection of the point within the space comprising a user input that identifies the point within the first presentation of the space displayed at the first client device;

causing display of an interface that includes a text field at the first client device in response to the selection of the point within the space;

receiving an input that comprises a text string via the text field presented within the interface;

generating a graphical element that based on the text string in response to the input, the graphical element including a user specified duration parameter that defines a temporal period;

determining a perspective of the first client device relative to the point based on a location of the first client device;

detecting an obstruction based on the perspective of the first client device, the obstruction comprising properties; and

rendering, by one or more processors, at least a portion of the graphical element at the point within the first presentation of the space at the first client device for the temporal period defined b the duration parameter, based on the properties of the obstruction, in response to the receiving the selection of the point at the first client device.

12. The method of claim 11;

wherein the method further comprises;

detecting an edge of the obstruction; and

rendering the graphical element at the point based on the perspective of the first client device relative to the point, the edge of the Obstruction, and the properties of the obstruction.

13. The method of claim 11, wherein the method further comprises:

receiving a request to display a second presentation of the space from a second client device;

accessing location data from the second client device in response to the receiving the request to display the second presentation of the space;

determining a perspective of the second client device relative to the point based on the location data; and

causing display of the second presentation of the space that includes the graphical element at the point within the second presentation of the space at the second client device based on the perspective of the second client device relative to the point.

14. The method of claim 11, wherein the causing display of the second presentation of the space includes:

identifying the location of the first client device within the second presentation of the space; and

causing display of an indication of the location of the first client device within the second presentation of the space.

15. The method of claim 14, wherein the causing display of the indication of the location of the first client device within the space includes:

accessing a user profile associated with a user of the first client device in response to the causing display of the second presentation of the space at the second client device;

retrieving personalization data of the user from the user profile;

causing display of the indication of the location of the first client device based on the personalization data.

16. The method of claim 11, wherein the user input is a first user input, and the method further comprises:

causing display of a set of graphical elements at the first client device in response to the receiving the selection of the point within the presentation of the space, the set of graphical elements including the graphical element;

receiving a second user input that selects the graphical element from among the set of graphical elements; and

rendering the graphical element at the point within the first presentation of the space in response to the receiving the second user input that selects the graphical element from among the set of graphical elements.

17. The method of claim 11, wherein the graphical elements an ephemeral element having a display duration, and the method further comprises:

detecting an expiration of the display duration of the ephemeral element; and

removing the graphical element from the first presentation of the space at the first client device and the second presentation of the space at the second client device, in response to the expiration of the display duration.

18. The method of claim 11, wherein the rendering, by the one or more processors, the graphical element at the point within the first presentation of the space includes:

detecting a visual obstruction in the first presentation of the space;

modifying the graphical element based on the visual obstruction; and

rendering the modified graphical element at the point within the first presentation.

19. The method of claim 11, wherein the method further comprises:

embedding the graphical element with coded data;

detecting the coded data at the second client device, the detecting the coded data in response to the causing display of the second presentation of the space that includes the graphical element at the second client device;

retrieving media content associated with the coded data in response to detecting the coded data; and

presenting the media content at the second client device.

20. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

causing display of a first presentation of a space at a first client device;

receiving, at the first client device;

a selection of a point within the space, the selection of the point within the space comprising a user input that identifies the point within the first presentation of the space displayed at the first client device;

causing display of an interface that includes a text field at the first client device in response to the selection of the point within the space;

receiving an input that comprises a text string via the text field presented within the interface;

generating a graphical element that based on the text string in response to the input, the graphical element including a user specified duration parameter that defines a temporal period;

determining a perspective of the first client device relative to the point based on a location of the first client device;

detecting an obstruction based on the perspective of the first client device, the obstruction comprising properties; and

rendering, by one or more processors, at least a portion of the graphical element at the point within the first presentation of the space at the first client device for the temporal period defined by the duration parameter based on the properties of the obstruction, in response to the receiving the selection of the point at the first client device.