AUGMENTED-REALITY SHOPPING USING A NETWORKED MOBILE DEVICE

• US 20140100997A1
• Filed: 10/05/2012
• Published: 04/10/2014
• Est. Priority Date: 10/05/2012
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Mayerle, Jochen, Klein, Udo, Jacob, Clemens, Koshy, Sanu, Hoyer, Timo, Tatzel, Steffen
• Granted Patent: US 9,449,343 B2
• Time in Patent Office: Days
• US Class Current: 705/27.2
• CPC Class Codes: G06Q 30/0268 at point-of-sale [POS]
• CPC Class Codes: G06Q 30/0276 Advertisement creation
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A consumer user can perform a number of augmented reality shopping operations using a local client shopping application executable on a mobile device. Features related to implementations of the current subject matter can include, but are not limited to, the ability to view an augmented reality depiction of a shopping display using the mobile device and to execute various “e-commerce” functions accessible via the augmented reality depiction. Related methods, systems, articles of manufacture, and the like are described.

Claims

1. A computer program product comprising a machine-readable medium storing instructions that, when executed by a mobile device comprising at least one programmable processor, cause the mobile device to perform operations comprising:

determining, from an augmented reality three dimensional model of a shopping display, a two dimensional overlay comprising an augmented reality marker corresponding to a physical item visible in a two dimensional image of the shopping display captured by a camera associated with the mobile device;

displaying, via a screen on the mobile device, an augmented reality view of the shopping display, the augmented reality view comprising the two dimensional image and the two dimensional overlay presented together such that the augmented reality marker is associated with the physical item in the augmented reality view;

receiving, via a user input on the screen, a selection of the augmented reality marker; and

presenting, on the screen of the mobile device in response to the received selection, an item page corresponding to the physical item, the item page comprising additional information about at least one of the physical item and a good or service represented by the physical item.

2. A computer program product as in claim 1, wherein the operations further comprise:

downloading, from a server to the mobile device, the augmented reality three dimensional model of the shopping display.

3. A computer program product as in claim 2, wherein the operations further comprise:

transmitting information from the mobile device to the server, the information being usable by the server to identify the augmented reality three dimensional model of the shopping display from a plurality of augmented reality three dimensional models of shopping displays stored at or accessible by the server.

4. A computer program product as in claim 1, wherein the determining, the receiving, the presenting, and the presenting are performed by a local client shopping application running on the mobile device.

5. A computer program product as in claim 1, wherein the additional information comprises at least one of identification information relating to the physical item or the good or service represented by the physical item, sizing information, user interface functionality that permits selection by the consumer user of a specific size or color or shape, user interface functionality for adding the physical item or the good or service represented by the physical item to a shopping cart or watch list, user interface functionality for immediately purchasing the physical item or the good or service represented by the physical item, user interface functionality that supports sending information about the physical item or the good or service represented by the physical item to another person, user interface functionality that supports sending a request for additional information about the physical item or the good or service represented by the physical item, user interface functionality that supports providing additional views of the physical item or the good or service represented by the physical item, and user interface functionality that supports viewing of related goods or services.

6. A computer program product as in claim 1, wherein the two dimensional overlay further comprises a second augmented reality marker comprising an image representation of a good or service, the image representation not being physically present in the shopping display.

7. A system comprising:

a mobile device comprising at least one programmable processor; and

a machine-readable medium storing instructions that, when executed by the at least one processor, cause the at least one programmable processor to perform operations comprising;

determining, from an augmented reality three dimensional model of a shopping display, a two dimensional overlay comprising an augmented reality marker corresponding to a physical item visible in a two dimensional image of the shopping display captured by a camera associated with the mobile device;

displaying, via a screen on the mobile device, an augmented reality view of the shopping display, the augmented reality view comprising the two dimensional image and the two dimensional overlay presented together such that the augmented reality marker is associated with the physical item in the augmented reality view;

receiving, via a user input on the screen, a selection of the augmented reality marker; and

presenting, on the screen of the mobile device in response to the received selection, an item page corresponding to the physical item, the item page comprising additional information about at least one of the physical item and a good or service represented by the physical item.

8. A system as in claim 7, wherein the operations further comprise:

downloading, from a server to the mobile device, the augmented reality three dimensional model of the shopping display.

9. A system as in claim 8, wherein the operations further comprise:

transmitting information from the mobile device to the server, the information being usable by the server to identify the augmented reality three dimensional model of the shopping display from a plurality of augmented reality three dimensional models of shopping displays stored at or accessible by the server.

10. A system as in claim 7, wherein the determining, the receiving, the presenting, and the presenting are performed by a local client shopping application running on the mobile device.

11. A system as in claim 7, wherein the additional information comprises at least one of identification information relating to the physical item or the good or service represented by the physical item, sizing information, user interface functionality that permits selection by the consumer user of a specific size or color or shape, user interface functionality for adding the physical item or the good or service represented by the physical item to a shopping cart or watch list, user interface functionality for immediately purchasing the physical item or the good or service represented by the physical item, user interface functionality that supports sending information about the physical item or the good or service represented by the physical item to another person, user interface functionality that supports sending a request for additional information about the physical item or the good or service represented by the physical item, user interface functionality that supports providing additional views of the physical item or the good or service represented by the physical item, and user interface functionality that supports viewing of related goods or services.

12. A system as in claim 7, wherein the two dimensional overlay further comprises a second augmented reality marker comprising an image representation of a good or service, the image representation not being physically present in the shopping display.

13. A computer-implemented method comprising:

determining, from an augmented reality three dimensional model of a shopping display, a two dimensional overlay comprising an augmented reality marker corresponding to a physical item visible in a two dimensional image of the shopping display captured by a camera associated with a mobile device;

displaying, via a screen on a mobile device, an augmented reality view of the shopping display, the augmented reality view comprising the two dimensional image and the two dimensional overlay presented together such that the augmented reality marker is associated with the physical item in the augmented reality view;

receiving, via a user input on the screen, a selection of the augmented reality marker; and

presenting, on the screen of the mobile device in response to the received selection, an item page corresponding to the physical item, the item page comprising additional information about at least one of the physical item and a good or service represented by the physical item.

14. A computer-implemented method as in claim 13, further comprising:

downloading, from a server to the mobile device, the augmented reality three dimensional model of the shopping display.

15. A computer-implemented method as in claim 14, further comprising:

transmitting information from the mobile device to the server, the information being usable by the server to identify the augmented reality three dimensional model of the shopping display from a plurality of augmented reality three dimensional models of shopping displays stored at or accessible by the server.

16. A computer-implemented method as in claim 13, wherein the determining, the receiving, the presenting, and the presenting are performed by a local client shopping application running on the mobile device.

17. A computer-implemented method as in claim 13, wherein the additional information comprises at least one of identification information relating to the physical item or the good or service represented by the physical item, sizing information, user interface functionality that permits selection by the consumer user of a specific size or color or shape, user interface functionality for adding the physical item or the good or service represented by the physical item to a shopping cart or watch list, user interface functionality for immediately purchasing the physical item or the good or service represented by the physical item, user interface functionality that supports sending information about the physical item or the good or service represented by the physical item to another person, user interface functionality that supports sending a request for additional information about the physical item or the good or service represented by the physical item, user interface functionality that supports providing additional views of the physical item or the good or service represented by the physical item, and user interface functionality that supports viewing of related goods or services.

18. A computer-implemented method as in claim 13, wherein the two dimensional overlay further comprises a second augmented reality marker comprising an image representation of a good or service, the image representation not being physically present in the shopping display.

19. A computer-implemented method as in claim 13, wherein at least one of the determining, the displaying, the receiving, and the presenting is performed by at least one programmable processor.